CN111860489A - Certificate image correction method, device, equipment and storage medium - Google Patents
Certificate image correction method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111860489A CN111860489A CN201911252892.3A CN201911252892A CN111860489A CN 111860489 A CN111860489 A CN 111860489A CN 201911252892 A CN201911252892 A CN 201911252892A CN 111860489 A CN111860489 A CN 111860489A
- Authority
- CN
- China
- Prior art keywords
- image
- certificate
- corner
- corrected
- coordinates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000003702 image correction Methods 0.000 title claims abstract description 51
- 230000009466 transformation Effects 0.000 claims abstract description 61
- 238000012937 correction Methods 0.000 claims abstract description 54
- 239000011159 matrix material Substances 0.000 claims abstract description 42
- 238000012549 training Methods 0.000 claims abstract description 39
- 238000001514 detection method Methods 0.000 claims abstract description 15
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 13
- 238000012015 optical character recognition Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000003672 processing method Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G06T5/80—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20164—Salient point detection; Corner detection
Abstract
The application provides a certificate image correction method, a certificate image correction device, certificate image correction equipment and a storage medium, wherein the certificate image correction method comprises the following steps: acquiring coordinates of corner points to be corrected of certificate corner points in an image to be recognized by adopting a preset correction model, wherein the preset correction model is acquired by training of a sample set, the sample set comprises a plurality of sample certificate images, and each sample certificate image is marked with the corner point coordinates; calculating and acquiring a transformation matrix according to each target corner coordinate of the corrected certificate and the corresponding corner coordinate of the certificate to be corrected; the certificate in the image to be recognized is subjected to perspective transformation according to the transformation matrix to obtain a corrected certificate image, the problems that in the prior art, the correction method is greatly influenced by an image background pattern, and if other edge line information exists in the image, the edge line detection of the certificate is interfered are solved, and the effect of improving the correction effect is achieved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a certificate image correction method, apparatus, device, and storage medium.
Background
The Optical Character Recognition (OCR) technology of the certificate is a technology that can automatically recognize characters in the certificate, such as common identification cards, driving licenses, bank cards, and the like. The OCR has wide application in different fields, such as bank identity information submission, network car booking driver registration information uploading and verification and the like, and characters in the certificate can be extracted quickly and accurately by utilizing the OCR so as to avoid the problems of low manual input efficiency and high error rate.
Before the information in the certificate image is identified, the certificate image generally needs to be corrected, and in the prior art, the correction of the certificate image is generally corrected according to the inclined directions of four edge lines of the certificate.
However, this kind of correction method is greatly affected by the image background pattern, and if there is other edge line information in the image, it will interfere with the detection of the certificate edge line, resulting in the failure of correction.
Disclosure of Invention
In view of this, an object of the present application is to provide a method, an apparatus, a device, and a storage medium for certificate image correction, in which a deep learning technique is adopted to output corner coordinates and corner numbers of a certificate, the anti-interference capability is strong, a certificate at any angle can be corrected, and a better correction effect is achieved.
In a first aspect of the present application, there is provided a method of document image correction, the method comprising:
acquiring coordinates and numbers of corner points to be corrected of certificate corner points in an image to be recognized by adopting a preset correction model, wherein the preset correction model is acquired by training of a sample set, the sample set comprises a plurality of sample certificate images, and each sample certificate image is marked with the corner point coordinates;
calculating and acquiring a transformation matrix according to the target corner coordinates of each corner and the corresponding corner coordinates to be corrected;
and carrying out perspective transformation on the certificate in the image to be identified according to the transformation matrix to obtain a corrected certificate image.
Optionally, the obtaining, by using the preset correction model, coordinates of corner points to be corrected of certificate corner points in the image to be recognized includes:
and according to the preset correction model, carrying out corner detection on the certificate in the image to be recognized, and acquiring the coordinates and the serial numbers of the corner points to be corrected of a preset number of corner points.
Optionally, the calculating and obtaining a transformation matrix according to the corrected target corner coordinates and the corresponding corner coordinates to be corrected includes:
and calculating a transformation matrix according to the target corner coordinates, the expected corner numbers, the to-be-corrected corner coordinates and the corner numbers.
Optionally, the method further comprises:
acquiring a plurality of sample sets, wherein each sample set comprises a plurality of sample certificate images, and each sample certificate image is marked with corner point coordinates and corner point numbers;
and performing regression training on a preset main body network according to the sample set to obtain the preset correction model.
Optionally, the performing regression training on the preset subject network according to the sample set includes:
and performing regression training on a preset main body network according to the sample set, and outputting the corner coordinates, the corner numbers and the corner confidence coefficients of the certificate images of the samples.
Optionally, performing regression training on a preset subject network according to the sample set, including:
and randomly rotating each sample certificate image, and performing regression training on a preset main body network according to the rotated sample certificate image.
Optionally, after performing perspective transformation on the certificate in the image to be recognized according to the transformation matrix to obtain a corrected certificate image, the method further includes:
performing field detection on the corrected certificate image;
and identifying the detected field according to a preset algorithm, and outputting character information corresponding to the image to be identified.
In a second aspect of the present application, there is also provided a document image correction apparatus, the apparatus comprising: the system comprises an acquisition module, a calculation module and a perspective transformation module, wherein:
the acquisition module is used for acquiring the coordinates and the serial numbers of the corner points to be corrected of the certificate corner points in the image to be recognized by adopting a preset correction model, the preset correction model is acquired by training of a sample set, the sample set comprises a plurality of sample certificate images, and each sample certificate image is marked with the coordinates and the serial numbers of the corner points;
the calculation module is used for calculating and acquiring a transformation matrix according to the target corner point coordinates of each corner point and the corresponding corner point coordinates to be corrected;
and the perspective transformation module is used for carrying out perspective transformation on the certificate in the image to be identified according to the transformation matrix to obtain a corrected certificate image.
Optionally, the obtaining module is further configured to perform corner detection on the certificate in the image to be recognized according to the preset correction model, and obtain coordinates and serial numbers of the corners to be corrected of a preset number of corners.
Optionally, the calculation module calculates a transformation matrix according to the target corner coordinates, the expected corner number, the to-be-corrected corner coordinates and the corner number.
Optionally, the obtaining module is further configured to obtain a plurality of sample sets, where each sample set includes a plurality of sample certificate images, and each sample certificate image is marked with an angular point coordinate and an angular point number; and performing regression training on a preset main body network according to the sample set to obtain the preset correction model.
Optionally, the apparatus further comprises: training module and output module, wherein:
the training module is used for carrying out regression training on a preset subject network according to the sample set;
and the output module is used for outputting the corner point coordinates, the corner point numbers and the corner point confidence degrees of the sample certificate images.
Optionally, the training module is further configured to randomly rotate each sample certificate image, and perform regression training on a preset main body network according to the rotated sample certificate image.
Optionally, the apparatus further includes a detection module, configured to perform field detection on the corrected certificate image;
the output module is further used for identifying the detected fields according to a preset algorithm and outputting the character information corresponding to the image to be identified.
In a third aspect of the present application, there is provided a document image correction device, a processor, a storage medium and a bus, the storage medium storing machine readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the document image correction device is in operation, the processor executing the machine readable instructions to perform the steps of any of the methods of the first aspect.
In a fourth aspect of the present application, there is also provided a storage medium having stored thereon a computer program for performing the steps of the method according to any one of the above first aspects when the computer program is executed by a processor.
Based on any aspect, the corner coordinates to be corrected of the certificate corner points in the image to be recognized can be obtained according to the preset correction model, the transformation matrix corresponding to the certificate in the current image to be recognized is calculated according to the target corner coordinates of each corner point and the corresponding corner coordinates to be corrected, and the certificate in the image to be recognized is subjected to perspective transformation according to the transformation matrix to obtain the corrected certificate image, so that the problem that the correction method in the prior art is greatly influenced by the image background pattern is solved, and the effect of improving the correction effect is achieved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
FIG. 1 illustrates a schematic diagram of a document image correction system according to an embodiment of the present application;
FIG. 2 is a schematic flow chart illustrating a method for correcting a document image according to an embodiment of the present application;
FIG. 3 is a schematic flow chart diagram illustrating a method for correcting a document image according to another embodiment of the present application;
FIG. 4 is a schematic flow chart diagram illustrating a method for correcting a document image according to another embodiment of the present application;
FIG. 5 is a schematic diagram of a document image correction device according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a document image correction device according to another embodiment of the present application;
FIG. 7 is a schematic diagram of a document image correction device according to another embodiment of the present application;
FIG. 8 shows a schematic structural diagram of a certificate image correction device according to an embodiment of the application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
To enable those skilled in the art to use the present disclosure, the following embodiments are presented in conjunction with the correction of an application specific scene document image. It will be apparent to those skilled in the art that the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the application. Although the present application is described primarily in the context of correction of a document image, it should be understood that this is merely an exemplary embodiment and that the present application may be applied in a variety of scenarios where picture correction is required, such as: contract image correction, invoice image correction, and the like.
It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.
One aspect of the present application relates to a document image correction system. The system can identify the corner point coordinates to be corrected of the certificate according to the image to be recognized containing the certificate and the preset correction model input by a user, calculate and acquire a transformation matrix according to the target corner point coordinates of each corner point and the corresponding corner point coordinates to be corrected, and perform perspective transformation on the certificate in the image to be recognized according to the transformation matrix to obtain a corrected certificate image, wherein the type of the certificate can be as follows: identity documents, bank cards, account book documents, passport documents, driver's license documents, etc., without limitation thereto.
It is noted that prior art usually corrects the document image according to the text tilt direction before the application, but this correction method can only correct the document image rotation, when the lens plane of the camera is parallel to the document plane. In a perspective shooting scene, a lens plane of a shooting camera may not be parallel to a certificate plane, and the shot certificate has a condition of large and small size, and at this time, a rotation angle cannot be used to describe a non-horizontal state of the certificate, so that the correction mode can cause correction failure.
The certificate image correction method can acquire the coordinates of the corner points to be corrected of the certificate in the image to be recognized according to the preset correction model, calculate the transformation matrix corresponding to the certificate in the image to be recognized according to the target corner point coordinates of each corner point and the corresponding coordinates of the corner points to be corrected, and perform perspective transformation on the certificate in the image to be recognized according to the transformation matrix to obtain the corrected certificate image.
Fig. 1 is a schematic structural diagram of a certificate image correction system 100 provided in an embodiment of the present application, for example: the document image correction system 100 can be a document upload service for identity card uploads, passport uploads and the like, or any platform or scenario involving document correction. As shown in fig. 1, the credential image correction system 100 can include one or more of a server 110, a network 120, a service terminal 130, and a database 140.
In some embodiments, the server 110 may include a processor. The processor may process information and/or data related to the service request to perform one or more of the functions described herein. For example, the processor may determine the user intent based on a service request obtained from the service terminal 130. In some embodiments, a processor may include one or more processing cores (e.g., a single-core processor (S) or a multi-core processor (S)). Merely by way of example, a Processor may include a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), an Application Specific Instruction Set Processor (ASIP), a Graphics Processing Unit (GPU), a Physical Processing Unit (PPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a microcontroller Unit, a reduced Instruction Set computer (reduced Instruction Set computer), a microprocessor, or the like, or any combination thereof.
In some embodiments, the device type to which the service terminal 130 corresponds may be a mobile device, such as may include a wearable device, a smart mobile device, a tablet computer, a laptop computer, and so on. Taking the calibration scene of the certificate image as an example, the service terminal 130 may be a mobile phone of the user, the user uploads the image to be identified through the mobile phone, and the server returns the calibrated image to the mobile phone uploading the image to be identified after calibrating the image to be identified.
In some embodiments, a database 140 can be connected to the network 120 to communicate with one or more components (e.g., the server 110, the service terminal 130, the service provider 140, etc.) in the credential image correction system 100. One or more components in the credential image correction system 100 can access data or instructions stored in the database 140 via the network 120. In some embodiments, the database 140 can be directly connected to one or more components in the credential image correction system 100, or the database 140 can also be part of the server 110.
The following describes in detail the certificate image correction method provided in the embodiment of the present application with reference to the content described in the certificate image correction system 100 shown in fig. 1, where the certificate image correction method is applied to the above system, an execution subject may be a service terminal or a server, a preset scene may be designed and adjusted according to user requirements, any scene related to certificate image correction or contract image correction may be used, and the two scenes provided in the embodiment are not limited.
Referring to fig. 2, a schematic flowchart of a certificate image correction method provided in an embodiment of the present application, which can be executed by a server or a service terminal in a certificate image correction system, includes:
s101: and acquiring coordinates and numbers of corner points to be corrected of certificate corner points in the image to be recognized by adopting a preset correction model.
Optionally, the preset correction model may further identify at least one of the following information: the information identified by the correction model is preset specifically for the angle point confidence of the certificate image in the image to be identified, the certificate outer frame, the certificate rotation direction and the like, and is not limited in any way in the application.
The preset correction model is obtained by training a sample set, the sample set comprises a plurality of sample certificate images, and each sample certificate image is marked with an angular point coordinate and an angular point number.
Optionally, in an embodiment of the present application, since most certificates are quadrangles (rectangles), the coordinates of the corner points to be corrected may include coordinates of the four corner points of the certificate image in the image to be recognized, and if there are other pentagonal, hexagonal, or other polygonal certificates, the coordinates of the corner points to be corrected of five corner points, six corner points, or other multiple corner points corresponding to the certificate in the image to be recognized are correspondingly obtained.
Optionally, the image to be identified may be shot by the user in real time, or may be selected and uploaded from the album for the user, and the specific acquisition mode of the image to be identified may be designed according to the user's needs, which is not limited herein.
Optionally, the image to be recognized is an image including at least one complete certificate, and if it is detected that the image to be recognized does not include the complete certificate, a correction failure instruction is returned.
S102: and calculating and acquiring a transformation matrix according to the target corner coordinates of each corner and the corresponding corner coordinates to be corrected.
In an embodiment of the present application, the transformation matrix is a perspective transformation matrix, and a specific calculation process is as follows: and (2) setting the coordinate of a certain corner point of the certificate to be corrected as [ u, v, w ], setting the corresponding coordinate of the target corner point as [ x ', y ', w ' ], and calculating a perspective transformation matrix according to the corner point coordinate and the corresponding coordinate of the target corner point as shown in a formula (1). Furthermore, each pixel value in the certificate image to be corrected is mapped to a pixel value corresponding to the image in the horizontal state by using the perspective transformation matrix, so that the corrected horizontal image is obtained.
Wherein the content of the first and second substances,
is a perspective transformation matrix.
The target corner point coordinates are preset by a user and used for indicating the coordinate condition that each corner point of the certificate image should be located under the condition that the certificate image is in a normal level, the difference value of the horizontal coordinates or the vertical coordinates between two adjacent corner points indicates the distance between the two adjacent corner points, namely the length or the width of the certificate.
S103: and carrying out perspective transformation on the certificate in the image to be recognized according to the transformation matrix to obtain a corrected certificate image.
By adopting the certificate image correction method provided by the embodiment of the application, the corner coordinates to be corrected of the certificate corners in the image to be recognized can be obtained according to the preset correction model, the transformation matrix corresponding to the certificate in the current image to be recognized is calculated according to the target corner coordinates of each corner point and the corresponding corner coordinates to be corrected, and the certificate in the image to be recognized is subjected to perspective transformation according to the transformation matrix to obtain the corrected certificate image, so that the problem that the correction method in the prior art is greatly influenced by the image background pattern is solved, and the effect of improving the correction effect is achieved.
Alternatively, S101 may include: and according to a preset correction model, carrying out corner detection on the certificate in the image to be recognized, and acquiring the coordinates and the serial numbers of the corner points to be corrected of a preset number of corner points.
In an embodiment of the present application, the corner point numbers in a normal horizontal state are respectively recorded in a clockwise direction from a corner point (e.g., the corner point at the upper left corner) at a preset position in the certificate image as: 1. 2, 3 and 4. However, the setting of the identification position of the corner point in the specific normal state may be designed according to the user requirement, and the present application is not limited herein.
Alternatively, S102 may include: and calculating a transformation matrix according to the target corner coordinates, the expected corner numbers, the corner coordinates to be corrected and the corner numbers.
In the calculation process, the target angular point coordinates and the angular point coordinates to be corrected are in one-to-one correspondence according to the preset angular point numbers and the angular point numbers of the certificate to be corrected, and the transformation matrix is calculated according to the corresponding angular point coordinate pairs.
Fig. 3 is a schematic flowchart of a certificate image correction according to another embodiment of the present application, and as shown in fig. 3, before step S101, the method further includes:
S104: a plurality of sample sets is acquired.
Each sample set comprises a plurality of sample certificate images, and each sample certificate image is marked with corner point coordinates and a corner point number.
Optionally, each sample certificate image includes at least one complete certificate image, and if the sample certificate image includes a plurality of complete certificate images, each complete certificate image needs to be labeled with its own angular point coordinate and angular point number.
S105: and performing regression training on the preset main body network according to the sample set to obtain a preset correction model.
Optionally, in the training process, each sample certificate image can be randomly rotated, and regression training is performed on the preset main body network according to the rotated sample certificate image; the method can also be used for inputting the sample certificate images under a plurality of different rotation angles, the selection setting mode of a specific sample can be adjusted according to the needs of a user, and the method is not limited at all and only needs to be used for training according to the sample certificate images under a plurality of different rotation angles in the training process.
Alternatively, S105 may include: and performing regression training on the preset main body network according to the sample set, and outputting the corner point coordinates of each sample certificate image and corresponding confidence information.
Optionally, in an embodiment of the present application, performing regression training on a preset subject network, and outputting an outer frame of a certificate image in a sample image and a direction of the certificate image; the direction of the certificate image can be presented in an angle form, for example: the north direction can be selected to be 0 degrees, and the direction from the clockwise direction to the north direction is 0 degrees to 360 degrees.
The confidence coefficient is a range value within 0-1, the confidence coefficient information is used for being compared with a preset confidence coefficient threshold value, and if the comparison result shows that the confidence coefficient information is larger than the preset confidence coefficient threshold value, a perspective transformation matrix can be calculated to correct the image; and if the comparison result shows that the confidence coefficient information is smaller than the preset confidence coefficient threshold value, the calculation of the corner point coordinates of the current certificate image is inaccurate, and the image cannot be corrected by using the perspective matrix.
Optionally, if the confidence information is smaller than a preset confidence threshold, the calculation result of the corner coordinates may be inaccurate, which results in inaccurate correction, and at this time, an uncorrectable instruction may be returned to indicate that the current certificate image is uncorrectable; the certificate can also be output after being roughly corrected by utilizing the certificate outer frame and the certificate rotation direction output by the model.
Optionally, a value closer to 1 indicates more accurate result, and a value closer to 0 indicates less accurate result, in an embodiment of the present application, the preset confidence threshold is set to 0.5, but a specific design of the preset confidence threshold may be set according to a user requirement, and the present application is not limited herein.
Fig. 4 is a schematic flowchart of a certificate image correction method according to another embodiment of the present application, as shown in fig. 4, after S103, the method further includes:
s106: and carrying out field detection on the corrected certificate image.
Optionally, in an embodiment of the present application, after detecting the field information of the certificate image, the image of the field area is cut out for character recognition. The method includes the steps of obtaining a document image, and cutting a field image in the document image, wherein the field image can be cut in the document image, and all the field images in the document image can be cut, and the selection of a specific intercepting function and the intercepting mode of the field image can be designed according to the needs of a user, and the method is not limited herein.
S107: and identifying the detected field according to a preset algorithm, and outputting character information corresponding to the image to be identified.
Optionally, in an embodiment of the present application, the field information in the captured picture of the image to be recognized is recognized according to a certificate OCR algorithm, and the recognized character information is output.
The OCR of the certificate can automatically identify the characters in the certificate, and common certificates comprise identity cards, driving licenses, bank cards and the like. The certificate OCR has wide application in different fields, such as bank identity information submission, network appointment driver registration information uploading and verification and the like, and characters in the certificate can be extracted quickly and accurately by utilizing the OCR so as to avoid the problems of low manual input efficiency and high error rate.
Compared with the prior art, as long as the traditional correction algorithm detects that the state of the certificate image is horizontal, horizontal and vertical, the direction of the certificate image cannot be corrected, but the certificate image may not be horizontal and positive at the moment, for example: such as the document image being inverted (180 deg. rotated) or the document image being 90 deg. rotated, the conventional algorithm will not correct the document image any more. That is, the traditional algorithm can only ensure that the corrected certificate image is horizontal and vertical, but cannot ensure that the corrected certificate image is horizontal and positive. The method can output the angular point number of the certificate image through the preset correction model, then can adjust the certificate to be in the horizontal forward direction according to the angular point number, and then carries out perspective transformation on the certificate image according to the transformation matrix to finally obtain the corrected certificate image.
By adopting the certificate image correction method provided by the application, the corner point coordinate information to be corrected of the certificate image in the image to be recognized can be acquired by adopting the preset correction model, the transformation matrix is calculated and acquired according to the target corner point coordinates of each corner point and the corresponding corner point coordinates to be corrected, the certificate image in the image to be recognized is subjected to perspective change according to the transformation matrix to obtain the corrected certificate file, and characters in the certificate are recognized and output by the certificate OCR method.
Based on the same inventive concept, the embodiment of the present application further provides a certificate image correction apparatus corresponding to the certificate image correction method, and as the principle of the apparatus in the embodiment of the present application for solving the problem is similar to that of the certificate image correction method in the embodiment of the present application, the implementation of the apparatus can refer to the implementation of the method, and the repeated points of the beneficial effects are not repeated.
Fig. 5 is a schematic structural diagram of a certificate image correction device according to an embodiment of the present application, and as shown in fig. 5, the device includes: an acquisition module 201, a calculation module 202 and a perspective transformation module 203, wherein:
The acquiring module 201 is configured to acquire to-be-corrected corner coordinates of a certificate corner in an image to be recognized by using a preset correction model, where the preset correction model is acquired by training a sample set, the sample set includes a plurality of sample certificate images, and each sample certificate image is marked with corner coordinates.
And the calculating module 202 is configured to calculate and obtain a transformation matrix according to the target corner coordinates of each corner and the corresponding corner coordinates to be corrected.
And the perspective transformation module 203 is used for performing perspective transformation on the certificate in the image to be recognized according to the transformation matrix to obtain a corrected certificate image.
Optionally, the obtaining module 201 is further configured to perform corner detection on a certificate in an image to be recognized according to a preset correction model, and obtain coordinates of a corner to be corrected, a corner number, a corner confidence, an outer frame of the certificate, and a rotation direction of the certificate, where the corner is a preset number of corners.
Optionally, the calculating module 202 is further configured to calculate a transformation matrix according to the target corner coordinates, the expected corner number, the corner coordinates to be corrected, and the corner number.
Optionally, the obtaining module 201 is further configured to obtain a plurality of sample sets, where each sample set includes a plurality of sample certificate images, and each sample certificate image is marked with a corner coordinate and a corner number; and performing regression training on the preset main body network according to the sample set to obtain a preset correction model.
Fig. 6 is a schematic structural diagram of a certificate image correction device according to an embodiment of the present application, and as shown in fig. 6, the device further includes: a training module 204 and an output module 205, wherein:
and the training module 204 is configured to perform regression training on the preset subject network according to the sample set.
And the output module 205 is configured to output the corner coordinates of each sample certificate image and the corresponding confidence information.
Optionally, the training module 204 is further configured to randomly rotate each sample certificate image, and perform regression training on the preset host network according to the rotated sample certificate image.
Fig. 7 is a schematic structural diagram of a certificate image correction device according to an embodiment of the present application, and as shown in fig. 7, the device further includes a detection module 206 for performing field detection on a corrected certificate image.
The output module 205 is further configured to identify the detected field according to a preset algorithm, and output character information corresponding to the image to be identified.
Fig. 8 is a schematic structural diagram of a certificate image correction device according to an embodiment of the present application, and as shown in fig. 8, the certificate image correction device includes: a processor 601, a memory 602, and a bus 603; the memory 602 stores machine-readable instructions executable by the processor 601, the processor 601 and the memory 602 communicating via the bus 603 when the credential image correction device is operating, the processor 601 executing the machine-readable instructions to perform the steps of the request processing method as provided by the foregoing method embodiments.
Specifically, the machine readable instructions stored in the memory 602 are execution steps of the request processing method described in the foregoing embodiment of the present application, and the processor 601 can execute the request processing method to process the request, so that the electronic device also has all the advantages described in the foregoing embodiment of the method, and the description of the present application is not repeated.
The electronic device may be a general-purpose computer, a special-purpose computer, a server for processing data, or the like, and all of the three may be used to implement the request processing method of the present application. Although the request processing method is described only by the computer and the server separately, for convenience, the functions described in the present application may be implemented in a distributed manner on a plurality of similar platforms to balance the processing load.
For example, an electronic device may include one or more processors for executing program instructions, a communication bus, and different forms of storage media, such as a disk, ROM, or RAM, or any combination thereof. Illustratively, the computer platform may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application may be implemented in accordance with these program instructions.
For ease of illustration, only one processor is depicted in the electronic device. However, it should be noted that the electronic device in the present application may also comprise a plurality of processors, and thus the steps performed by one processor described in the present application may also be performed by a plurality of processors in combination or individually.
The embodiment of the application also provides a storage medium, wherein the storage medium is stored with a computer program, and the computer program is executed by a processor to execute the steps of the certificate image correction method.
Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, and the like, and when a computer program on the storage medium is run, the certificate image correction method can be executed, so that the problems that a sentence library is too large in scale and occupies too many resources due to various language expression combination forms and a large amount of information in the prior art are solved, and the effect of reducing resource occupation is achieved.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to corresponding processes in the method embodiments, and are not described in detail in this application. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (12)
1. A method of document image correction, the method comprising:
acquiring coordinates and angular point numbers of the angular points of the certificate in the image to be identified by adopting a preset correction model, wherein the preset correction model is acquired by training a sample set, the sample set comprises a plurality of sample certificate images, and each sample certificate image is marked with angular point coordinates and angular point numbers;
calculating and acquiring a transformation matrix according to the target corner coordinates of each corner and the corresponding corner coordinates to be corrected;
and carrying out perspective transformation on the certificate in the image to be identified according to the transformation matrix to obtain a corrected certificate image.
2. The method as claimed in claim 1, wherein the obtaining coordinates of the corner points to be corrected of the certificate corner points in the image to be recognized by using the preset correction model comprises:
And according to the preset correction model, carrying out corner detection on the certificate in the image to be recognized, and acquiring the coordinates and the serial numbers of the corner points to be corrected of a preset number of corner points.
3. The method of claim 2, wherein the calculating a transformation matrix according to the target corner point coordinates of each corner point and the corresponding corner point coordinates to be corrected comprises:
and calculating a transformation matrix according to the target corner coordinates, the expected corner numbers, the to-be-corrected corner coordinates and the corner numbers.
4. The method of claim 2, wherein the method further comprises:
acquiring a plurality of sample sets, wherein each sample set comprises a plurality of sample certificate images, and each sample certificate image is marked with corner point coordinates and corner point numbers;
and performing regression training on a preset main body network according to the sample set to obtain the preset correction model.
5. The method of claim 4, wherein the performing regression training on a pre-defined subject network according to the sample set comprises:
and performing regression training on a preset main body network according to the sample set, and outputting the corner coordinates, the corner numbers and the corner confidence coefficients of the certificate images of the samples.
6. The method of claim 4 or 5, wherein the performing regression training on a preset subject network according to the sample set comprises:
and randomly rotating each sample certificate image, and performing regression training on a preset main body network according to the rotated sample certificate image.
7. The method as claimed in claim 6, wherein after the perspective transformation of the document in the image to be recognized according to the transformation matrix to obtain the corrected document image, the method further comprises:
performing field detection on the corrected certificate image;
and identifying the detected field according to a preset algorithm, and outputting character information corresponding to the image to be identified.
8. An apparatus for correcting an image of a document, the apparatus comprising: the system comprises an acquisition module, a calculation module and a perspective transformation module, wherein:
the acquisition module is used for acquiring the coordinates and the serial numbers of the corner points to be corrected of the certificate corner points in the image to be recognized by adopting a preset correction model, the preset correction model is acquired by training of a sample set, the sample set comprises a plurality of sample certificate images, and each sample certificate image is marked with the coordinates and the serial numbers of the corner points;
The calculation module is used for calculating and acquiring a transformation matrix according to the target corner point coordinates of each corner point and the corresponding corner point coordinates to be corrected;
and the perspective transformation module is used for carrying out perspective transformation on the certificate in the image to be identified according to the transformation matrix to obtain a corrected certificate image.
9. The apparatus of claim 8, wherein the apparatus further comprises: training module and output module, wherein:
the training module is used for carrying out regression training on a preset subject network according to the sample set;
and the output module is used for outputting the corner point coordinates, the corner point numbers and the corner point confidence degrees of the sample certificate images.
10. The apparatus of claim 9, further comprising a detection module to field detect the corrected document image;
the output module is further used for identifying the detected fields according to a preset algorithm and outputting the character information corresponding to the image to be identified.
11. An apparatus for correcting an image of a document, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating over the bus when the credential image correction device is in operation, the processor executing the machine-readable instructions to perform the steps of the method of any one of claims 1 to 7.
12. A storage medium, having stored thereon a computer program which, when executed by a processor, performs the steps of the method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911252892.3A CN111860489A (en) | 2019-12-09 | 2019-12-09 | Certificate image correction method, device, equipment and storage medium |
| PCT/CN2020/122362 WO2021078133A1 (en) | 2019-10-24 | 2020-10-21 | Systems and methods for image processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911252892.3A CN111860489A (en) | 2019-12-09 | 2019-12-09 | Certificate image correction method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111860489A true CN111860489A (en) | 2020-10-30 |
Family
ID=72970771
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911252892.3A Pending CN111860489A (en) | 2019-10-24 | 2019-12-09 | Certificate image correction method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111860489A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112508011A (en) * | 2020-12-02 | 2021-03-16 | 上海逸舟信息科技有限公司 | OCR (optical character recognition) method and device based on neural network |
| CN113256583A (en) * | 2021-05-24 | 2021-08-13 | 北京百度网讯科技有限公司 | Image quality detection method and apparatus, computer device, and medium |
| CN113392747A (en) * | 2021-06-07 | 2021-09-14 | 北京优创新港科技股份有限公司 | Goods packing box identification method and system for stereoscopic warehouse |
| CN113436079A (en) * | 2021-06-23 | 2021-09-24 | 平安科技（深圳）有限公司 | Certificate image detection method and device, electronic equipment and storage medium |
| CN113486941A (en) * | 2021-06-30 | 2021-10-08 | 广州方硅信息技术有限公司 | Live image training sample generation method, model training method and electronic equipment |
| CN113538291A (en) * | 2021-08-02 | 2021-10-22 | 广州广电运通金融电子股份有限公司 | Card image tilt correction method and device, computer equipment and storage medium |
| CN113657384A (en) * | 2021-09-02 | 2021-11-16 | 京东科技控股股份有限公司 | Certificate image correction method and device, storage medium and electronic equipment |
| CN113674352A (en) * | 2021-07-28 | 2021-11-19 | 浙江大华技术股份有限公司 | Switch state detection method, electronic device and storage medium |
| CN115630663A (en) * | 2022-12-19 | 2023-01-20 | 成都爱旗科技有限公司 | Two-dimensional code identification method and device and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004228759A (en) * | 2003-01-21 | 2004-08-12 | Nec Corp | Method and apparatus for correcting image |
| CN110046529A (en) * | 2018-12-11 | 2019-07-23 | 阿里巴巴集团控股有限公司 | Two-dimensional code identification method, device and equipment |
| WO2019174130A1 (en) * | 2018-03-14 | 2019-09-19 | 平安科技（深圳）有限公司 | Bill recognition method, server, and computer readable storage medium |
| CN111860527A (en) * | 2019-10-24 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Image correction method, image correction device, computer device, and storage medium |
-
2019
- 2019-12-09 CN CN201911252892.3A patent/CN111860489A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004228759A (en) * | 2003-01-21 | 2004-08-12 | Nec Corp | Method and apparatus for correcting image |
| WO2019174130A1 (en) * | 2018-03-14 | 2019-09-19 | 平安科技（深圳）有限公司 | Bill recognition method, server, and computer readable storage medium |
| CN110046529A (en) * | 2018-12-11 | 2019-07-23 | 阿里巴巴集团控股有限公司 | Two-dimensional code identification method, device and equipment |
| CN111860527A (en) * | 2019-10-24 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Image correction method, image correction device, computer device, and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 田文利;: "基于霍夫直线检测与二维透视变换的图像校正恢复算法", 电子测量技术, no. 09 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112508011A (en) * | 2020-12-02 | 2021-03-16 | 上海逸舟信息科技有限公司 | OCR (optical character recognition) method and device based on neural network |
| CN113256583A (en) * | 2021-05-24 | 2021-08-13 | 北京百度网讯科技有限公司 | Image quality detection method and apparatus, computer device, and medium |
| CN113392747A (en) * | 2021-06-07 | 2021-09-14 | 北京优创新港科技股份有限公司 | Goods packing box identification method and system for stereoscopic warehouse |
| CN113436079A (en) * | 2021-06-23 | 2021-09-24 | 平安科技（深圳）有限公司 | Certificate image detection method and device, electronic equipment and storage medium |
| CN113486941A (en) * | 2021-06-30 | 2021-10-08 | 广州方硅信息技术有限公司 | Live image training sample generation method, model training method and electronic equipment |
| CN113674352A (en) * | 2021-07-28 | 2021-11-19 | 浙江大华技术股份有限公司 | Switch state detection method, electronic device and storage medium |
| CN113538291A (en) * | 2021-08-02 | 2021-10-22 | 广州广电运通金融电子股份有限公司 | Card image tilt correction method and device, computer equipment and storage medium |
| CN113657384A (en) * | 2021-09-02 | 2021-11-16 | 京东科技控股股份有限公司 | Certificate image correction method and device, storage medium and electronic equipment |
| CN113657384B (en) * | 2021-09-02 | 2024-04-05 | 京东科技控股股份有限公司 | Certificate image correction method and device, storage medium and electronic equipment |
| CN115630663A (en) * | 2022-12-19 | 2023-01-20 | 成都爱旗科技有限公司 | Two-dimensional code identification method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111860489A (en) | Certificate image correction method, device, equipment and storage medium | |
| CN107798299B (en) | Bill information identification method, electronic device and readable storage medium | |
| CN111598074B (en) | Edge detection method and device, electronic equipment and storage medium | |
| US10726580B2 (en) | Method and device for calibration | |
| CN108875731B (en) | Target identification method, device, system and storage medium | |
| CN111951290A (en) | Edge detection method and device for object in image | |
| WO2014123619A1 (en) | System and method for identifying similarities in different images | |
| CN110647882A (en) | Image correction method, device, equipment and storage medium | |
| CN107545223B (en) | Image recognition method and electronic equipment | |
| CN106845508A (en) | The methods, devices and systems of release in a kind of detection image | |
| CN112085798B (en) | Camera calibration method and device, electronic equipment and storage medium | |
| US20220398716A1 (en) | Method of detecting product defects, electronic device, and storage medium | |
| CN111950554A (en) | Identification card identification method, device, equipment and storage medium | |
| CN111141208A (en) | Parallel line detection method and device | |
| WO2021077903A1 (en) | Finger image collection method and apparatus, and storage medium | |
| CN110909816B (en) | Picture identification method and device | |
| CN112733773A (en) | Object detection method and device, computer equipment and storage medium | |
| CN110660091A (en) | Image registration processing method and device and photographing correction operation system | |
| CN108629219B (en) | Method and device for identifying one-dimensional code | |
| CN115937003A (en) | Image processing method, image processing device, terminal equipment and readable storage medium | |
| US10032073B1 (en) | Detecting aspect ratios of document pages on smartphone photographs by learning camera view angles | |
| CN111401365B (en) | OCR image automatic generation method and device | |
| CN111429399A (en) | Straight line detection method and device | |
| CN113920525A (en) | Text correction method, device, equipment and storage medium | |
| CN111259703B (en) | Face inclination angle detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |