CN112183296B - Simulated bill image generation and bill image recognition method and device - Google Patents
Simulated bill image generation and bill image recognition method and device Download PDFInfo
- Publication number
- CN112183296B CN112183296B CN202011010927.5A CN202011010927A CN112183296B CN 112183296 B CN112183296 B CN 112183296B CN 202011010927 A CN202011010927 A CN 202011010927A CN 112183296 B CN112183296 B CN 112183296B
- Authority
- CN
- China
- Prior art keywords
- image
- bill
- simulated
- text
- pixel points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/412—Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Processing Or Creating Images (AREA)
- Character Input (AREA)
Abstract
The application relates to a simulated bill image generation method, a simulated bill image generation device, computer equipment and a storage medium. The method comprises the following steps: acquiring a bill image: determining a text to be replaced in the bill image; acquiring a target replacement text from a pre-established corpus database according to the semantic concept of the text to be replaced; and replacing the text to be replaced in the bill image with the target replacement text to obtain a simulated bill image. By adopting the method, the training cost of the bill recognition model can be reduced.
Description
Technical Field
The application relates to the technical field of computers, in particular to a method and a device for generating a simulation bill image and identifying the bill image, computer equipment and a storage medium.
Background
Document identification is a very pressing need in the financial industry. Currently, note recognition training data generally adopts marking data distributed from a crowd-sourcing platform. Because the financial bill has very high confidentiality degree and is usually required to be distributed in a slicing desensitization mode, but the training of the bill identification model is usually required to be carried out on the bill with complete content, the difficulty in acquiring the training data of the bill identification model is high, and the training cost of the bill identification model is also improved.
Therefore, the training cost of the bill recognition model in the prior art is high.
Disclosure of Invention
In view of the above, it is desirable to provide a method, an apparatus, a computer device, and a storage medium for generating a simulated bill image, for recognizing a bill image, which can reduce the training cost of a bill recognition model.
The embodiment of the invention provides a method for generating a simulated bill image, which comprises the following steps:
acquiring a bill image:
determining a text to be replaced in the bill image;
acquiring a target replacement text from a pre-established corpus database according to the semantic concept of the text to be replaced;
and replacing the text to be replaced in the bill image with the target replacement text to obtain a simulated bill image.
In one embodiment, extracting a target replacement text from a pre-established corpus database according to the semantic concept of the text to be replaced includes:
acquiring a plurality of candidate replacement texts which are the same as or similar to the semantic concepts in the corpus database according to the semantic concepts of the texts to be replaced;
and randomly extracting one candidate replacement text from the candidate replacement texts as the target replacement text.
In one embodiment, the replacing the text to be replaced in the ticket image with the target replacement text to obtain a simulated ticket image includes:
erasing the text to be replaced in the bill image to obtain a bill template image;
acquiring region marking information corresponding to the bill template image; the region labeling information is information obtained by labeling semantic concepts corresponding to each content region of the bill template image in advance;
determining a character filling area in each content area of the bill template image according to the area marking information; the semantic concept labeled by the character filling area is the same as that of the target replacement text;
and adding the target replacement text into the character filling area to obtain the simulated bill image.
In one embodiment, the adding the target replacement text to the text-filled area to obtain the simulated ticket image includes:
adding the target replacement text into the character filling area to obtain an image with changed content;
and carrying out simulation processing on the image after the content is changed to obtain the simulated bill image.
In one embodiment, the adding the target replacement text to the text-filled area to obtain an image with changed content includes:
adding the target replacement text into the character filling area to obtain an initially changed image;
performing pixel filling processing on the character edge pixel points in the image after the initial change to obtain a processed image serving as the image after the content change, wherein the appearance similarity between the character edge pixel points in the processed image and surrounding pixel points meets a preset condition; and the pixel distance between the surrounding pixel points and the character edge pixel points is smaller than a preset distance threshold.
In one embodiment, the simulating the image after the content change to obtain the simulated bill image includes:
randomly generating the noise adding probability of the image with the changed content;
when the noise adding probability is larger than a preset probability threshold value, adding image noise to the image with the changed content to obtain an image with the noise added;
and taking the noise-increased image as the simulated bill image.
In one embodiment, the simulating the image after the content change to obtain the simulated bill image includes:
acquiring a pre-established bill background database; the bill background database comprises a plurality of candidate bill background images;
randomly extracting one of the candidate bill background images from the candidate bill background images to serve as a target bill background image;
fusing the image with the changed content with the bill background image to obtain a fused image; wherein the image after the content change is a foreground in the fused image;
and taking the fused image as the simulated bill image.
In one embodiment, the simulating the image after the content change to obtain the simulated bill image includes:
carrying out image distortion processing on the image with the changed content to obtain a distorted image;
and taking the distorted image as the simulated bill image.
The embodiment of the invention also provides a bill image identification method, which comprises the following steps:
acquiring a bill image to be identified;
inputting the bill image to be recognized into a trained bill recognition model to obtain a character recognition result aiming at the bill image to be recognized; the trained bill recognition model is obtained by training the initial bill recognition model by adopting a simulated bill image; the simulated bill image is obtained by the simulated bill image generation method.
The embodiment of the invention also provides a device for generating the simulated bill image, which comprises the following components:
the acquisition module is used for acquiring the bill image:
the determining module is used for determining the text to be replaced in the bill image;
the extraction module is used for acquiring a target replacement text from a pre-established corpus database according to the semantic concept of the text to be replaced;
and the replacing module is used for replacing the text to be replaced in the bill image with the target replacing text to obtain the simulated bill image.
The embodiment of the invention also provides a bill image recognition device, which comprises:
the bill acquiring module is used for acquiring a bill image to be identified;
the input module is used for inputting the bill image to be recognized to the trained bill recognition model to obtain a character recognition result aiming at the bill image to be recognized; the trained bill recognition model is obtained by training the initial bill recognition model by adopting a simulated bill image; the simulated bill image is obtained according to the simulated bill image generation device.
The embodiment of the present invention further provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the method when executing the computer program.
Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the above-mentioned method.
The simulated bill image generation and bill image recognition method, device, computer equipment and storage medium provided by the embodiment of the invention are characterized in that the bill image is obtained, the text to be replaced in the bill image is determined, and the target replacement text with the same semantic concept in the bill as the text to be replaced is extracted from the pre-established corpus database; then, replacing the text to be replaced with a target replacement text in the bill image to obtain a simulated bill image for training the bill recognition model, so that the trained bill recognition model is used for performing text detection on the input bill image; therefore, the simulated bill image which is high in simulation degree and complete in content and used for training the bill recognition model can be generated on the basis of the bill image, the difficulty in acquiring the training data of the bill recognition model is effectively reduced, and the training cost of the bill recognition model is reduced.
Drawings
FIG. 1 is a diagram of an application environment of a method for generating a simulated document image according to an embodiment;
FIG. 2 is a schematic flow chart diagram of a method for generating a simulated document image according to one embodiment;
FIG. 3 is a schematic flow chart diagram illustrating a method for generating a simulated document image according to another embodiment;
FIG. 4 is a flow chart illustrating a method for document image recognition according to one embodiment;
FIG. 5 is a block diagram showing the structure of a simulated document image generating apparatus according to an embodiment;
FIG. 6 is a block diagram of a document image recognition apparatus according to an embodiment;
FIG. 7 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The method for generating the simulated bill image can be applied to the application environment shown in fig. 1. Wherein, the computer device 110 acquires the ticket image: then, the computer device 110 determines that the text in the ticket image is to be replaced; then, the computer device 110 obtains a target replacement text from a pre-established corpus database according to the semantic concept of the text to be replaced; and replacing the text to be replaced in the bill image with the target replacement text to obtain a simulated bill image. In practical applications, the computer device 110 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and may also be implemented by an independent server or a server cluster composed of a plurality of servers.
In one embodiment, as shown in fig. 2, there is provided a simulated document image generation method, which is described by taking the method as an example applied to the computer device in fig. 1, and comprises the following steps:
step S210, acquiring a bill image.
In practical application, the bill image can be an image obtained by image acquisition of various real financial bills such as various invoices, consumption certificates, checks and the like. In practical application, the bill image can also be named as a real bill image.
In a specific implementation, the computer device may acquire the bill image before the computer device needs to acquire the training data of the bill recognition model. Specifically, the user can upload images, namely bill images, obtained by image acquisition of various real financial bills such as various invoices, consumption certificates, checks and the like to the computer device for reception by the computer device, so that the computer device can acquire the bill images.
Step S220, determining the text to be replaced in the ticket image.
The text to be replaced may be a text that needs to be replaced in the ticket image.
In a specific implementation, after the computer device obtains the bill image, the computer device determines that the text to be replaced in the bill image is to be replaced.
Specifically, the computer device can obtain the region annotation data corresponding to the bill image. Wherein, the region marking data is marked with each content changeable region in the bill image. And the text in the content changeable area is the text to be replaced.
In practical application, the computer device may obtain the region annotation data corresponding to the bill image, and determine each content changeable region in the bill image according to the region annotation data. And finally, the computer device takes the text in each content changeable area as the text to be replaced.
Step S230, obtaining a target replacement text from the pre-established corpus database according to the semantic concept of the text to be replaced.
The semantic concept can refer to a concept corresponding to the semantic of the text in the bill. For example, if text a is "X city, ltd" and text B is "Y city, ltd", the semantic concept of text a and text B in the ticket is the company name.
In specific implementation, the region annotation data corresponding to the bill image further records semantic concepts corresponding to texts (i.e., texts to be replaced) in each content-changeable region in the bill image. After the computer equipment determines that the text to be replaced in the bill image is to be replaced, the computer equipment acquires the semantic concept of the text to be replaced according to the region marking data corresponding to the bill image.
Of course, the computer device may also input the text to be replaced into a pre-trained neural network (e.g., a trained semantic concept recognition model) through which the semantic concept of the text to be replaced is recognized.
And then, the computer equipment randomly extracts a target replacing text which has the same or similar semantic concept with the text to be replaced in the bill in the pre-established corpus database.
In practical application, the corpus database can be established based on the text information carried by a large number of real bills collected in advance. For example, for the semantic concept of "company name", the corpus database records a plurality of company name information which has a corresponding relationship with the semantic concept "company name" and is determined based on text information carried by a large number of real tickets.
For another example, for the semantic concept "commodity name", the corpus database records a plurality of commodity name information which has a corresponding relationship with the semantic concept "commodity name" and is determined based on text information carried by a large number of real bills.
And S240, replacing the text to be replaced in the bill image with the target replacement text to obtain the simulated bill image.
And the simulated bill image is used for training the bill recognition model.
The trained bill recognition model is used for carrying out text detection on the input bill image.
In specific implementation, after the computer device determines the target replacement text, the computer device replaces the text to be replaced with the target replacement text in the bill image to obtain the simulated bill image.
Specifically, the computer equipment can erase the text to be replaced in the bill image to obtain an erased image; wherein, the image after the erasing process comprises a character filling area; the character filling area is an area obtained after erasing processing is carried out on an image area where the text to be replaced is located; then, the computer equipment adds the target replacing text into the character filling area to obtain the simulated bill image.
In practical application, training samples for training the bill recognition model can be generated based on the simulated bill images. Wherein the training sample comprises two parts including: the simulated bill image and label data corresponding to the simulated bill image are the labeling data.
When the training target of the ticket recognition model is the detection of the training text region, the label data may be the rectangular coordinates of each text region. When the training target of the ticket recognition model is the detection of the content of the training text, the labeled data can be the content of each character area.
In the simulated bill image generation method, the bill image is obtained, the text to be replaced in the bill image is determined, and the target replacement text with the semantic concept same as that of the text to be replaced in the bill is extracted from the pre-established corpus database; then, replacing the text to be replaced with a target replacement text in the bill image to obtain a simulated bill image for training the bill recognition model, so that the trained bill recognition model is used for performing text detection on the input bill image; therefore, the simulated bill image which is high in simulation degree and complete in content and used for training the bill recognition model can be generated on the basis of the bill image, the difficulty in acquiring the training data of the bill recognition model is effectively reduced, and the training cost of the bill recognition model is also reduced.
In another embodiment, replacing the text to be replaced in the bill image with the target replacement text to obtain a simulated bill image, including: erasing the text to be replaced in the bill image to obtain a bill template image; acquiring region marking information corresponding to the bill template image; determining a character filling area in each content area of the bill template image according to the area marking information; and adding the target replacement text into the character filling area to obtain the simulated bill image.
The region labeling information is information obtained by labeling semantic concepts corresponding to each content region of the bill template image in advance.
And the semantic concept marked in the character filling area is the same as that of the target replacing text.
In practical application, different types and different positions of content areas have corresponding area marking information.
In the specific implementation, the process of replacing the text to be replaced in the bill image with the target replacement text by the computer device to obtain the simulated bill image specifically includes: the computer equipment carries out erasing processing on the text to be replaced in the bill image to obtain a bill template image; and then, the computer equipment acquires the region marking information corresponding to the bill template image. Then, the computer equipment determines a character filling area to be replaced with the text in the bill image based on the area marking information. The character filling area is an area obtained by erasing the image area where the text to be replaced is located. Namely, the semantic concept marked by the character filling area is the same as that of the target replacing text.
And finally, adding the target replacement text into the character filling area by the computer equipment to obtain the simulated bill image.
According to the technical scheme of the embodiment, the region marking information corresponding to the bill image is obtained; and according to the region marking information, quickly determining a character filling region in the bill image, which comprises the characters needing to be added with the target replacement characters, so that the text to be replaced in the bill image can be quickly replaced with the target replacement text, and the simulated bill image can be obtained.
In another embodiment, extracting the target replacement text from the pre-established corpus database comprises: acquiring a plurality of candidate replacement texts which are the same as or similar to the semantic concepts in a corpus database according to the semantic concepts of the texts to be replaced; and randomly extracting one candidate replacement text from the candidate replacement texts as a target replacement text.
In a specific implementation, in a process of extracting a target replacement text from a pre-established corpus database, the computer device specifically includes: the computer device obtains semantic concept information corresponding to the text-filled area. Then, based on the semantic concept information, a plurality of candidate replacement texts with the same or similar semantic concepts in the bill to the text to be replaced are inquired in the corpus database. Then, the computer device randomly extracts one candidate replacement text from the candidate replacement texts as a target replacement text.
According to the technical scheme, the target replacement text with the semantic concept which is the same as that of the text to be replaced in the bill can be accurately extracted from the pre-established corpus database, and the text to be replaced can be replaced conveniently in the follow-up process.
In another embodiment, adding the target replacement text to the text-filled area to obtain a simulated ticket image comprises: adding the target replacement text into the character filling area to obtain an image with changed content; and carrying out simulation processing on the image with the changed content to obtain a simulated bill image.
In a specific implementation, the process of adding the target replacement text to the character filling area by the computer device to obtain the simulated bill image specifically includes: and the computer equipment adds the target replacing text into the character filling area to obtain the image with the changed content. In particular, the computer device may randomly add the target replacement text to any location in the word fill area. And finally, the computer equipment carries out simulation processing on the image after the content change by means of randomly adding image noise, adding a picture background, carrying out distortion processing and the like to the image after the content change to obtain a simulated bill image.
According to the technical scheme of the embodiment, the text to be replaced is erased in the character filling area corresponding to the bill image, so that an erased image is obtained; adding the target replacement text into a character filling area corresponding to the erased image to obtain an image with changed content; and performing simulation processing on the image with the changed content to enable the simulated bill image to have high simulation degree, so that the bill recognition model obtained based on the training of the simulated bill image can more accurately generate the character recognition result corresponding to the bill image to be recognized.
In another embodiment, adding the target replacement text to the text-filled area to obtain the content-altered image includes: adding the target replacement text into the character filling area to obtain an image after initial change; and carrying out pixel filling processing on character edge pixel points in the image after the initial change to obtain a processed image serving as the image after the content change.
And the appearance similarity between the character edge pixel points and the surrounding pixel points in the processed image accords with a preset condition.
The text edge pixel points may refer to pixel points located at the edges of the text pixel points.
And the pixel distance between the surrounding pixel points and the character edge pixel points is smaller than a preset distance threshold value.
In a specific implementation, in a process of adding a target replacement text to a character filling area corresponding to an erased image and obtaining an image with changed content, a computer device specifically includes: the computer device may first add the target replacement text to the text-filled area corresponding to the erased image to obtain an initially altered image. And then, the computer equipment carries out pixel filling processing on the character edge pixel points in the image after the initial change in a neighborhood filling mode to obtain a processed image.
Specifically, the computer device may determine text edge pixel points in the initially changed image as pixel points to be filled. Then, the computer device searches for N × N points centered on the pixel point to be filled in the image matrix by a loop. In practice, N may be equal to 5. Then, eliminating pixels with similar color values and character color values (namely pixels within a specified color threshold range) from all the pixels searched by the computer equipment; and then, obtaining the region with the minimum standard deviation in the rest pixel points, and then obtaining the weighted average value of the region as the pixel value of the pixel point to be filled.
According to the technical scheme, the target replacement text is added to the character filling area corresponding to the erased image to obtain the initially changed image, pixel filling processing is carried out on character edge pixel points in the initially changed image to obtain the processed image which is used as the content changed image, so that the target replacement text in the content changed image is closer to characters in the bill image shot under a real scene, the simulation degree of the simulated bill image is improved, and then the bill identification model obtained based on the simulated bill image training can more accurately generate the character identification result corresponding to the bill image to be identified.
In another embodiment, the simulating the image after the content change to obtain a simulated bill image includes: randomly generating the noise adding probability of the image with the changed content; when the noise adding probability is larger than a preset probability threshold value, adding image noise to the image with the changed content to obtain an image with the noise added; and taking the noise-increased image as a simulated bill image.
In the specific implementation, the process that the computer device performs simulation processing on the image after the content is changed to obtain the simulated bill image specifically includes: after the computer device obtains the content-altered image, the computer device may randomly generate a noise-addition probability corresponding to the content-altered image. Then, the computer device determines whether or not it is necessary to add noise to the content-modified image generated this time, based on the noise addition probability. And when the computer equipment judges that the noise adding probability is greater than a preset probability threshold value, the computer equipment adds image noise to the image with the changed content to obtain an image with the added noise, and the image is used as a simulated bill image.
Specifically, when the computer device adds image noise to the post-content-alteration image, the computer device may randomly acquire noise to be added from a noise library. Wherein, the noise library comprises conventional noise and custom noise. The conventional noise comprises Gaussian filtering, Poisson filtering, salt and pepper filtering, contour filtering, depth smoothing filtering, sharpening filtering, custom convolution kernel filtering and grade filtering with a given size.
According to the technical scheme, in the process of carrying out simulation processing on the image after content change to obtain the simulated bill image, the noise adding probability of the image after content change is randomly generated, image noise is added to the image after content change to obtain the image after noise increase, the image after noise increase is used as the simulated bill image, the obtained simulated bill image can be closer to the bill image shot in a real scene, and further the bill recognition model obtained based on the training of the simulated bill image can more accurately generate the character recognition result corresponding to the bill image to be recognized.
In another embodiment, the simulating the image after the content change to obtain a simulated bill image includes: acquiring a pre-established bill background database; the bill background database comprises a plurality of candidate bill background images; randomly extracting one of the candidate bill background images from the plurality of candidate bill background images to serve as a target bill background image; fusing the image with the bill background image after the content is changed to obtain a fused image; wherein, the image after the content change is the foreground in the fused image; and taking the fused image as a simulated bill image.
The bill background database may be a database storing a plurality of candidate bill background images.
In practical application, the candidate bill background image can be an image obtained by cutting out background parts from a plurality of bill images with remarkable characteristics. Of course, the bill background database can also be named as a custom background library, a bill background library, and the like.
In the specific implementation, the process that the computer device performs simulation processing on the image after the content is changed to obtain the simulated bill image specifically includes: after the computer device obtains the image with the changed content, the computer device can randomly extract a candidate bill background image in a pre-established bill background database to serve as the bill background image. And then, the computer equipment fuses the image with the bill background image after the content is changed to obtain a fused image. Specifically, the computer device may superimpose the content-modified image on the bill background image during image processing to obtain a fused image as the simulated bill image. So that the image with changed content is the foreground in the fused image.
In the technical scheme of the embodiment, in the process of obtaining the simulated bill image by performing simulation processing on the image after content change, the bill background image is queried in a pre-established bill background database: fusing the image with the bill background image after the content is changed to obtain a fused image; wherein, the image after the content change is the foreground in the fused image; taking the fused image as a simulated bill image; therefore, the obtained simulated bill image is closer to the bill image shot in the real scene, and the bill recognition model obtained based on the training of the simulated bill image can more accurately generate the character recognition result corresponding to the bill image to be recognized.
In another embodiment, the simulating the image after the content change to obtain a simulated bill image includes: carrying out image distortion processing on the image with the changed content to obtain a distorted image; and taking the distorted image as a simulated bill image.
In the specific implementation, the process that the computer device performs simulation processing on the image after the content is changed to obtain the simulated bill image specifically includes: and the computer equipment carries out image distortion processing on the image with the changed content to obtain a distorted image. Specifically, the computer device may perform operations such as random rotation, simulated tilting, etc. on the content-modified image within a threshold range, and randomly perform distortion on the content of the content-modified image within the threshold range (where the distortion may employ various matrix transformation algorithms, such as radial transformation, perspective transformation, etc.), thereby implementing the simulated warped picture. Of course, there is also an image distortion algorithm based on a deep neural network, for example, by using a pre-trained image distortion model, the image after content change is subjected to image distortion processing to obtain a distorted image. And finally, the computer equipment takes the distorted image as a simulated bill image.
According to the technical scheme, in the process of carrying out simulation processing on the image after the content is changed to obtain the simulated bill image, the image after the content is changed is subjected to image distortion processing to obtain the distorted image as the simulated bill image, and then the condition of curling and deformation of the bill in a real scene is simulated, so that the obtained simulated bill image is closer to the bill image shot in the real scene, and further the bill identification model obtained based on the training of the simulated bill image can more accurately generate the character identification result corresponding to the bill image to be identified.
In another embodiment, as shown in fig. 3, a method for generating a simulated bill image is provided, which is described by taking the method as an example of being applied to the terminal in fig. 1, and includes the following steps: step S310, acquiring a bill image. And step S320, determining the text to be replaced in the bill image. Step S330, a plurality of candidate replacement texts which are the same as or similar to the semantic concept are obtained in the corpus database according to the semantic concept of the text to be replaced. Step S340, randomly extracting one of the candidate replacement texts from the candidate replacement texts as a target replacement text. Step S350, replacing the text to be replaced in the bill image with the target replacement text to obtain a simulated bill image; the simulated bill image is used for training a bill recognition model; and the trained bill recognition model is used for carrying out text detection on the input bill image. It should be noted that, the specific limitations of the above steps can be referred to the above specific limitations of a method for generating a simulated bill image.
In one embodiment, as shown in fig. 4, there is provided a bill image recognition method, which is described by taking the method as an example applied to the computer device in fig. 1, and includes the following steps:
and step S410, acquiring a bill image to be identified.
In a specific implementation, when text recognition needs to be performed on a bill image to be recognized, for example, bill text content recognition and bill text region recognition, a computer device acquires the bill image to be recognized.
Step S420, inputting the bill image to be recognized into the trained bill recognition model to obtain a character recognition result aiming at the bill image to be recognized; the trained bill recognition model is obtained by training the initial bill recognition model by adopting a simulated bill image; a simulated sheet image obtained by the method for generating a simulated sheet image according to the claim.
In the specific implementation, after the computer device obtains the bill image to be recognized, the computer device inputs the bill image to be recognized into the trained bill recognition model. And determining a character recognition result aiming at the bill image to be recognized through the trained bill recognition model.
The trained bill recognition model is obtained by training the initial bill recognition model by adopting a simulated bill image; a simulated sheet image obtained by the method for generating a simulated sheet image according to the claim.
It should be understood that although the steps in the flowcharts of fig. 2, 3 and 4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 3 and 4 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 5, there is provided a simulated document image generating apparatus including:
an acquiring module 510, configured to acquire a ticket image:
a determining module 520, configured to determine to-be-replaced texts in the ticket image;
an extraction module 530, configured to obtain a target replacement text from a pre-established corpus database according to the semantic concept of the text to be replaced;
and a replacing module 540, configured to replace the text to be replaced in the ticket image with the target replacement text, so as to obtain a simulated ticket image.
In one embodiment, the extracting module 530 is specifically configured to obtain, in the corpus database, a plurality of candidate replacement texts that are the same as or similar to the semantic concept of the text to be replaced according to the semantic concept; and randomly extracting one candidate replacement text from the candidate replacement texts as the target replacement text.
In one embodiment, the replacing module 540 is specifically configured to erase the text to be replaced in the bill image to obtain a bill template image; acquiring region marking information corresponding to the bill template image; the region labeling information is information obtained by labeling semantic concepts corresponding to each region of the bill template image in advance; determining a character filling area in each content area of the bill template image according to the area marking information; the semantic concept labeled by the character filling area is the same as that of the target replacement text; and adding the target replacement text into the character filling area to obtain the simulated bill image.
In one embodiment, the replacing module 540 is specifically configured to add the target replacing text to the text-filled area to obtain an image with changed content; and carrying out simulation processing on the image after the content is changed to obtain the simulated bill image.
In one embodiment, the replacing module 540 is specifically configured to add the target replacing text to the character filling area to obtain an initial modified image; performing pixel filling processing on the character edge pixel points in the image after the initial change to obtain a processed image serving as the image after the content change, wherein the appearance similarity between the character edge pixel points in the processed image and surrounding pixel points meets a preset condition; and the pixel distance between the surrounding pixel points and the character edge pixel points is smaller than a preset distance threshold.
In one embodiment, the replacing module 540 is specifically configured to randomly generate a noise adding probability of the content-modified image; when the noise adding probability is larger than a preset probability threshold value, adding image noise to the image with the changed content to obtain an image with the noise added; and taking the noise-increased image as the simulated bill image.
In one embodiment, the replacing module 540 is specifically configured to obtain a pre-established ticket context database; the bill background database comprises a plurality of candidate bill background images; randomly extracting one of the candidate bill background images from the candidate bill background images to serve as a target bill background image; fusing the image with the changed content with the bill background image to obtain a fused image; wherein the image after the content change is a foreground in the fused image; and taking the fused image as the simulated bill image.
In one embodiment, the replacing module 540 is specifically configured to perform image distortion processing on the content-modified image to obtain a distorted image; and taking the distorted image as the simulated bill image.
In one embodiment, as shown in fig. 6, there is provided a bill image recognition apparatus including:
the bill acquiring module 610 is used for acquiring a bill image to be identified;
the input module 620 is configured to input the to-be-recognized bill image to the trained bill recognition model, so as to obtain a character recognition result for the to-be-recognized bill image; the trained bill recognition model is obtained by training the initial bill recognition model by adopting a simulated bill image; the simulated bill image is obtained according to the simulated bill image generation device.
For the specific limitations of the device for generating and identifying the simulated bill image, reference may be made to the above limitations of the method for generating and identifying the simulated bill image, which are not described herein again. All or part of the modules in the simulated bill image generation and bill image recognition device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 7. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a simulated document image generation, document image recognition method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of a simulated document image generation, document image recognition method as described above. Here, the steps of a method for generating a simulated sheet image and identifying a sheet image may be the steps of a method for generating a simulated sheet image and identifying a sheet image according to the above embodiments.
In one embodiment, a computer readable storage medium is provided, storing a computer program which, when executed by a processor, causes the processor to perform the steps of a simulated document image generation, document image recognition method as described above. Here, the steps of a method for generating a simulated sheet image and identifying a sheet image may be the steps of a method for generating a simulated sheet image and identifying a sheet image according to the above embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method for generating a simulated bill image, comprising:
acquiring a bill image:
determining a text to be replaced in the bill image;
acquiring a target replacement text from a pre-established corpus database according to the semantic concept of the text to be replaced;
erasing the text to be replaced to obtain a bill template image;
acquiring region marking information corresponding to the bill template image; the region labeling information is information obtained by labeling semantic concepts corresponding to each region of the bill template image in advance;
determining character filling areas with the semantic concepts the same as that of the target replacement text in each area according to the area marking information;
adding the target replacement text into the character filling area to obtain an image after initial change;
taking the character edge pixel points in the image after the initial change as pixel points to be filled; eliminating pixel points within a specified color threshold range from surrounding pixel points of the character edge pixel points to obtain residual pixel points; the surrounding pixel points are pixel points of which the pixel distance between the surrounding pixel points and the character edge pixel points is smaller than a preset distance threshold;
obtaining a weighted average value of the region with the minimum standard deviation in the rest pixel points, and taking the weighted average value as the pixel value of the pixel point to be filled to obtain an image with changed content;
and carrying out simulation processing on the image after the content is changed to obtain a simulated bill image.
2. The method according to claim 1, wherein extracting a target replacement text from a pre-established corpus database according to the semantic concept of the text to be replaced comprises:
acquiring a plurality of candidate replacement texts which are the same as or similar to the semantic concepts in the corpus database according to the semantic concepts of the texts to be replaced;
and randomly extracting one candidate replacement text from the candidate replacement texts as the target replacement text.
3. The method of claim 1, wherein the simulating the content-altered image to obtain a simulated ticket image comprises:
randomly generating the noise adding probability of the image with the changed content;
when the noise adding probability is larger than a preset probability threshold value, adding image noise to the image with the changed content to obtain an image with the noise added;
and taking the noise-increased image as the simulated bill image.
4. The method of claim 1, wherein the simulating the content-altered image to obtain a simulated ticket image comprises:
acquiring a pre-established bill background database; the bill background database comprises a plurality of candidate bill background images;
randomly extracting one of the candidate bill background images from the candidate bill background images to serve as a target bill background image;
fusing the image with the changed content with the bill background image to obtain a fused image; wherein the image after the content change is a foreground in the fused image;
and taking the fused image as the simulated bill image.
5. The method of claim 1, wherein the simulating the content-altered image to obtain the simulated ticket image comprises:
carrying out image distortion processing on the image with the changed content to obtain a distorted image;
and taking the distorted image as the simulated bill image.
6. A bill image recognition method is characterized by comprising the following steps:
acquiring a bill image to be identified;
inputting the bill image to be recognized into a trained bill recognition model to obtain a character recognition result aiming at the bill image to be recognized; the trained bill recognition model is obtained by training the initial bill recognition model by adopting a simulated bill image; the simulated bill image is obtained by the simulated bill image generation method according to any one of claims 1 to 5.
7. An apparatus for generating a simulated document image, the apparatus comprising:
the acquisition module is used for acquiring the bill image:
the determining module is used for determining the text to be replaced in the bill image;
the extraction module is used for acquiring a target replacement text from a pre-established corpus database according to the semantic concept of the text to be replaced;
the replacing module is used for erasing the text to be replaced to obtain a bill template image; acquiring region marking information corresponding to the bill template image; the region labeling information is information obtained by labeling semantic concepts corresponding to each region of the bill template image in advance; determining character filling areas with the semantic concepts the same as that of the target replacement text in each area according to the area marking information; adding the target replacement text into the character filling area to obtain an image after initial change; taking the character edge pixel points in the image after the initial change as pixel points to be filled; eliminating pixel points within a specified color threshold range from surrounding pixel points of the character edge pixel points to obtain residual pixel points; the surrounding pixel points are pixel points of which the pixel distance between the surrounding pixel points and the character edge pixel points is smaller than a preset distance threshold; obtaining a weighted average value of the region with the minimum standard deviation in the rest pixel points, and taking the weighted average value as the pixel value of the pixel point to be filled to obtain an image with changed content; and carrying out simulation processing on the image after the content is changed to obtain a simulated bill image.
8. A document image recognition apparatus, comprising:
the bill acquiring module is used for acquiring a bill image to be identified;
the input module is used for inputting the bill image to be recognized to the trained bill recognition model to obtain a character recognition result aiming at the bill image to be recognized; the trained bill recognition model is obtained by training the initial bill recognition model by adopting a simulated bill image; the simulated bill image is obtained by the simulated bill image generation method according to any one of claims 1 to 5.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 6.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011010927.5A CN112183296B (en) | 2020-09-23 | 2020-09-23 | Simulated bill image generation and bill image recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011010927.5A CN112183296B (en) | 2020-09-23 | 2020-09-23 | Simulated bill image generation and bill image recognition method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112183296A CN112183296A (en) | 2021-01-05 |
| CN112183296B true CN112183296B (en) | 2022-03-04 |
Family
ID=73956886
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011010927.5A Active CN112183296B (en) | 2020-09-23 | 2020-09-23 | Simulated bill image generation and bill image recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112183296B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113158808B (en) * | 2021-03-24 | 2023-04-07 | 华南理工大学 | Method, medium and equipment for Chinese ancient book character recognition, paragraph grouping and layout reconstruction |
| CN113065940B (en) * | 2021-04-27 | 2023-11-17 | 江苏环迅信息科技有限公司 | Method, device, equipment and storage medium for reimbursement of invoice based on artificial intelligence |
| CN113223117A (en) * | 2021-05-12 | 2021-08-06 | 北京世纪好未来教育科技有限公司 | Image processing method and related device |
| CN114648814A (en) * | 2022-02-25 | 2022-06-21 | 北京百度网讯科技有限公司 | Face living body detection method, training method, device, equipment and medium of model |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368295A (en) * | 2017-06-07 | 2017-11-21 | 努比亚技术有限公司 | A kind of terminal wallpaper generation method, terminal and computer-readable recording medium |
| CN109492643A (en) * | 2018-10-11 | 2019-03-19 | 平安科技（深圳）有限公司 | Certificate recognition methods, device, computer equipment and storage medium based on OCR |
| CN110347983A (en) * | 2019-04-22 | 2019-10-18 | 五八有限公司 | Training sample store method, device, electronic equipment and storage medium |
| CN110458918A (en) * | 2019-08-16 | 2019-11-15 | 北京百度网讯科技有限公司 | Method and apparatus for output information |
| CN110807823A (en) * | 2019-11-13 | 2020-02-18 | 四川大学 | Image simulation generation method for dot matrix character printing effect |
| CN111414906A (en) * | 2020-03-05 | 2020-07-14 | 北京交通大学 | Data synthesis and text recognition method for paper bill picture |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160012594A1 (en) * | 2014-07-10 | 2016-01-14 | Ditto Labs, Inc. | Systems, Methods, And Devices For Image Matching And Object Recognition In Images Using Textures |
| CN108320265B (en) * | 2018-01-31 | 2021-09-21 | 努比亚技术有限公司 | Image processing method, terminal and computer readable storage medium |
| US10678848B2 (en) * | 2018-02-12 | 2020-06-09 | Wipro Limited | Method and a system for recognition of data in one or more images |
| CN109522975A (en) * | 2018-09-18 | 2019-03-26 | 平安科技（深圳）有限公司 | Handwriting samples generation method, device, computer equipment and storage medium |
| CN110503100B (en) * | 2019-08-16 | 2022-05-03 | 湖南星汉数智科技有限公司 | Medical document identification method and device, computer device and computer-readable storage medium |
| CN110598686B (en) * | 2019-09-17 | 2023-08-04 | 携程计算机技术（上海）有限公司 | Invoice identification method, system, electronic equipment and medium |
-
2020
- 2020-09-23 CN CN202011010927.5A patent/CN112183296B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368295A (en) * | 2017-06-07 | 2017-11-21 | 努比亚技术有限公司 | A kind of terminal wallpaper generation method, terminal and computer-readable recording medium |
| CN109492643A (en) * | 2018-10-11 | 2019-03-19 | 平安科技（深圳）有限公司 | Certificate recognition methods, device, computer equipment and storage medium based on OCR |
| CN110347983A (en) * | 2019-04-22 | 2019-10-18 | 五八有限公司 | Training sample store method, device, electronic equipment and storage medium |
| CN110458918A (en) * | 2019-08-16 | 2019-11-15 | 北京百度网讯科技有限公司 | Method and apparatus for output information |
| CN110807823A (en) * | 2019-11-13 | 2020-02-18 | 四川大学 | Image simulation generation method for dot matrix character printing effect |
| CN111414906A (en) * | 2020-03-05 | 2020-07-14 | 北京交通大学 | Data synthesis and text recognition method for paper bill picture |
Non-Patent Citations (1)
| Title |
|---|
| 票据表单样数据文本检测与识别;王剑强;《中国优秀硕士学位论文全文数据库 信息科技辑》;20200815(第8期);第I138-628页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112183296A (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112183296B (en) | Simulated bill image generation and bill image recognition method and device | |
| CN111027563A (en) | Text detection method, device and recognition system | |
| CN111615702B (en) | Method, device and equipment for extracting structured data from image | |
| CN111898411B (en) | Text image labeling system, method, computer device and storage medium | |
| CN107886082B (en) | Method and device for detecting mathematical formulas in images, computer equipment and storage medium | |
| CN111507330B (en) | Problem recognition method and device, electronic equipment and storage medium | |
| JP2019079347A (en) | Character estimation system, character estimation method, and character estimation program | |
| CN113111880A (en) | Certificate image correction method and device, electronic equipment and storage medium | |
| CN111507285A (en) | Face attribute recognition method and device, computer equipment and storage medium | |
| CN110414622B (en) | Classifier training method and device based on semi-supervised learning | |
| CN113673528B (en) | Text processing method, text processing device, electronic equipment and readable storage medium | |
| CN112749639B (en) | Model training method and device, computer equipment and storage medium | |
| CN112396047B (en) | Training sample generation method and device, computer equipment and storage medium | |
| US10691884B2 (en) | System and method for cheque image data masking using data file and template cheque image | |
| CN113468906B (en) | Graphic code extraction model construction method, identification device, equipment and medium | |
| CN115424001A (en) | Scene similarity estimation method and device, computer equipment and storage medium | |
| CN114821062A (en) | Commodity identification method and device based on image segmentation | |
| CN111414728B (en) | Numerical data display method, device, computer equipment and storage medium | |
| CN111027325B (en) | Model generation method, entity identification device and electronic equipment | |
| CN111476090A (en) | Watermark identification method and device | |
| CN113591857A (en) | Character image processing method and device and ancient Chinese book image identification method | |
| CN113536169B (en) | Method, device, equipment and storage medium for typesetting characters of webpage | |
| CN112464720B (en) | Document image processing method, document image processing device, document image model training method, document image model processing device, document image model training device and computer equipment | |
| US20230394865A1 (en) | Methods and systems for performing data capture | |
| CN116884019A (en) | Signature recognition method, signature recognition device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100192 Room 401, building 4, area C, Dongsheng Science Park, 66 xixiaokou Road, Haidian District, Beijing Applicant after: Zhongdian Jinxin Software Co.,Ltd. Address before: 100192 Room 401, building 4, area C, Dongsheng Science Park, 66 xixiaokou Road, Haidian District, Beijing Applicant before: Beijing Wensi Haihui Jinxin Software Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |