CN109635627A - Pictorial information extracting method, device, computer equipment and storage medium - Google Patents
Pictorial information extracting method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN109635627A CN109635627A CN201811236301.9A CN201811236301A CN109635627A CN 109635627 A CN109635627 A CN 109635627A CN 201811236301 A CN201811236301 A CN 201811236301A CN 109635627 A CN109635627 A CN 109635627A
- Authority
- CN
- China
- Prior art keywords
- text
- sequence
- bill picture
- picture
- text filed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/412—Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
Abstract
The invention discloses pictorial information extracting method, device, computer equipment and storage mediums.It goes forward side by side the adjustment of line skew and illumination this method comprises: obtaining bill picture to be identified, bill picture after being pre-processed；Identification obtains after pretreatment included multiple text filed in bill picture；Obtain it is multiple it is text filed in each text filed space coordinate, concatenated its corresponding vector to obtain text box sequence by splicing sequence；The input that text box sequence is made to sequence marking model obtains the corresponding subsequence in region to be extracted；By text filed carry out text identification corresponding to the corresponding subsequence in region to be extracted, the corresponding text information in region to be extracted is obtained.This method is not necessarily to carry out text extraction and identification to all text information frames of complicated bill, without the association successively calculated between text, reduce calculation amount, and the training of the bill picture labeled data of various angles, distortion can be had good robustness using image recognition technology.
Description
Technical field
The present invention relates to image identification technical fields more particularly to a kind of pictorial information extracting method, device, computer to set
Standby and storage medium.
Background technique
Currently, because of the complicated multiplicity of its page text of complicated bill arrangement relationship, frequently with following extraction process:
1) text filed mark: all text filed in mark bill picture；
2) content of text is extracted and is identified: to all text filed carry out Text regions, extracting text；
3) content of text is associated with: in conjunction with the position of text information and text in picture, carrying out content pass to text information
Connection.
The above process, which has the following deficiencies:, to be needed to extract text information one by one to text informations all in bill region, and
It is associated with one by one, leads to that computationally intensive, execution efficiency is low.And in a practical situation, the large amount of text information pair in bill
It is unrelated in task.
Summary of the invention
The embodiment of the invention provides a kind of pictorial information extracting method, device, computer equipment and storage mediums, it is intended to
Solution extracts text information to text informations all in bill region in the prior art one by one, and is associated with one by one, causes to locate
A large amount of unrelated content of text are managed, so that the entire problem that treatment process is computationally intensive, execution efficiency is low.
In a first aspect, the embodiment of the invention provides a kind of pictorial information extracting methods comprising:
Bill picture to be identified is obtained, the adjustment of deflection and illumination is carried out to bill picture to be identified, obtains pre- place
Bill picture after reason；
Identification obtains after pretreatment included multiple text filed in bill picture；
Obtain it is multiple it is text filed in each text filed space coordinate, will be by each text filed space coordinate pair
The vector answered is concatenated by preset splicing sequence, obtains text box sequence；
Sequence labelling model trained in advance is obtained, text box sequence is made to the input of sequence marking model, is obtained wait mention
Take the corresponding subsequence in region；
By text filed carry out text identification corresponding to the corresponding subsequence in region to be extracted, region pair to be extracted is obtained
The text information answered.
Second aspect, the embodiment of the invention provides a kind of pictorial information extraction elements comprising:
Picture pretreatment unit, for obtaining bill picture to be identified, to bill picture to be identified carry out deflection and
The adjustment of illumination, bill picture after being pre-processed；
Text filed recognition unit obtains after pretreatment included multiple text filed in bill picture for identification；
Text box retrieval unit, for obtain it is multiple it is text filed in each text filed space coordinate, will be by
The corresponding vector of each text filed space coordinate is concatenated by preset splicing sequence, obtains text box sequence；
Text box sequence is made sequence for obtaining sequence labelling model trained in advance by target subsequences acquiring unit
The input of marking model obtains the corresponding subsequence in region to be extracted；
Text information extraction unit, for by text filed carry out text corresponding to the corresponding subsequence in region to be extracted
Identification, obtains the corresponding text information in region to be extracted.
The third aspect, the embodiment of the present invention provide a kind of computer equipment again comprising memory, processor and storage
On the memory and the computer program that can run on the processor, the processor execute the computer program
Pictorial information extracting method described in the above-mentioned first aspect of Shi Shixian.
Fourth aspect, the embodiment of the invention also provides a kind of storage mediums, wherein the storage medium is stored with calculating
Machine program, the computer program make the processor execute the letter of picture described in above-mentioned first aspect when being executed by a processor
Cease extracting method.
The embodiment of the invention provides a kind of pictorial information extracting method, device, computer equipment and storage mediums.The party
Method by bill picture to be identified carry out deflection and illumination pretreatment, then identify it is therein multiple text filed, will be more
It is a it is text filed be converted into text box sequence after, text box sequence is made to the input of sequence marking model, obtains region to be extracted
Corresponding subsequence finally only needs the corresponding text filed progress text identification of sub-sequences that can obtain user's text of interest
This information.Without carrying out text extraction and identification to all text information frames of complicated bill in this method, without successively counting
The association between text is calculated, reduces calculation amount, and can have to the training of the bill picture labeled data of various angles, distortion
Good robustness.
Detailed description of the invention
Technical solution in order to illustrate the embodiments of the present invention more clearly, below will be to needed in embodiment description
Attached drawing is briefly described, it should be apparent that, drawings in the following description are some embodiments of the invention, general for this field
For logical technical staff, without creative efforts, it is also possible to obtain other drawings based on these drawings.
Fig. 1 is the flow diagram of pictorial information extracting method provided in an embodiment of the present invention；
Fig. 2 is the sub-process schematic diagram of pictorial information extracting method provided in an embodiment of the present invention；
Fig. 3 is another sub-process schematic diagram of pictorial information extracting method provided in an embodiment of the present invention；
Fig. 4 is another sub-process schematic diagram of pictorial information extracting method provided in an embodiment of the present invention；
Fig. 5 is another sub-process schematic diagram of pictorial information extracting method provided in an embodiment of the present invention；
Fig. 6 is the schematic block diagram of pictorial information extraction element provided in an embodiment of the present invention；
Fig. 7 is the subelement schematic block diagram of pictorial information extraction element provided in an embodiment of the present invention；
Fig. 8 is another subelement schematic block diagram of pictorial information extraction element provided in an embodiment of the present invention；
Fig. 9 is another subelement schematic block diagram of pictorial information extraction element provided in an embodiment of the present invention；
Figure 10 is another subelement schematic block diagram of pictorial information extraction element provided in an embodiment of the present invention；
Figure 11 is the schematic block diagram of computer equipment provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
It should be appreciated that ought use in this specification and in the appended claims, term " includes " and "comprising" instruction
Described feature, entirety, step, operation, the presence of element and/or component, but one or more of the other feature, whole is not precluded
Body, step, operation, the presence or addition of element, component and/or its set.
It is also understood that mesh of the term used in this description of the invention merely for the sake of description specific embodiment
And be not intended to limit the present invention.As description of the invention and it is used in the attached claims, unless on
Other situations are hereafter clearly indicated, otherwise " one " of singular, "one" and "the" are intended to include plural form.
It will be further appreciated that the term "and/or" used in description of the invention and the appended claims is
Refer to any combination and all possible combinations of one or more of associated item listed, and including these combinations.
Referring to Fig. 1, Fig. 1 is the flow diagram of pictorial information extracting method provided in an embodiment of the present invention, the picture
Information extracting method is applied in management server, and this method is held by the application software being installed in management server
Row, management server is the enterprise terminal for carrying out pictorial information extraction.
As shown in Figure 1, the method comprising the steps of S101~S105.
S101, bill picture to be identified is obtained, the adjustment of deflection and illumination is carried out to bill picture to be identified, is obtained
Bill picture after pretreatment.
In the present embodiment, bill picture to be identified is provided by business end, and is uploaded to management server by managing
The received carry out identifying processing of server.I.e. then business end obtains bill picture to be identified by the modes such as taking pictures or scanning
It is uploaded to management server, such as business personnel and the complicated bill of insurance contract, the vehicle new car quality certification of client's signature of company,
These complicated bills and simple bill (such as identity card picture can be considered a kind of simple bill, during identified point
Also the limited information such as name, identification card number, identity card validity period need to only be extracted) difference be that text information is more, it is required
The information of identification is also more.
Since business end obtains bill picture to be identified by the modes such as taking pictures or scanning, may because of shooting angle or
The problems such as light, causes the identification degree of picture not high, needs to pre-process picture to be identified at this time.
In one embodiment, as shown in Fig. 2, step S101 includes:
S1011, deflection adjustment is carried out to bill picture to be identified by Hough straight-line detection, schemed after obtaining deflection adjustment
Piece；
S1012, illumination adjustment, bill after being pre-processed are carried out to picture after deflection adjustment by histogram equalization
Piece.
In the present embodiment, Hough straight-line detection is realized using Hough transformation.Hough transformation is the warp in image transformation
One of allusion quotation means are mainly used to isolate the geometry (e.g., straight line, circle etc.) with certain same characteristic features from image.Suddenly
Husband converts the method for finding straight line and circle compared to other methods, can preferably reduce noise jamming.Classical Hough transformation
It is commonly used to detection straight line, circle, ellipse etc..
However in field of image processing, the pixel coordinate P (x, y) of image is known, and the corresponding pole of image is sat
R in mark, theta are then the variables that need to be found.Each (r, theta) value is if possible drawn, according to pixel coordinate P (x, y)
If value, then being just transformed into polar coordinates hough space system, this change from point to curve from image cartesian coordinate system
It is referred to as the Hough transformation of straight line.Transformation is that limited value is spaced equal part or cumulative grid by quantization Hough parameter space.
When Hough transformation algorithm starts, each pixel coordinate point P (x, y) is switched to above the curve point of (r, theta), is added to pair
The grid data point answered illustrates when a wave crest occurs with the presence of straight line.By Hough straight-line detection to bill picture into
The adjustment of line skew corrects it in order to subsequent Text region.
" central idea " of histogram equalization processing is the grey level histogram of original image from some for comparing concentration
Gray scale interval becomes being uniformly distributed in whole tonal ranges.Histogram equalization is exactly to carry out Nonlinear extension to image,
Image pixel value is redistributed, keeps the pixel quantity in certain tonal range roughly the same.Histogram equalization is exactly given
The histogram distribution of image is changed to the distribution of " uniform " distribution histogram.
The basic thought of histogram equalization is that the histogram of original graph is transformed to equally distributed form, is thus increased
The dynamic range of pixel gray value is added to can reach the effect of enhancing image overall contrast ratio.If original image is at (x, y)
The gray scale at place is f, and the image after changing is g, then can be expressed as reflecting the gray scale f at (x, y) to the method for image enhancement
It penetrates as g.The mapping function of image may be defined as in gray-level histogram equalizationization processing: g=EQ (f), this mapping function
EQ (f) must satisfy two conditions (the wherein number of greyscale levels that L is image):
A) EQ (f) is a monodrome single-increasing function within the scope of 0≤f≤L-1.This is to guarantee enhancing processing without beating
The gray scale arrangement order of random original image, each gray level of original image still keep the row from black to white (or from white to black) after the conversion
Column.
B) there is 0≤g≤L-1 for 0≤f≤L-1, this condition ensure that the consistent of transformation front and back gray value dynamic range
Property.
By carrying out the adjustment of deflection and illumination to bill picture to be identified, the identification success of picture can be effectively improved
Rate, and have good robustness.
S102, identification obtain after pretreatment included multiple text filed in bill picture.
In the present embodiment, after completing the pretreatment of bill picture to be identified, to after pretreatment in bill picture
It is included it is text filed identified one by one, carry out text filed identification at this time only to judge which region is including text
, the region for including text is carried out without specifically including which text identifies in text filed to identification, namely only
Positioning, and text filed rectangle frame is drawn for each.Due to only positioning the region of text, and all texts are not carried out
Identification, reduces calculation amount, improves data-handling efficiency.
It is in one embodiment, multiple text filed included by after identification acquisition pretreatment in bill picture, comprising:
Propose that network algorithm is examined bill picture after pretreatment by the spatial window of pre-set dimension by connection text
Survey, after pre-process in bill picture included by it is multiple text filed.
In the present embodiment, propose that (CTPN algorithm, full name are Connectionist to network algorithm by connection text
Text Proposal Network) text filed positioning is carried out to bill picture after pretreatment, the treatment process of CTPN algorithm is such as
Under:
A1) firstly, (VGG16 is the VGG convolutional neural networks model that Oxford University put forward in 2014 using VGG16
A mutation, possess 16 layers of model for one) extract feature as base net (i.e. base net), obtain conv5_3
The feature of (conv5_3 refers to the third convolutional layer inside the 5th convolution block) as characteristics of image (i.e. feature map),
The size of characteristics of image is W × H × C；
A2 sliding window then) is done on this feature map, window size is 3 × 3.Namely each window can obtain
The feature vector for being 3 × 3 × C to length, these feature vectors will be used to predict the anchor of position k (anchor's
Define similar with Faster RCNN) corresponding classification information, location information；
A3) by feature obtained in the previous step, i.e. the feature of 3*3*C (W*3*3*C) is input in a two-way LSTM, obtains
It is the output of W × 256 to length, then connects one 512 full articulamentum (fc layers), prepares output.
A4) mainly there are three outputs for output layer part.2k vertical coordinate (rectangular coordinate system), because
One anchor's is that the height (y-coordinate) of center and two values of height of rectangle frame indicate, so one with 2k
Output (notices that is exported here is the offset relative to anchor).
A5) fc layers of feature are input to three classification or return in layer.What second 2k scores was indicated is k
The classification information (be character or be not character) of anchor.First 2k vertical coordinate and third k
Side-refinement is the location information for returning k anchor.2k vertical coordinate indicate be
The height of bounding box and the y-axis coordinate (can determine up-and-down boundary) at center, what k side-refinement was indicated
The horizontal translation amount of bounding box.
A6 the textproposal (a part of line of text, it can be understood as an elongated rectangle) that) classification is obtained
It is merged into line of text.
S103, obtain it is multiple it is text filed in each text filed space coordinate, will be by each text filed space
The corresponding vector of coordinate is concatenated by preset splicing sequence, obtains text box sequence.
In the present embodiment, obtain it is multiple it is text filed in each text filed space coordinate when, due to each text
Region is the region of rectangle, therefore need to only obtain the coordinate and concatenation on text filed four vertex of each rectangle, can be obtained
To vector corresponding with this article one's respective area.It is identified by the coordinate on text filed four vertex to each rectangle
It realizes to all text filed positioning.
In one embodiment, as shown in figure 3, step S103 includes:
S1031, obtain it is multiple it is text filed in each text filed corresponding rectangular area four apex coordinates；
S1032, four apex coordinates of each rectangular area are pressed into preset sequential concatenation, obtained and each rectangular area
Corresponding vector；
S1033, the sequencing according to each rectangular area in bill picture to be identified, by each rectangular area pair
The vector answered sequentially is concatenated, and text box sequence is obtained.
In the present embodiment, obtain it is multiple it is text filed in each text filed corresponding rectangular area, can be denoted as respectively
First text filed-the N is text filed, and the coordinate of the first text filed top left corner apex is (x11, y11), the first text area
The coordinate of the upper right angular vertex in domain is (x12, y12), and the coordinate of the first text filed lower-left angular vertex is (x13, y13), the
The coordinate of one text filed bottom right angular vertex be (x14, y14), by (x11, y11), (x12, y12), (x13, y13) and
(x14, y14) sequentially concatenation obtains primary vector box1=[x11y11x12y12x13y13x14y14].
And so on the text filed corresponding N vector boxN=[xN1yN1xN2yN2xN3yN3xN4yN4] of N.By
One vector box1, secondary vector box2 ..., N vector boxN concatenated, obtain text box sequence
box1box2box3……boxN.By above-mentioned processing, simple text box sequence has been converted by bill picture to be processed,
Indicate after pretreatment included multiple text filed in bill picture by text box sequence.
S104, sequence labelling model trained in advance is obtained, text box sequence is made to the input of sequence marking model, obtained
The corresponding subsequence in region to be extracted.
In the present embodiment, sequence labelling model trained in advance is used for the interested region of user in text box sequence
It is extracted.Due to each type bill picture to be identified (insurance contract, vehicle of the business personnel of such as company and client's signature
The complexity such as new car quality certification bill) its text template for using is all the same, and the region that only some users fill in, which is only, needs weight
The region of point concern.
In one embodiment, as shown in figure 4, step S104 includes:
S1041, the history bill picture for obtaining multiple types classify according to the type of history bill picture is corresponding, obtain
Bill picture after classification；
S1042, by the bill picture of each classification carries out the adjustment of deflection and illumination in bill picture after classification, obtain
Bill picture after training data pretreatment；
It is text filed that S1043, identification obtain multiple training datas included in bill picture after training data pre-processes；
The text filed space coordinate of each training data during S1044, the multiple training datas of acquisition are text filed, will be by every
The corresponding vector of the text filed space coordinate of one training data is concatenated by preset splicing sequence, obtains training data text
This frame sequence；
S1045, selected vector in training data text box sequence is labeled, is obtained and training data text box
The corresponding subsequence to be extracted of sequence；
S1046, will make with the one-to-one training data text box sequence of bill picture after the pretreatment of multiple training datas
For the input of initiation sequence marking model, the one-to-one sub- sequence to be extracted of bill picture after being pre-processed with multiple training datas
Column are trained initiation sequence marking model, obtain sequence labelling model as output.
Wherein, sequence labelling model can be using any one in RNN, LSTM, bi-LSTM+crf model；Wherein,
RNN model is Recognition with Recurrent Neural Network model, and LSTM model is shot and long term memory models, and bi-LSTM+crf model is condition random
The composite model of field and bidirectional circulating neural network.
In the training sequence labelling model, a large amount of history bill picture can be first collected, it is advanced according to bill type
Row classification, such as it is divided into insurance contract class, vehicle new car quality certification class, later:
B1 it) first carries out such as the step in S101-S103, obtains text box sequence corresponding with history bill picture；
B2) user's vector interested that need to be extracted in each text box sequence is labeled, is obtained and text frame sequence
Arrange the corresponding subsequence as composed by the vector that need to be extracted；
For example, text box sequence box1box2box3 ... boxN is indicated, it is corresponding with text frame sequence by that need to mention
Subsequence composed by the vector taken can be indicated with box6box7 ... boxN-1 namely subsequence is the son of text box sequence
Collection.
It B3, will be with) using text box sequence corresponding with history bill picture as the input of initiation sequence marking model
Output of the corresponding subsequence of text box sequence as initiation sequence marking model, is trained initiation sequence marking model,
Obtain sequence labelling model.
By the above process, sequence labelling model corresponding with type bill picture to be identified can be obtained.Later
The sequence labelling model that training obtains may be used for the extraction that area-of-interest is carried out to bill picture to be identified, without complete
Text carries out text identification, reduces calculation amount, improves the efficiency of text identification.
S105, by text filed carry out text identification corresponding to the corresponding subsequence in region to be extracted, obtain to be extracted
The corresponding text information in region.
In the present embodiment, when obtaining the corresponding subsequence in region to be extracted, need to correspond to lookup will be in the subsequence
Each vector is corresponding text filed, by it is above-mentioned it is text filed using Text region model carry out text extraction, can be obtained to
Extract the corresponding text information in region.
For example, can (this article one's respective area can to text filed using CRNN model (i.e. convolution loop neural network model)
Picture with the part being considered as in bill picture to be identified) it is identified, it obtains that the corresponding text information in region need to be extracted.
In one embodiment, as shown in figure 5, step S105 includes:
S1051, vector included in the corresponding subsequence in region to be extracted is obtained；
S1052, each vector corresponding rectangular area in bill picture is obtained；
S1053, text in rectangular area is identified by the convolution loop neural network model for text identification,
Obtain the corresponding text information in region to be extracted.
In the present embodiment, after the corresponding subsequence in acquisition region to be extracted, you can learn that bill picture to be identified
In which region it is corresponding it is text filed in content of text need to be extracted, only need at this time for these it is specified it is text filed into
Row text identification is avoided to all text filed carry out Text regions with extracting text information to extract text, drop
Low calculation amount, improves treatment effeciency.
Without carrying out text extraction and identification to all text information frames of complicated bill in this method, without successively counting
The association between text is calculated, reduces calculation amount, and can have to the training of the bill picture labeled data of various angles, distortion
Good robustness.
The embodiment of the present invention also provides a kind of pictorial information extraction element, and the pictorial information extraction element is aforementioned for executing
Any embodiment of pictorial information extracting method.Specifically, referring to Fig. 6, Fig. 6 is pictorial information provided in an embodiment of the present invention
The schematic block diagram of extraction element.The pictorial information extraction element 100 can be configured in management server or terminal.
As shown in fig. 6, pictorial information extraction element 100 includes picture pretreatment unit 101, text filed recognition unit
102, text box retrieval unit 103, target subsequences acquiring unit 104 and text information extraction unit 105.
Picture pretreatment unit 101 carries out deflection to bill picture to be identified for obtaining bill picture to be identified
With the adjustment of illumination, bill picture after being pre-processed.
In the present embodiment, bill picture to be identified is provided by business end, and is uploaded to management server by managing
The received carry out identifying processing of server.I.e. then business end obtains bill picture to be identified by the modes such as taking pictures or scanning
It is uploaded to management server, such as business personnel and the complicated bill of insurance contract, the vehicle new car quality certification of client's signature of company,
These complicated bills and simple bill (such as identity card picture can be considered a kind of simple bill, during identified point
Also the limited information such as name, identification card number, identity card validity period need to only be extracted) difference be that text information is more, it is required
The information of identification is also more.
Since business end obtains bill picture to be identified by the modes such as taking pictures or scanning, may because of shooting angle or
The problems such as light, causes the identification degree of picture not high, needs to pre-process picture to be identified at this time.
In one embodiment, as shown in fig. 7, picture pretreatment unit 101 includes:
Picture deflection adjustment unit 1011, for carrying out deflection tune to bill picture to be identified by Hough straight-line detection
It is whole, obtain picture after deflection adjustment；
Picture illumination adjustment 1012 is obtained for carrying out illumination adjustment to picture after deflection adjustment by histogram equalization
Bill picture after to pretreatment.
In the present embodiment, Hough straight-line detection is realized using Hough transformation.Hough transformation is the warp in image transformation
One of allusion quotation means are mainly used to isolate the geometry (e.g., straight line, circle etc.) with certain same characteristic features from image.Suddenly
Husband converts the method for finding straight line and circle compared to other methods, can preferably reduce noise jamming.Classical Hough transformation
It is commonly used to detection straight line, circle, ellipse etc..
However in field of image processing, the pixel coordinate P (x, y) of image is known, and the corresponding pole of image is sat
R in mark, theta are then the variables that need to be found.Each (r, theta) value is if possible drawn, according to pixel coordinate P (x, y)
If value, then being just transformed into polar coordinates hough space system, this change from point to curve from image cartesian coordinate system
It is referred to as the Hough transformation of straight line.Transformation is that limited value is spaced equal part or cumulative grid by quantization Hough parameter space.
When Hough transformation algorithm starts, each pixel coordinate point P (x, y) is switched to above the curve point of (r, theta), is added to pair
The grid data point answered illustrates when a wave crest occurs with the presence of straight line.By Hough straight-line detection to bill picture into
The adjustment of line skew corrects it in order to subsequent Text region.
" central idea " of histogram equalization processing is the grey level histogram of original image from some for comparing concentration
Gray scale interval becomes being uniformly distributed in whole tonal ranges.Histogram equalization is exactly to carry out Nonlinear extension to image,
Image pixel value is redistributed, keeps the pixel quantity in certain tonal range roughly the same.Histogram equalization is exactly given
The histogram distribution of image is changed to the distribution of " uniform " distribution histogram.
The basic thought of histogram equalization is that the histogram of original graph is transformed to equally distributed form, is thus increased
The dynamic range of pixel gray value is added to can reach the effect of enhancing image overall contrast ratio.If original image is at (x, y)
The gray scale at place is f, and the image after changing is g, then can be expressed as reflecting the gray scale f at (x, y) to the method for image enhancement
It penetrates as g.The mapping function of image may be defined as in gray-level histogram equalizationization processing: g=EQ (f), this mapping function
EQ (f) must satisfy two conditions (the wherein number of greyscale levels that L is image):
A) EQ (f) is a monodrome single-increasing function within the scope of 0≤f≤L-1.This is to guarantee enhancing processing without beating
The gray scale arrangement order of random original image, each gray level of original image still keep the row from black to white (or from white to black) after the conversion
Column.
B) there is 0≤g≤L-1 for 0≤f≤L-1, this condition ensure that the consistent of transformation front and back gray value dynamic range
Property.
By carrying out the adjustment of deflection and illumination to bill picture to be identified, the identification success of picture can be effectively improved
Rate, and have good robustness.
Text filed recognition unit 102 obtains multiple text areas included in bill picture after pre-processing for identification
Domain.
In the present embodiment, after completing the pretreatment of bill picture to be identified, to after pretreatment in bill picture
It is included it is text filed identified one by one, carry out text filed identification at this time only to judge which region is including text
, the region for including text is carried out without specifically including which text identifies in text filed to identification, namely only
Positioning, and text filed rectangle frame is drawn for each.Due to only positioning the region of text, and all texts are not carried out
Identification, reduces calculation amount, improves data-handling efficiency.
It is in one embodiment, multiple text filed included by after identification acquisition pretreatment in bill picture, comprising:
Propose that network algorithm is examined bill picture after pretreatment by the spatial window of pre-set dimension by connection text
Survey, after pre-process in bill picture included by it is multiple text filed.
In the present embodiment, propose that (CTPN algorithm, full name are network algorithm by connection text
ConnectionistTextProposal Network) text filed positioning is carried out to bill picture after pretreatment.
Text box retrieval unit 103, for obtain it is multiple it is text filed in each text filed space coordinate, will
It is concatenated by the corresponding vector of each text filed space coordinate by preset splicing sequence, obtains text box sequence.
In the present embodiment, obtain it is multiple it is text filed in each text filed space coordinate when, due to each text
Region is the region of rectangle, therefore need to only obtain the coordinate and concatenation on text filed four vertex of each rectangle, can be obtained
To vector corresponding with this article one's respective area.It is identified by the coordinate on text filed four vertex to each rectangle
It realizes to all text filed positioning.
In one embodiment, as shown in figure 8, text box retrieval unit 103 includes:
Rectangular area vertex acquiring unit 1031, for obtain it is multiple it is text filed in each text filed corresponding rectangle
Four apex coordinates in region；
Region vector acquiring unit 1032, for four apex coordinates of each rectangular area to be pressed preset sequence string
It connects, obtains vector corresponding with each rectangular area；
Vector concatenation unit 1033, for the sequencing according to each rectangular area in bill picture to be identified,
The corresponding vector in each rectangular area is sequentially concatenated, text box sequence is obtained.
In the present embodiment, obtain it is multiple it is text filed in each text filed corresponding rectangular area, can be denoted as respectively
First text filed-the N is text filed, and the coordinate of the first text filed top left corner apex is (x11, y11), the first text area
The coordinate of the upper right angular vertex in domain is (x12, y12), and the coordinate of the first text filed lower-left angular vertex is (x13, y13), the
The coordinate of one text filed bottom right angular vertex be (x14, y14), by (x11, y11), (x12, y12), (x13, y13) and
(x14, y14) sequentially concatenation obtains primary vector box1=[x11y11x12y12x13y13x14y14].
And so on the text filed corresponding N vector boxN=[xN1yN1xN2yN2xN3yN3xN4yN4] of N.By
One vector box1, secondary vector box2 ..., N vector boxN concatenated, obtain text box sequence
box1box2box3……boxN.By above-mentioned processing, simple text box sequence has been converted by bill picture to be processed,
Indicate after pretreatment included multiple text filed in bill picture by text box sequence.
Target subsequences acquiring unit 104 contributes a foreword text box sequence for obtaining sequence labelling model trained in advance
The input of column marking model obtains the corresponding subsequence in region to be extracted.
In the present embodiment, sequence labelling model trained in advance is used for the interested region of user in text box sequence
It is extracted.Due to each type bill picture to be identified (insurance contract, vehicle of the business personnel of such as company and client's signature
The complexity such as new car quality certification bill) its text template for using is all the same, and the region that only some users fill in, which is only, needs weight
The region of point concern.
In one embodiment, as shown in figure 9, target subsequences acquiring unit 104 includes:
History picture classification unit 1041, for obtaining the history bill picture of multiple types, according to history bill picture
The corresponding classification of type, bill picture after being classified；
History picture pretreatment unit 1042, for carrying out the bill picture of each classification in bill picture after classification
The adjustment of deflection and illumination obtains bill picture after training data pretreatment；
History text area acquisition unit 1043 obtains included in bill picture after training data pre-processes for identification
Multiple training datas it is text filed；
History text frame retrieval unit 1044, for obtain multiple training datas it is text filed in each training data
Text filed space coordinate, will be suitable by preset splicing by the corresponding vector of the text filed space coordinate of each training data
Sequence is concatenated, and training data text box sequence is obtained；
History mark unit 1045 is obtained for selected vector in training data text box sequence to be labeled
Subsequence to be extracted corresponding with training data text box sequence；
Sequence labelling model training unit 1046 is corresponded for bill picture after pre-processing with multiple training datas
Input of the training data text box sequence as initiation sequence marking model, bill after being pre-processed with multiple training datas
Subsequence to be extracted is trained initiation sequence marking model, obtains sequence labelling model piece as output correspondingly.
Wherein, sequence labelling model can be using any one in RNN, LSTM, bi-LSTM+crf model；Wherein,
RNN model is Recognition with Recurrent Neural Network model, and LSTM model is shot and long term memory models, and bi-LSTM+crf model is condition random
The composite model of field and bidirectional circulating neural network.
By the above process, sequence labelling model corresponding with type bill picture to be identified can be obtained.Later
The sequence labelling model that training obtains may be used for the extraction that area-of-interest is carried out to bill picture to be identified, without complete
Text carries out text identification, reduces calculation amount, improves the efficiency of text identification.
Text information extraction unit 105, for by text filed progress corresponding to the corresponding subsequence in region to be extracted
Text identification obtains the corresponding text information in region to be extracted.
In the present embodiment, when obtaining the corresponding subsequence in region to be extracted, need to correspond to lookup will be in the subsequence
Each vector is corresponding text filed, by it is above-mentioned it is text filed using Text region model carry out text extraction, can be obtained to
Extract the corresponding text information in region.
For example, can (this article one's respective area can to text filed using CRNN model (i.e. convolution loop neural network model)
Picture with the part being considered as in bill picture to be identified) it is identified, it obtains that the corresponding text information in region need to be extracted.
In one embodiment, as shown in Figure 10, text information extraction unit 105 includes:
Subsequence vector extraction unit 1051, for obtaining vector included in the corresponding subsequence in region to be extracted；
Rectangular area extraction unit 1052, for obtaining each vector corresponding rectangular area in bill picture；
Rectangular area Text Feature Extraction unit 1053, for passing through the convolution loop neural network model pair for text identification
Text is identified in rectangular area, obtains the corresponding text information in region to be extracted.
In the present embodiment, after the corresponding subsequence in acquisition region to be extracted, you can learn that bill picture to be identified
In which region it is corresponding it is text filed in content of text need to be extracted, only need at this time for these it is specified it is text filed into
Row text identification is avoided to all text filed carry out Text regions with extracting text information to extract text, drop
Low calculation amount, improves treatment effeciency.
Without carrying out text extraction and identification to all text information frames of complicated bill in the device, without successively counting
The association between text is calculated, reduces calculation amount, and can have to the training of the bill picture labeled data of various angles, distortion
Good robustness.
Above-mentioned pictorial information extraction element can be implemented as the form of computer program, which can such as scheme
It is run in computer equipment shown in 11.
Figure 11 is please referred to, Figure 11 is the schematic block diagram of computer equipment provided in an embodiment of the present invention.
Refering to fig. 11, which includes processor 502, memory and the net connected by system bus 501
Network interface 505, wherein memory may include non-volatile memory medium 503 and built-in storage 504.
The non-volatile memory medium 503 can storage program area 5031 and computer program 5032.The computer program
5032 are performed, and processor 502 may make to execute pictorial information extracting method.
The processor 502 supports the operation of entire computer equipment 500 for providing calculating and control ability.
The built-in storage 504 provides environment for the operation of the computer program 5032 in non-volatile memory medium 503, should
When computer program 5032 is executed by processor 502, processor 502 may make to execute pictorial information extracting method.
The network interface 505 is for carrying out network communication, such as the transmission of offer data information.Those skilled in the art can
To understand, structure shown in Figure 11, only the block diagram of part-structure relevant to the present invention program, is not constituted to this hair
The restriction for the computer equipment 500 that bright scheme is applied thereon, specific computer equipment 500 may include than as shown in the figure
More or fewer components perhaps combine certain components or with different component layouts.
Wherein, the processor 502 is for running computer program 5032 stored in memory, to realize following function
Can: bill picture to be identified is obtained, the adjustment of deflection and illumination, ticket after being pre-processed are carried out to bill picture to be identified
According to picture；Identification obtains after pretreatment included multiple text filed in bill picture；Obtain it is multiple it is text filed in it is each
Text filed space coordinate will be gone here and there by the corresponding vector of each text filed space coordinate by preset splicing sequence
It connects, obtains text box sequence；Sequence labelling model trained in advance is obtained, text box sequence is made into the defeated of sequence marking model
Enter, obtains the corresponding subsequence in region to be extracted；By the text of text filed progress corresponding to the corresponding subsequence in region to be extracted
This identification obtains the corresponding text information in region to be extracted.
In one embodiment, processor 502 is executing the adjustment that deflection and illumination are carried out to bill picture to be identified, obtains
After to pretreatment when the step of bill picture, perform the following operations: by Hough straight-line detection to bill picture to be identified into
Line skew adjustment obtains picture after deflection adjustment；Illumination adjustment is carried out to picture after deflection adjustment by histogram equalization, is obtained
Bill picture after to pretreatment.
In one embodiment, the multiple texts included in bill picture after executing identification acquisition pretreatment of processor 502
It when the step of one's respective area, performs the following operations: proposing network algorithm to bill picture after pretreatment by default by connection text
The spatial window of size is detected, included multiple text filed in bill picture after being pre-processed.
In one embodiment, processor 502 execute obtain it is multiple it is text filed in each text filed space coordinate,
It will be concatenated by the corresponding vector of each text filed space coordinate by preset splicing sequence, obtain text box sequence
When step, perform the following operations: obtain it is multiple it is text filed in four vertex of each text filed corresponding rectangular area sit
Mark；Four apex coordinates of each rectangular area are pressed into preset sequential concatenation, obtain vector corresponding with each rectangular area；
According to sequencing of each rectangular area in bill picture to be identified, by the corresponding vector in each rectangular area sequentially into
Row concatenation, obtains text box sequence.
In one embodiment, processor 502 is also held before executing the step of obtaining sequence labelling model trained in advance
The following operation of row: obtaining the history bill picture of multiple types, classifies according to the type of history bill picture is corresponding, is classified
Bill picture afterwards；By the bill picture of each classification carries out the adjustment of deflection and illumination in bill picture after classification, instructed
Bill picture after white silk data prediction；Identification obtains multiple training datas included in bill picture after training data pre-processes
It is text filed；Obtain multiple training datas it is text filed in the text filed space coordinate of each training data, will be by each instruction
The corresponding vector of space coordinate for practicing data text region is concatenated by preset splicing sequence, obtains training data text box
Sequence；Selected vector in training data text box sequence is labeled, is obtained corresponding with training data text box sequence
Subsequence to be extracted；The one-to-one training data text box sequence of bill picture is made after pre-processing with multiple training datas
For the input of initiation sequence marking model, the one-to-one sub- sequence to be extracted of bill picture after being pre-processed with multiple training datas
Column are trained initiation sequence marking model, obtain sequence labelling model as output；Wherein the sequence labelling model is
The composite model of Recognition with Recurrent Neural Network model or shot and long term memory models or condition random field and bidirectional circulating neural network.
In one embodiment, processor 502 execute will be text filed corresponding to the corresponding subsequence in region to be extracted
It carries out text identification to perform the following operations when obtaining the step of the corresponding text information in region to be extracted: obtaining region to be extracted
Included vector in corresponding subsequence；Obtain each vector corresponding rectangular area in bill picture；By for text
The convolution loop neural network model of this identification identifies text in rectangular area, obtains the corresponding text in region to be extracted
Information.
It will be understood by those skilled in the art that the embodiment of computer equipment shown in Figure 11 is not constituted to computer
The restriction of equipment specific composition, in other embodiments, computer equipment may include components more more or fewer than diagram, or
Person combines certain components or different component layouts.For example, in some embodiments, computer equipment can only include depositing
Reservoir and processor, in such embodiments, the structure and function of memory and processor are consistent with embodiment illustrated in fig. 11,
Details are not described herein.
It should be appreciated that in embodiments of the present invention, processor 502 can be central processing unit (Central
Processing Unit, CPU), which can also be other general processors, digital signal processor (Digital
Signal Processor, DSP), specific integrated circuit (Application Specific Integrated Circuit,
ASIC), ready-made programmable gate array (Field-Programmable GateArray, FPGA) or other programmable logic devices
Part, discrete gate or transistor logic, discrete hardware components etc..Wherein, general processor can be microprocessor or
The processor is also possible to any conventional processor etc..
Storage medium is provided in another embodiment of the invention.The storage medium can be that non-volatile computer can
Read storage medium.The storage medium is stored with computer program, and following step is realized when wherein computer program is executed by processor
It is rapid: to obtain bill picture to be identified, the adjustment of deflection and illumination, ticket after being pre-processed are carried out to bill picture to be identified
According to picture；Identification obtains after pretreatment included multiple text filed in bill picture；Obtain it is multiple it is text filed in it is each
Text filed space coordinate will be gone here and there by the corresponding vector of each text filed space coordinate by preset splicing sequence
It connects, obtains text box sequence；Sequence labelling model trained in advance is obtained, text box sequence is made into the defeated of sequence marking model
Enter, obtains the corresponding subsequence in region to be extracted；By the text of text filed progress corresponding to the corresponding subsequence in region to be extracted
This identification obtains the corresponding text information in region to be extracted.
In one embodiment, the adjustment that deflection and illumination are carried out to bill picture to be identified, after obtaining pretreatment
Bill picture, comprising: deflection adjustment is carried out to bill picture to be identified by Hough straight-line detection, is schemed after obtaining deflection adjustment
Piece；Illumination adjustment, bill picture after being pre-processed are carried out to picture after deflection adjustment by histogram equalization.
It is in one embodiment, multiple text filed included by after the identification acquisition pretreatment in bill picture, comprising:
Propose that network algorithm is detected bill picture after pretreatment by the spatial window of pre-set dimension by connection text, obtains pre-
It is included multiple text filed in bill picture after processing.
In one embodiment, it is described obtain it is multiple it is text filed in each text filed space coordinate, will be by each text
The corresponding vector of the space coordinate of one's respective area is concatenated by preset splicing sequence, obtains text box sequence, comprising: obtain more
It is a it is text filed in each text filed corresponding rectangular area four apex coordinates；By four vertex of each rectangular area
Coordinate presses preset sequential concatenation, obtains vector corresponding with each rectangular area；According to each rectangular area to be identified
The corresponding vector in each rectangular area is sequentially concatenated, obtains text box sequence by the sequencing in bill picture.
In one embodiment, before acquisition sequence labelling model trained in advance, further includes: obtain multiple types
History bill picture is classified according to the type of history bill picture is corresponding, bill picture after being classified；Bill after classifying
The bill picture of each classification carries out the adjustment of deflection and illumination in piece, obtains bill picture after training data pretreatment；Know
Multiple training datas that Huo Qu be not included in bill picture after training data pretreatment are text filed；Obtain multiple training datas
The text filed space coordinate of each training data in text filed, by the space coordinate pair text filed by each training data
The vector answered is concatenated by preset splicing sequence, obtains training data text box sequence；By training data text box sequence
In selected vector be labeled, obtain subsequence to be extracted corresponding with training data text box sequence；It will be with multiple instructions
Input of the one-to-one training data text box sequence of bill picture as initiation sequence marking model after white silk data prediction,
Subsequence to be extracted marks initiation sequence as exporting bill picture correspondingly after pre-processing with multiple training datas
Model is trained, and obtains sequence labelling model；Wherein the sequence labelling model is Recognition with Recurrent Neural Network model or shot and long term
The composite model of memory models or condition random field and bidirectional circulating neural network.
In one embodiment, described by text filed carry out text knowledge corresponding to the corresponding subsequence in region to be extracted
, do not obtain the corresponding text information in region to be extracted, comprising: obtain in the corresponding subsequence in region to be extracted it is included to
Amount；Obtain each vector corresponding rectangular area in bill picture；Pass through the convolution loop neural network for text identification
Model identifies text in rectangular area, obtains the corresponding text information in region to be extracted.
It is apparent to those skilled in the art that for convenience of description and succinctly, foregoing description is set
The specific work process of standby, device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
Those of ordinary skill in the art may be aware that unit described in conjunction with the examples disclosed in the embodiments of the present disclosure and algorithm
Step can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and software
Interchangeability generally describes each exemplary composition and step according to function in the above description.These functions are studied carefully
Unexpectedly the specific application and design constraint depending on technical solution are implemented in hardware or software.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
The scope of the present invention.
In several embodiments provided by the present invention, it should be understood that disclosed unit and method, it can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only logical function partition, there may be another division manner in actual implementation, can also will be with the same function
Unit set is at a unit, such as multiple units or components can be combined or can be integrated into another system or some
Feature can be ignored, or not execute.In addition, shown or discussed mutual coupling, direct-coupling or communication connection can
Be through some interfaces, the indirect coupling or communication connection of device or unit, be also possible to electricity, mechanical or other shapes
Formula connection.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.Some or all of unit therein can be selected to realize the embodiment of the present invention according to the actual needs
Purpose.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, is also possible to two or more units and is integrated in one unit.It is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in one storage medium.Based on this understanding, technical solution of the present invention is substantially in other words to existing
The all or part of part or the technical solution that technology contributes can be embodied in the form of software products, should
Computer software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be
Personal computer, server or network equipment etc.) execute all or part of step of each embodiment the method for the present invention
Suddenly.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), magnetic disk or
The various media that can store program code such as person's CD.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or replace
It changes, these modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with right
It is required that protection scope subject to.
Claims (10)
1. a kind of pictorial information extracting method characterized by comprising
Bill picture to be identified is obtained, the adjustment of deflection and illumination is carried out to bill picture to be identified, after obtaining pretreatment
Bill picture；
Identification obtains after pretreatment included multiple text filed in bill picture；
Obtain it is multiple it is text filed in each text filed space coordinate, will be corresponding by each text filed space coordinate
Vector is concatenated by preset splicing sequence, obtains text box sequence；
Sequence labelling model trained in advance is obtained, text box sequence is made to the input of sequence marking model, obtains area to be extracted
The corresponding subsequence in domain；
By text filed carry out text identification corresponding to the corresponding subsequence in region to be extracted, it is corresponding to obtain region to be extracted
Text information.
2. pictorial information extracting method according to claim 1, which is characterized in that it is described to bill picture to be identified into
The adjustment of line skew and illumination, bill picture after being pre-processed, comprising:
Deflection adjustment is carried out to bill picture to be identified by Hough straight-line detection, obtains picture after deflection adjustment；
Illumination adjustment, bill picture after being pre-processed are carried out to picture after deflection adjustment by histogram equalization.
3. pictorial information extracting method according to claim 1, which is characterized in that the identification obtains bill after pretreatment
It is multiple text filed included by picture, comprising:
Propose that network algorithm is detected bill picture after pretreatment by the spatial window of pre-set dimension by connection text, obtains
It is included multiple text filed in bill picture after to pretreatment.
4. pictorial information extracting method according to claim 1, which is characterized in that it is described obtain it is multiple it is text filed in it is every
One text filed space coordinate will be carried out by the corresponding vector of each text filed space coordinate by preset splicing sequence
Concatenation, obtains text box sequence, comprising:
Obtain it is multiple it is text filed in each text filed corresponding rectangular area four apex coordinates；
By four apex coordinates of each rectangular area press preset sequential concatenation, obtain it is corresponding with each rectangular area to
Amount；
According to sequencing of each rectangular area in bill picture to be identified, by the corresponding vector in each rectangular area according to
Sequence is concatenated, and text box sequence is obtained.
5. pictorial information extracting method according to claim 1, which is characterized in that the sequence labelling model is circulation mind
Composite model through network model or shot and long term memory models or condition random field and bidirectional circulating neural network；
Before acquisition sequence labelling model trained in advance, further includes:
The history bill picture for obtaining multiple types is classified according to the type of history bill picture is corresponding, bill after being classified
Picture；
By the bill picture of each classification carries out the adjustment of deflection and illumination in bill picture after classification, it is pre- to obtain training data
Bill picture after processing；
It is text filed that identification obtains multiple training datas included in bill picture after training data pre-processes；
Obtain multiple training datas it is text filed in the text filed space coordinate of each training data, will be by each training data
The corresponding vector of text filed space coordinate is concatenated by preset splicing sequence, obtains training data text box sequence；
Selected vector in training data text box sequence is labeled, is obtained corresponding with training data text box sequence
Subsequence to be extracted；
The one-to-one training data text box sequence of bill picture is as initiation sequence after pre-processing with multiple training datas
The input of marking model, will with multiple training datas pre-process after bill picture correspondingly subsequence to be extracted as defeated
Out, initiation sequence marking model is trained, obtains sequence labelling model.
6. pictorial information extracting method according to claim 4, which is characterized in that described by the corresponding son in region to be extracted
Text filed carry out text identification, obtains the corresponding text information in region to be extracted corresponding to sequence, comprising:
Obtain vector included in the corresponding subsequence in region to be extracted；
Obtain each vector corresponding rectangular area in bill picture；
Text in rectangular area is identified by the convolution loop neural network model for text identification, is obtained to be extracted
The corresponding text information in region.
7. a kind of pictorial information extraction element characterized by comprising
Picture pretreatment unit carries out deflection and illumination to bill picture to be identified for obtaining bill picture to be identified
Adjustment, bill picture after being pre-processed；
Text filed recognition unit obtains after pretreatment included multiple text filed in bill picture for identification；
Text box retrieval unit, for obtain it is multiple it is text filed in each text filed space coordinate, will be by each
The corresponding vector of text filed space coordinate is concatenated by preset splicing sequence, obtains text box sequence；
Text box sequence is made sequence labelling for obtaining sequence labelling model trained in advance by target subsequences acquiring unit
The input of model obtains the corresponding subsequence in region to be extracted；
Text information extraction unit, for by text filed carry out text knowledge corresponding to the corresponding subsequence in region to be extracted
Not, the corresponding text information in region to be extracted is obtained.
8. pictorial information extraction element according to claim 7, which is characterized in that the text box retrieval unit,
Include:
Rectangular area positioning unit, for obtain it is multiple it is text filed in each text filed corresponding rectangular area four tops
Point coordinate；
Region vector acquiring unit, for by four apex coordinates of each rectangular area press preset sequential concatenation, obtain with
The corresponding vector in each rectangular area；
Vector concatenation unit, for the sequencing according to each rectangular area in bill picture to be identified, by each square
The corresponding vector in shape region is sequentially concatenated, and text box sequence is obtained.
9. a kind of computer equipment, including memory, processor and it is stored on the memory and can be on the processor
The computer program of operation, which is characterized in that the processor realizes such as claim 1 to 6 when executing the computer program
Any one of described in pictorial information extracting method.
10. a kind of storage medium, which is characterized in that the storage medium is stored with computer program, and the computer program is worked as
The processor is set to execute such as pictorial information extracting method as claimed in any one of claims 1 to 6 when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811236301.9A CN109635627A (en) | 2018-10-23 | 2018-10-23 | Pictorial information extracting method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811236301.9A CN109635627A (en) | 2018-10-23 | 2018-10-23 | Pictorial information extracting method, device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109635627A true CN109635627A (en) | 2019-04-16 |
Family
ID=66066531
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811236301.9A Pending CN109635627A (en) | 2018-10-23 | 2018-10-23 | Pictorial information extracting method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109635627A (en) |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110162757A (en) * | 2019-04-29 | 2019-08-23 | 北京百度网讯科技有限公司 | A kind of tableau format extracting method and system |
| CN110210400A (en) * | 2019-06-03 | 2019-09-06 | 上海眼控科技股份有限公司 | A kind of form document detection method and equipment |
| CN110263790A (en) * | 2019-04-18 | 2019-09-20 | 汕头大学 | A kind of power plant's ammeter character locating and recognition methods based on convolutional neural networks |
| CN110751143A (en) * | 2019-09-26 | 2020-02-04 | 中电万维信息技术有限责任公司 | Electronic invoice information extraction method and electronic equipment |
| CN110866495A (en) * | 2019-11-14 | 2020-03-06 | 杭州睿琪软件有限公司 | Bill image recognition method, bill image recognition device, bill image recognition equipment, training method and storage medium |
| CN110866457A (en) * | 2019-10-28 | 2020-03-06 | 世纪保众(北京)网络科技有限公司 | Electronic insurance policy obtaining method and device, computer equipment and storage medium |
| CN110909733A (en) * | 2019-10-28 | 2020-03-24 | 世纪保众(北京)网络科技有限公司 | Template positioning method and device based on OCR picture recognition and computer equipment |
| CN111209865A (en) * | 2020-01-06 | 2020-05-29 | 中科鼎富（北京）科技发展有限公司 | File content extraction method and device, electronic equipment and storage medium |
| CN111241974A (en) * | 2020-01-07 | 2020-06-05 | 深圳追一科技有限公司 | Bill information acquisition method and device, computer equipment and storage medium |
| CN111291742A (en) * | 2020-02-10 | 2020-06-16 | 北京百度网讯科技有限公司 | Object recognition method and device, electronic equipment and storage medium |
| CN111353458A (en) * | 2020-03-10 | 2020-06-30 | 腾讯科技（深圳）有限公司 | Text box marking method and device and storage medium |
| CN111582085A (en) * | 2020-04-26 | 2020-08-25 | 中国工商银行股份有限公司 | Document shooting image identification method and device |
| CN111652232A (en) * | 2020-05-29 | 2020-09-11 | 泰康保险集团股份有限公司 | Bill identification method and device, electronic equipment and computer readable storage medium |
| CN111666937A (en) * | 2020-04-17 | 2020-09-15 | 广州多益网络股份有限公司 | Method and system for recognizing text in image |
| CN111709339A (en) * | 2020-06-09 | 2020-09-25 | 北京百度网讯科技有限公司 | Bill image recognition method, device, equipment and storage medium |
| CN111832423A (en) * | 2020-06-19 | 2020-10-27 | 北京邮电大学 | Bill information identification method, device and system |
| CN111858994A (en) * | 2019-04-26 | 2020-10-30 | 深圳市蓝灯鱼智能科技有限公司 | Character retrieval method and device |
| CN111950353A (en) * | 2020-06-30 | 2020-11-17 | 深圳市雄帝科技股份有限公司 | Seal text recognition method and device and electronic equipment |
| CN112036406A (en) * | 2020-11-05 | 2020-12-04 | 北京智源人工智能研究院 | Text extraction method and device for image document and electronic equipment |
| CN112101356A (en) * | 2020-09-27 | 2020-12-18 | 中国建设银行股份有限公司 | Method and device for positioning specific text in picture and storage medium |
| CN112257712A (en) * | 2020-10-29 | 2021-01-22 | 湖南星汉数智科技有限公司 | Train ticket image rectification method and device, computer device and computer readable storage medium |
| CN112541443A (en) * | 2020-12-16 | 2021-03-23 | 平安科技（深圳）有限公司 | Invoice information extraction method and device, computer equipment and storage medium |
| CN112818823A (en) * | 2021-01-28 | 2021-05-18 | 建信览智科技(北京)有限公司 | Text extraction method based on bill content and position information |
| CN113033534A (en) * | 2021-03-10 | 2021-06-25 | 北京百度网讯科技有限公司 | Method and device for establishing bill type identification model and identifying bill type |
| CN113112567A (en) * | 2021-04-16 | 2021-07-13 | 中国工商银行股份有限公司 | Method and device for generating editable flow chart, electronic equipment and storage medium |
| CN113657279A (en) * | 2021-08-18 | 2021-11-16 | 北京玖安天下科技有限公司 | Bill image layout analysis method and device |
| CN113688268A (en) * | 2021-08-31 | 2021-11-23 | 中国平安人寿保险股份有限公司 | Picture information extraction method and device, computer equipment and storage medium |
| WO2022142627A1 (en) * | 2020-12-28 | 2022-07-07 | 深圳壹账通智能科技有限公司 | Address information extraction method and apparatus, device and medium |
| CN114821568A (en) * | 2022-06-27 | 2022-07-29 | 深圳前海环融联易信息科技服务有限公司 | Menu element extraction method and device, computer equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108446621A (en) * | 2018-03-14 | 2018-08-24 | 平安科技（深圳）有限公司 | Bank slip recognition method, server and computer readable storage medium |
| WO2018157862A1 (en) * | 2017-03-02 | 2018-09-07 | 腾讯科技（深圳）有限公司 | Vehicle type recognition method and device, storage medium and electronic device |
-
2018
- 2018-10-23 CN CN201811236301.9A patent/CN109635627A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018157862A1 (en) * | 2017-03-02 | 2018-09-07 | 腾讯科技（深圳）有限公司 | Vehicle type recognition method and device, storage medium and electronic device |
| CN108446621A (en) * | 2018-03-14 | 2018-08-24 | 平安科技（深圳）有限公司 | Bank slip recognition method, server and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| ZHI TIAN ET.AL: "Detecting Text in Natural Image with Connectionist Text Proposal Network", ARXIV, pages 1 - 16 * |
Cited By (53)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263790A (en) * | 2019-04-18 | 2019-09-20 | 汕头大学 | A kind of power plant's ammeter character locating and recognition methods based on convolutional neural networks |
| CN111858994A (en) * | 2019-04-26 | 2020-10-30 | 深圳市蓝灯鱼智能科技有限公司 | Character retrieval method and device |
| CN110162757A (en) * | 2019-04-29 | 2019-08-23 | 北京百度网讯科技有限公司 | A kind of tableau format extracting method and system |
| CN110162757B (en) * | 2019-04-29 | 2023-08-18 | 北京百度网讯科技有限公司 | Table structure extraction method and system |
| CN110210400A (en) * | 2019-06-03 | 2019-09-06 | 上海眼控科技股份有限公司 | A kind of form document detection method and equipment |
| CN110751143A (en) * | 2019-09-26 | 2020-02-04 | 中电万维信息技术有限责任公司 | Electronic invoice information extraction method and electronic equipment |
| CN110909733A (en) * | 2019-10-28 | 2020-03-24 | 世纪保众(北京)网络科技有限公司 | Template positioning method and device based on OCR picture recognition and computer equipment |
| CN110866457A (en) * | 2019-10-28 | 2020-03-06 | 世纪保众(北京)网络科技有限公司 | Electronic insurance policy obtaining method and device, computer equipment and storage medium |
| CN110866495B (en) * | 2019-11-14 | 2022-06-28 | 杭州睿琪软件有限公司 | Bill image recognition method, bill image recognition device, bill image recognition equipment, training method and storage medium |
| CN110866495A (en) * | 2019-11-14 | 2020-03-06 | 杭州睿琪软件有限公司 | Bill image recognition method, bill image recognition device, bill image recognition equipment, training method and storage medium |
| CN111209865A (en) * | 2020-01-06 | 2020-05-29 | 中科鼎富（北京）科技发展有限公司 | File content extraction method and device, electronic equipment and storage medium |
| CN111241974A (en) * | 2020-01-07 | 2020-06-05 | 深圳追一科技有限公司 | Bill information acquisition method and device, computer equipment and storage medium |
| CN111241974B (en) * | 2020-01-07 | 2023-10-27 | 深圳追一科技有限公司 | Bill information acquisition method, device, computer equipment and storage medium |
| KR102610456B1 (en) * | 2020-02-10 | 2023-12-06 | 베이징 바이두 넷컴 사이언스 테크놀로지 컴퍼니 리미티드 | Object recognition methods and devices, electronic devices, and storage media |
| JP7389824B2 (en) | 2020-02-10 | 2023-11-30 | ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド | Object identification method and device, electronic equipment and storage medium |
| EP3979129A4 (en) * | 2020-02-10 | 2023-07-12 | Beijing Baidu Netcom Science Technology Co., Ltd. | Object recognition method and apparatus, and electronic device and storage medium |
| WO2021159843A1 (en) * | 2020-02-10 | 2021-08-19 | 北京百度网讯科技有限公司 | Object recognition method and apparatus, and electronic device and storage medium |
| JP2022536320A (en) * | 2020-02-10 | 2022-08-15 | ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド | Object identification method and device, electronic device and storage medium |
| CN111291742A (en) * | 2020-02-10 | 2020-06-16 | 北京百度网讯科技有限公司 | Object recognition method and device, electronic equipment and storage medium |
| KR20210113620A (en) * | 2020-02-10 | 2021-09-16 | 베이징 바이두 넷컴 사이언스 테크놀로지 컴퍼니 리미티드 | Object recognition method and device, electronic device, storage medium |
| CN111353458B (en) * | 2020-03-10 | 2023-08-18 | 腾讯科技（深圳）有限公司 | Text box labeling method, device and storage medium |
| CN111353458A (en) * | 2020-03-10 | 2020-06-30 | 腾讯科技（深圳）有限公司 | Text box marking method and device and storage medium |
| CN111666937A (en) * | 2020-04-17 | 2020-09-15 | 广州多益网络股份有限公司 | Method and system for recognizing text in image |
| CN111582085A (en) * | 2020-04-26 | 2020-08-25 | 中国工商银行股份有限公司 | Document shooting image identification method and device |
| CN111582085B (en) * | 2020-04-26 | 2023-10-10 | 中国工商银行股份有限公司 | Document shooting image recognition method and device |
| CN111652232A (en) * | 2020-05-29 | 2020-09-11 | 泰康保险集团股份有限公司 | Bill identification method and device, electronic equipment and computer readable storage medium |
| CN111652232B (en) * | 2020-05-29 | 2023-08-22 | 泰康保险集团股份有限公司 | Bill identification method and device, electronic equipment and computer readable storage medium |
| CN111709339A (en) * | 2020-06-09 | 2020-09-25 | 北京百度网讯科技有限公司 | Bill image recognition method, device, equipment and storage medium |
| US11854246B2 (en) | 2020-06-09 | 2023-12-26 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method, apparatus, device and storage medium for recognizing bill image |
| EP3836016A1 (en) * | 2020-06-09 | 2021-06-16 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Recognizing invoice images |
| KR102612295B1 (en) * | 2020-06-09 | 2023-12-12 | 베이징 바이두 넷컴 사이언스 앤 테크놀로지 코., 엘티디. | Bill image recognition method, device, apparatus, and storage medium |
| CN111709339B (en) * | 2020-06-09 | 2023-09-19 | 北京百度网讯科技有限公司 | Bill image recognition method, device, equipment and storage medium |
| KR20210152931A (en) * | 2020-06-09 | 2021-12-16 | 베이징 바이두 넷컴 사이언스 앤 테크놀로지 코., 엘티디. | Bill image recognition method, device, apparatus, and storage medium |
| CN111832423A (en) * | 2020-06-19 | 2020-10-27 | 北京邮电大学 | Bill information identification method, device and system |
| CN111950353A (en) * | 2020-06-30 | 2020-11-17 | 深圳市雄帝科技股份有限公司 | Seal text recognition method and device and electronic equipment |
| CN111950353B (en) * | 2020-06-30 | 2024-04-19 | 深圳市雄帝科技股份有限公司 | Seal text recognition method and device and electronic equipment |
| CN112101356A (en) * | 2020-09-27 | 2020-12-18 | 中国建设银行股份有限公司 | Method and device for positioning specific text in picture and storage medium |
| CN112257712B (en) * | 2020-10-29 | 2024-02-27 | 湖南星汉数智科技有限公司 | Train ticket image alignment method and device, computer device and computer readable storage medium |
| CN112257712A (en) * | 2020-10-29 | 2021-01-22 | 湖南星汉数智科技有限公司 | Train ticket image rectification method and device, computer device and computer readable storage medium |
| CN112036406A (en) * | 2020-11-05 | 2020-12-04 | 北京智源人工智能研究院 | Text extraction method and device for image document and electronic equipment |
| CN112541443A (en) * | 2020-12-16 | 2021-03-23 | 平安科技（深圳）有限公司 | Invoice information extraction method and device, computer equipment and storage medium |
| WO2022142627A1 (en) * | 2020-12-28 | 2022-07-07 | 深圳壹账通智能科技有限公司 | Address information extraction method and apparatus, device and medium |
| CN112818823B (en) * | 2021-01-28 | 2024-04-12 | 金科览智科技(北京)有限公司 | Text extraction method based on bill content and position information |
| CN112818823A (en) * | 2021-01-28 | 2021-05-18 | 建信览智科技(北京)有限公司 | Text extraction method based on bill content and position information |
| CN113033534B (en) * | 2021-03-10 | 2023-07-25 | 北京百度网讯科技有限公司 | Method and device for establishing bill type recognition model and recognizing bill type |
| CN113033534A (en) * | 2021-03-10 | 2021-06-25 | 北京百度网讯科技有限公司 | Method and device for establishing bill type identification model and identifying bill type |
| CN113112567A (en) * | 2021-04-16 | 2021-07-13 | 中国工商银行股份有限公司 | Method and device for generating editable flow chart, electronic equipment and storage medium |
| CN113657279A (en) * | 2021-08-18 | 2021-11-16 | 北京玖安天下科技有限公司 | Bill image layout analysis method and device |
| CN113657279B (en) * | 2021-08-18 | 2023-05-05 | 北京玖安天下科技有限公司 | Bill image layout analysis method and device |
| CN113688268A (en) * | 2021-08-31 | 2021-11-23 | 中国平安人寿保险股份有限公司 | Picture information extraction method and device, computer equipment and storage medium |
| CN113688268B (en) * | 2021-08-31 | 2024-04-02 | 中国平安人寿保险股份有限公司 | Picture information extraction method, device, computer equipment and storage medium |
| CN114821568A (en) * | 2022-06-27 | 2022-07-29 | 深圳前海环融联易信息科技服务有限公司 | Menu element extraction method and device, computer equipment and storage medium |
| CN114821568B (en) * | 2022-06-27 | 2022-09-16 | 深圳前海环融联易信息科技服务有限公司 | Menu element extraction method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109635627A (en) | Pictorial information extracting method, device, computer equipment and storage medium | |
| CN109508681B (en) | Method and device for generating human body key point detection model | |
| KR102629380B1 (en) | Method for Distinguishing a Real Three-Dimensional Object from a Two-Dimensional Spoof of the Real Object | |
| Fang et al. | Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum | |
| US8792722B2 (en) | Hand gesture detection | |
| CN111860398B (en) | Remote sensing image target detection method and system and terminal equipment | |
| CN111291629A (en) | Method and device for recognizing text in image, computer equipment and computer storage medium | |
| CN111524106A (en) | Skull fracture detection and model training method, device, equipment and storage medium | |
| CN110363084A (en) | A kind of class state detection method, device, storage medium and electronics | |
| CN109472193A (en) | Method for detecting human face and device | |
| CN115457531A (en) | Method and device for recognizing text | |
| CN108846385B (en) | Image identification and correction method and device based on convolution-deconvolution neural network | |
| CN109242796A (en) | Character image processing method, device, electronic equipment and computer storage medium | |
| CN108288064B (en) | Method and device for generating pictures | |
| CN110599455A (en) | Display screen defect detection network model, method and device, electronic equipment and storage medium | |
| CN109816694A (en) | Method for tracking target, device and electronic equipment | |
| CN117597703A (en) | Multi-scale converter for image analysis | |
| CN111931783A (en) | Training sample generation method, machine-readable code identification method and device | |
| CN113160231A (en) | Sample generation method, sample generation device and electronic equipment | |
| CN109492601A (en) | Face comparison method and device, computer-readable medium and electronic equipment | |
| CN112085094A (en) | Document image reproduction detection method and device, computer equipment and storage medium | |
| CN109285181A (en) | The method and apparatus of image for identification | |
| CN116311214A (en) | License plate recognition method and device | |
| CN112651333B (en) | Silence living body detection method, silence living body detection device, terminal equipment and storage medium | |
| CN113516697A (en) | Image registration method and device, electronic equipment and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |