CN108320290A - Target Photo extracts antidote and device, computer equipment and recording medium - Google Patents
Target Photo extracts antidote and device, computer equipment and recording medium Download PDFInfo
- Publication number
- CN108320290A CN108320290A CN201711483213.4A CN201711483213A CN108320290A CN 108320290 A CN108320290 A CN 108320290A CN 201711483213 A CN201711483213 A CN 201711483213A CN 108320290 A CN108320290 A CN 108320290A
- Authority
- CN
- China
- Prior art keywords
- coordinate
- quadrangle
- units
- vertex
- coordinates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000000729 antidote Substances 0.000 title abstract description 4
- 239000000284 extract Substances 0.000 title description 5
- 238000012549 training Methods 0.000 claims abstract description 54
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 33
- 238000003062 neural network model Methods 0.000 claims abstract description 32
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 9
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 9
- 238000000034 method Methods 0.000 claims description 39
- 238000012937 correction Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 14
- 239000011159 matrix material Substances 0.000 claims description 8
- 238000005260 corrosion Methods 0.000 claims description 6
- 230000007797 corrosion Effects 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000000605 extraction Methods 0.000 abstract description 17
- 238000010586 diagram Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 15
- 238000012545 processing Methods 0.000 description 6
- 238000003708 edge detection Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011017 operating method Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G06T5/70—
Abstract
The present invention relates to Target Photo extraction antidote and device, computer equipment and recording mediums.The Target Photo extraction antidote includes the following steps：Scene picture is predicted using multiple neural network models for customizing training to generate multiple target labels figures；The synthesis label figure on the vertex at four angles that can cover the quadrangle Target Photo is obtained based on the multiple target labels figure；The apex coordinate at four angles is obtained from the comprehensive label figure using clustering algorithm；The correspondence for determining four angles of four acquired apex coordinates and quadrangle, to adjust the sequence of four apex coordinates；And perspective transform is carried out to extract the quadrangle Target Photo and be corrected to it to the scene picture based on four apex coordinates after adjustment.
Description
Technical field
The invention belongs to technical field of image processing, it is related to for quadrangle Target Photo to be extracted and corrected from scene picture
Method and device, computer equipment, recording medium.
Background technology
In many business, need to extract required Target Photo from scene picture.Currently used for from scene picture
The method of Target Photo needed for extraction is mostly target detection and recognition methods based on opencv.For such as paper, bank
The rectangular pictures such as card, generally detect the side of target in the picture after binaryzation or Laplace transform using HoughLine
Boundary's straight line, to obtain rectangular angular coordinate to realize the extraction and correction of target image.But this method is anti-interference
Ability is weaker, for example, background and foreground separation be not obvious or foreground picture in comprising the excessive picture of longer color.Due to
It is difficult to correctly distinguish boundary, so bringing huge difficulty for Objective extraction.In addition, the error in straight-line detection is obtaining angle
It will be amplified when point position, the Target Photo needed for accurately extracting and correcting will be impacted.
Invention content
The present invention is one or more to overcome the above disadvantages or other disadvantages and completes, used technology
Scheme is as follows.
According to one aspect of the present invention, it provides a kind of for quadrangle Target Photo to be extracted and corrected from scene picture
Method, including：Step S1：Scene picture is predicted using multiple neural network models for customizing training multiple to generate
Target labels figure；Step S2：Four angles of the quadrangle Target Photo can be covered by being obtained based on the multiple target labels figure
Vertex synthesis label figure；Step S3：The vertex that four angles are obtained from the comprehensive label figure using clustering algorithm is sat
Mark；Step S4：The correspondence for determining four angles of four acquired apex coordinates and quadrangle, to adjust described four
The sequence of a apex coordinate；And step S5：Perspective change is carried out to the scene picture based on four apex coordinates after adjustment
It changes to extract the quadrangle Target Photo and be corrected to it.
Further, in an aspect in accordance with the invention, further include：Step S00：Utilize random background picture and instruction
Practice and generates training input picture and training label figure with quadrangle Target Photo；Step S01：Different parameters are set to build
Found the multiple neural network model；And step S02：Using the training with input picture and the training with label figure come
The multiple neural network model of training.
Further, in an aspect in accordance with the invention, the neural network model is HED models.
Further, in an aspect in accordance with the invention, the step S2 includes：Step S21：Retain the multiple
The all the points on the vertex at angle are predicted to be in target labels figure simultaneously；Step S22：The matrix core of certain size is taken to come to the institute
A little execute corrosion and expansive working；And step S23：The comprehensive mark is obtained based on the result of the step S21 and S22
Label figure.
Further, in an aspect in accordance with the invention, the step S3 includes：Step S31：Obtain the synthesis
The coordinate on all vertex for being judged as angle in label figure；And step S32：To the coordinate using the clustering algorithm to obtain
Obtain the apex coordinate at four angles.
Further, in an aspect in accordance with the invention, the clustering algorithm is kmeans algorithms.
Further, in an aspect in accordance with the invention, the step S4 includes：Step S41：For acquired
Each coordinate in four apex coordinates sums to obtain four coordinates with four vertex correspondences to its transverse and longitudinal coordinate
With；Step S42：Coordinate corresponding to minimum value and maximum value of four coordinates in is identified as to a left side for quadrangle
The coordinate of lower angular vertex and upper right angular vertex；Step S43：The abscissa of remaining two coordinates is compared；Step S44：Root
According to the comparison result of step S43, pair of the top left corner apex of the two coordinates and quadrangle and the coordinate of bottom right angular vertex is determined
It should be related to；And step S45：Four apex coordinates are correspondingly adjusted according to the result of the step S42, the step S44
Sequence.
Further, in an aspect in accordance with the invention, the step S5 includes：Step S51：Based on the adjustment
Rear four apex coordinates and predetermined correction representative points coordinate determine perspective transform operator；And step S52：
The quadrangle Target Photo is extracted to scene picture progress perspective transform using the perspective transform operator and to it
It is corrected.
Other side according to the invention provides a kind of for quadrangle Target Photo to be extracted and corrected from scene picture
Device, including：Unit the 1st is used to predict scene picture using multiple neural network models for customizing training
To generate multiple target labels figures；Unit the 2nd is used to that based on the acquisition of the multiple target labels figure the quadrangle can be covered
The synthesis label figure on the vertex at four angles of Target Photo；Unit the 3rd is used to use clustering algorithm from the comprehensive label figure
Obtain the apex coordinate at four angles；Unit the 4th is used to determine the described of four acquired apex coordinates and quadrangle
The correspondence at four angles, to adjust the sequence of four apex coordinates；And Unit the 5th, it is used for based on after adjustment
The perspective transform relationship of four apex coordinates and corresponding object quadrangle vertex carries out perspective transform to carry to the scene picture
It takes the quadrangle Target Photo and it is corrected.
Further, according to another aspect of the present invention, further include：For utilizing random background picture and training
The unit that training inputs picture and training label figure is generated with quadrangle Target Photo；It is built for different parameters to be arranged
Found the unit of the multiple neural network model；And for utilizing training input picture and the training label figure
To train the unit of the multiple neural network model.
Further, according to another aspect of the present invention, the neural network model is HED models.
Further, according to another aspect of the present invention, Unit the 2nd includes：2A units are used to protect
Stay all the points on the vertex in the multiple target labels figure while being predicted to be angle；2B units are used to take certain size
Matrix core come to all the points execute corrosion and expansive working；And 2C units, it is used to be based on the 2A units
The comprehensive label figure is obtained with the result in 2B units.
Further, according to another aspect of the present invention, Unit the 3rd includes：3A units are used to obtain
Obtain the coordinate on all vertex for being judged as angle in the comprehensive label figure；And 3B units, it is used to answer the coordinate
With the clustering algorithm to obtain the apex coordinate at four angles.
Further, according to another aspect of the present invention, the clustering algorithm is kmeans algorithms.
Further, according to another aspect of the present invention, Unit the 4th includes：4A units are used for needle
It sums each coordinate pair its transverse and longitudinal coordinate in four acquired apex coordinates to obtain and four vertex correspondences
Four coordinates and；4B units are used for the coordinate point corresponding to the minimum value and maximum value by four coordinates in
It is not determined as the coordinate of the lower-left angular vertex and upper right angular vertex of quadrangle；4C units, by the abscissa of remaining two coordinates
It is compared；4D units, are used for according to the comparison result in the 4C units, determine the two coordinates and quadrangle
The correspondence of top left corner apex and the coordinate of bottom right angular vertex；And 4E units, it is used for according to the 4B units, institute
The result of 4D units is stated correspondingly to adjust the sequence of four apex coordinates.
Further, according to another aspect of the present invention, Unit the 5th includes：5A units：It is used for base
Four apex coordinates and predetermined correction representative points coordinate after the adjustment determine perspective transform operator；With
And 5B units：Perspective transform is carried out to extract the quadrangle target to the scene picture using the perspective transform operator
Picture simultaneously corrects it.
Another aspect according to the invention, provides a kind of computer equipment, including memory, processor and is stored in
On reservoir and the computer program that can run on a processor, which is characterized in that the processor is realized when executing described program
The step of method according to an aspect of the present invention.
Another aspect according to the invention provides a kind of computer readable storage medium, is stored thereon with computer journey
Sequence, which is characterized in that the program is computer-executed the step of to realize method according to an aspect of the present invention.
Compared with the existing technology, the present invention can obtain the one or more of following advantageous effect：
1) in accordance with the invention it is possible to efficiently extract quadrangle target figure from any background (especially complicated background)
Piece, moreover, having stronger anti-interference ability to variations such as perspective, displacements；
2) according to the present invention, label figure is generated by using multiple models, so as to reduce error and influence of noise；
3) portable strong according to the present invention, corresponding training can be used for different application scenarios and business
Data are to obtain the model of corresponding meet demand.
Description of the drawings
Fig. 1 is according to embodiment of the present invention for quadrangle Target Photo to be extracted and corrected from scene picture
Method example flow diagram.
Fig. 2 is the generation schematic diagram of training input picture according to an embodiment of the invention.
Fig. 3 is training label map generalization schematic diagram according to an embodiment of the invention.
Fig. 4 is the exemplary plot of target labels figure and comprehensive label figure according to embodiment of the present invention.
Fig. 5 is the quadrangle Target Photo depicted according to coordinate correspondence relationship according to embodiment of the present invention
Profile generation schematic diagram.
Fig. 6 is the schematic diagram of extraction and correction quadrangle Target Photo according to embodiment of the present invention.
Fig. 7 is according to embodiment of the present invention for quadrangle Target Photo to be extracted and corrected from scene picture
Device example block diagram.
Fig. 8 is the computer equipment for executing method shown in Fig. 1 according to embodiment of the present invention
Example block diagram.
Specific implementation mode
Below with reference to attached drawing to of the present invention for quadrangle Target Photo to be extracted and corrected from scene picture
Method and device, computer equipment and recording medium are described in further detail.It should be noted that specific implementation below
Mode is exemplary rather than limitation, is intended to provide the basic understanding to the present invention, it is no intended to confirm the key of the present invention
Conclusive element or limit scope of the claimed.
This hair described below with reference to block diagram explanation, the block diagram and or flow chart of the method and apparatus of the embodiment of the present invention
It is bright.It will be understood that these flow charts illustrate and/or each frame and flow chart of block diagram illustrate and/or the combination of block diagram can be by
Computer program instructions are realized.These computer program instructions can be supplied to all-purpose computer, special purpose computer or its
The processor of its programmable data processing device is to constitute machine, so as to by computer or other programmable data processing devices
These instructions that processor executes are created for implementing these flow charts and/or frame and/or one or more flow diagram middle fingers
The component of fixed function/operation.
These computer program instructions can be stored in computer-readable memory, these instructions can indicate to calculate
Machine or other programmable processors realize function in a specific way, to be stored in these instructions in computer-readable memory
Constitute the making production of the instruction unit for the function/operation specified in one or more frames comprising implementing procedure figure and/or block diagram
Product.
These computer program instructions can be loaded on computer or other programmable data processors so that a system
The operating procedure of row executes on computer or other programmable processors, to constitute computer implemented process, so that meter
These instructions executed on calculation machine or other programmable data processors provide one for implementing this flowchart and or block diagram
Or in multiple frames specify functions or operations the step of.It is further noted that in some alternative realizations, function/behaviour shown in frame
Work can not be occurred by order shown in flow chart.For example, two frames shown successively actually can be executed essentially simultaneously
Or these frames can execute in reverse order sometimes, be specifically dependent upon involved function/operation.
Method and device according to the present invention for quadrangle Target Photo to be extracted and corrected from scene picture can be with
For under multiple business scene, for example, being carried from the scene picture acquired in the image acquiring devices such as camera, video camera
Take and correct the quadrangle Target Photo of such as identity card, bank card, member card, various files etc.Below with from scene graph
Illustrate the present invention for the Target Photo of extraction and correction bank card in piece, it will be understood by those skilled in the art that according to being carried
The type conversion for the Target Photo for taking and correcting, the method and apparatus of the extraction of following example and the Target Photo of correction bank card
It can be applied to the extraction and correction to other types of Target Photo in conversion.
Fig. 1 show according to embodiment of the present invention for from scene picture extract and correct quadrangle target
The flow chart of the method for picture.As shown in Figure 1, this method S100 includes the following steps：Utilize multiple nerves for customizing training
Network model is predicted scene picture to generate multiple target labels figures (step S1).
In one embodiment, as shown in Figure 1, the method S100 can also include the following steps：Based on the multiple mesh
Mark label figure obtains the synthesis label figure (step S2) on the vertex at four angles that can cover the quadrangle Target Photo.
In one embodiment, as shown in Figure 1, the method S100 can also include the following steps：Using clustering algorithm from
The comprehensive label figure obtains the apex coordinate (step S3) at four angles.
In one embodiment, as shown in Figure 1, the method S100 can also include the following steps：Determine acquired four
The correspondence at four angles of a apex coordinate and quadrangle, to adjust the sequence (step of four apex coordinates
S4)。
In one embodiment, as shown in Figure 1, the method S100 can also include the following steps：And after being based on adjustment
Four apex coordinates to the scene picture carry out perspective transform to extract the quadrangle Target Photo and be rectified to it
Positive (step S5).
Before above-mentioned steps S1 is described in detail to S5, first to how to obtain the instruction of multiple customization in step S1
The process of experienced neural network model is simply illustrated.
First, by the way of artificial synthesized to random background picture and training with quadrangle Target Photo handled come
Generate training input picture and training label figure (step S00).
In one embodiment, in step S00, a large amount of random pictures are chosen, they are cut respectively, to obtain with after
The input random background picture of the same size for the neural network model that face illustrates, for example, Background shown in Fig. 2.In turn,
A large amount of training quadrangle Target Photos are chosen, for example, bank card foreground picture shown in Fig. 2 is (with the bank of two bank cards
Card foreground picture is example).Following perspective transforms are carried out to these bank card foreground pictures：
(1) initial position is randomly chosen to each bank card foreground picture, is extended to Background size, boundary is mended
0, and record the initial coordinate on four vertex；
(2) four vertex of random movement ensure four vertex all to obtain the new coordinate on four vertex in moving process
The boundary of Background is not spilt over, and ensures constant (the i.e. phase of lower-left, upper left, upper right, bottom right of geometry relative ranks on four vertex
To relationship)；
(3) perspective transform operator is obtained according to initial coordinate and new coordinate；
(4) perspective transform operator is utilized to realize perspective transform to the foreground picture after extension.It finally, will be after perspective transform
Bank card foreground picture is merged with Background to obtain the input picture that final training uses, i.e., training with input picture (such as such as
Composite diagram shown in Fig. 2).
In addition, the new coordinate after the random movement in above-mentioned (2) is preserved, and in the blank picture of no Background
On the new coordinate on four vertex depicted into the label figure used as final training, i.e. training label figure, for example, Fig. 3
Shown in label figure.
Then, different parameters are set to establish multiple (for example, n, wherein n is positive integer) neural network model (steps
S01).The neural network model can be HED models, still, it should be recognized by those skilled in the art that nerve herein
Network model is not limited to HED models, and the other types of neural network model with edge detection feature can also be applied to this,
For realizing essentially identical function or effect.
Then, it is used using multiple the described training input pictures obtained in above-mentioned steps S00 and the corresponding training
Label figure trains the n HED models (step S02).
In one embodiment, input picture is used to be input to as input picture each Zhang Xunlian obtained in step S00
Each in the n HED models generates corresponding label figure by the operation and processing inside model.Specifically, will
The label figure generated is compared with the corresponding training obtained in step S00 with label figure, and the difference between them is used
Loss function J is indicated, by minimizing loss function J by the parameter adjustment of used HED models at optimal value, to give birth to
At n different HED model of trained parameter.As loss function J, since the label data in Fig. 3 is extremely asymmetric, institute
With loss function J can be for example indicated with the numerical expression (1) of the cross entropy of following Weight：
Wherein, y is the label data in the label figure that final training uses,For in the label figure that is generated by model
Label data, p are positive label weight, and the label data is the matrix that value is (0,1), for example, the position where apex coordinate
It sets and is noted as label data 1, remaining position is noted as label data 0.
After obtaining n different HED model of trained parameter, to needing progress Target Photo extraction and correction
Scene picture is handled, specifically, by scene picture respectively by trained above-mentioned n HED models to generate n target
Label figure, for example, 5 target labels figures (a) (b) (c) (d) (e) (step S1) shown in Fig. 4.Using n model come simultaneously
Generate target labels figure be in order to reduce the influence of error and noise, wherein the specific value of n >=2 and n can according to demand and
Computing resource is set.
Next, being merged to this n (being in Fig. 4 5) target labels figures.Specifically, first, retain this 5
The all the points on the vertex at angle are predicted to be in target labels figure simultaneously, obtain result I1(step S21) then takes certain size
The matrix core of (for example, size is (2,2)) comes to I1A burn into expansion is executed successively to eliminate noise, obtains result I2(step
Rapid S22), then, in order to ensure that expansion will not add new point, by I1With I2It is multiplied, to obtain the energy as shown in (f) in Fig. 4
Cover the synthesis label figure (step S23) on the vertex at four angles of bank card picture.
Then, the apex coordinate (step S3) at four angles is obtained from the comprehensive label figure using clustering algorithm.Tool
Body, obtain the coordinate C on all vertex for being judged as angle in the comprehensive label figure as shown in (f) in Fig. 41(step S31),
Further, to the coordinate C1Using clustering algorithm to obtain the apex coordinate C (step S32) at four angles.The cluster
Algorithm can be kmeans algorithms, still, it should be recognized by those skilled in the art that clustering algorithm herein is without being limited thereto.
Since the sequence of the apex coordinate C at four angles obtained by above step is usually random, that is, it is uncertain with
The top left corner apex of bank card picture, the correspondence of upper right angular vertex, lower-left angular vertex, bottom right angular vertex, therefore, it is necessary to right
The apex coordinate C at four angles is adjusted with the clearly described correspondence (step S4).
In one embodiment, it is id to take final coordinate sequence, then, for every in four acquired apex coordinates
One coordinate, sum to its transverse and longitudinal coordinate with obtain with four coordinates of four vertex correspondences and, i.e. S=[S1, S2, S3,
S4] (step S41).By S1, S2, S3, the coordinate corresponding to minimum value and maximum value in S4 is identified as a left side for quadrangle
Lower angular vertex (corresponding id1) and upper right angular vertex (corresponding id3) coordinate, i.e. id1=argmin (S), id3=argmax (S),
Wherein argmin indicates that the serial number of minimum value, argmax indicate the serial number (step S42) of maximum value.Then, by remaining in C two
A coordinate is denoted asAnd serial number is denoted asIfThenIfThen
That is, x coordinate component smaller is id2, corresponding top left corner apex, x coordinate component the greater is id3, corresponding bottom right angular vertex (step
Rapid S43 and S44).Then, the profile diagram of bank card is depicted according to correct correspondence, for example, as shown in Figure 5.
By above-mentioned adjustment, four apex coordinates that final coordinate sequence is id are obtained.On the other hand, it is known that extraction and
The correction representative points coordinate C on target length and width i.e. four vertex of the quadrangle Target Photo after correction0, for example, C0=[[0,
0], [0,107], [170,107], [170,0]], with the lower-left angular vertex of the bank card picture after correction, top left corner apex,
Bottom right angular vertex, upper right angular vertex correspond.Representative points coordinate C is corrected by calculating0It is suitable with the final coordinate that is obtained
Sequence is id1、id2、id3、id4The transformation relations of four apex coordinates determine perspective transform operator M (step S51), then,
Using the perspective transform operator M come to the scene picture for needing the progress extraction of bank card picture and correction shown in the left sides Fig. 6
It carries out perspective transform and cuts out the bank card picture, as shown in the right sides Fig. 6 (step S52).
By above step, it can be extracted from any scene picture and correct required quadrangle Target Photo.
Then, illustrate shown in Fig. 1 to be used to that quadrangle to be extracted and corrected from scene picture for executing with reference to Fig. 7
The device of Target Photo.As shown in fig. 7, described device includes Unit the 1st to Unit the 5th.Although illustrate only in the figure 7 including
This 5 units, but described device can also include other units, it is preferable that it is used including the use of random background picture and training
Quadrangle Target Photo is described to establish to generate training input picture and the training unit of label figure, setting different parameters
The unit of multiple neural network models and using training input picture and the training with label figure come described in training
The unit of multiple neural network models.The neural network model can be HED models, and still, those skilled in the art should
It recognizes, neural network model herein is not limited to HED models, as long as the neural network mould with edge detection feature
Type each falls within protection scope of the present invention.
The function of the 1st unit to Unit the 5th is described in detail below.
1st unit 101 be for using it is multiple customize training neural network models to scene picture predicted with
Generate the unit of multiple target labels figures.
2nd unit 102 is for that can cover the quadrangle Target Photo based on the acquisition of the multiple target labels figure
The unit of the synthesis label figure on the vertex at four angles.Preferably, Unit the 2nd includes：For retaining the multiple target mark
The 2A units of all the points on the vertex at angle are predicted to be in label figure simultaneously, for taking the matrix core of certain size to come to the institute
A little execution corrosion is obtained with the 2B units of expansive working and based on the result in the 2A units and 2B units
The 2C units of the comprehensive label figure.
3rd unit 103 is the apex coordinate for using clustering algorithm to obtain from the comprehensive label figure four angles
Unit.Preferably, Unit the 3rd includes：For obtaining all vertex for being judged as angle in the comprehensive label figure
The 3A units of coordinate and apex coordinate for obtaining four angles using the clustering algorithm to the coordinate
3B units.Above-mentioned clustering algorithm can be kmeans algorithms, still, it should be recognized by those skilled in the art that herein
Clustering algorithm is without being limited thereto.
Unit the 4th is the correspondence at four angles for determining four acquired apex coordinates and quadrangle,
To adjust the unit of the sequence of four apex coordinates.Preferably, Unit the 4th includes：For for acquired four
Each coordinate pair its transverse and longitudinal coordinate in a apex coordinate sum with obtain with four coordinates of four vertex correspondences and
4A units；For by four coordinates and in minimum value and maximum value corresponding to coordinate be identified as quadrangle
Lower-left angular vertex and upper right angular vertex coordinate 4B units；For what the abscissa of remaining two coordinates was compared
4C units；For determining the top left corner apex of the two coordinates and quadrangle according to the comparison result in the 4C units
With the 4D units of the correspondence of the coordinate of bottom right angular vertex；And for according to the 4B units, the 4D units
Result correspondingly adjust the 4E units of the sequence of four apex coordinates.
Unit the 5th is for carrying out perspective transform to the scene picture to extract based on four apex coordinates after adjustment
The quadrangle Target Photo and the unit that it is corrected.Preferably, Unit the 5th includes：After the adjustment
Four apex coordinates and it is predetermined correction representative points coordinate come determine perspective transform operator 5A units and
For using the perspective transform operator to carry out perspective transform to the scene picture to extract the quadrangle Target Photo simultaneously
The 5B units that it is corrected.
Although before this for from scene picture extract and correct quadrangle Target Photo method and device reality
Apply and be illustrated centered on mode, but the present invention is not limited to these embodiments, the present invention can also be embodied as with
Under type：For execute the computer equipment of the above method either the mode of the computer program for executing the above method or
There is the computer-readable of the computer program for realizing the mode or record of the computer program of the function of above-mentioned apparatus
Recording medium mode.
Be shown in FIG. 8 according to embodiment of the present invention for executing shown in Fig. 1 from scene picture
The computer equipment of the method for extraction and correction quadrangle Target Photo.As shown in figure 8, computer equipment 200 includes memory
201 and processor 202.Although it is not shown, still computer equipment 200 further includes being stored on memory 201 and can handling
The computer program run on device 202.The processor realizes following steps when executing described program：It is instructed using multiple customization
Experienced neural network model is predicted scene picture to generate multiple target labels figures (step S1)；Based on the multiple mesh
Mark label figure obtains the synthesis label figure (step S2) on the vertex at four angles that can cover the quadrangle Target Photo；Using poly-
Class algorithm obtains the apex coordinate (step S3) at four angles from the comprehensive label figure；Determine that four acquired vertex are sat
The correspondence of mark and four angles of quadrangle, to adjust the sequence (step S4) of four apex coordinates；And base
Four apex coordinates after adjustment carry out perspective transform to extract the quadrangle Target Photo and right to the scene picture
It is corrected (step S5).
Other than above-mentioned steps S1 to S5, the processor 202 also realizes following steps when executing described program：It utilizes
Random background picture and training generate training input picture and training label figure (step with quadrangle Target Photo
S00)；Different parameters are set to establish the multiple neural network model (step S01)；And utilize training input figure
The multiple neural network model (step S02) is trained in piece and the training with label figure.
It should be noted that above-mentioned neural network model can be HED models, still, those skilled in the art should recognize
To know, neural network model herein is not limited to HED models, as long as the neural network model with edge detection feature,
Each fall within protection scope of the present invention.
Preferably, above-mentioned steps S2 includes：Retain the vertex in the multiple target labels figure while being predicted to be angle
All the points (step S21)；The matrix core of certain size is taken to execute corrosion and expansive working (step S22) to all the points；
And the comprehensive label figure (step S23) is obtained based on the result of the step S21 and S22.
Preferably, above-mentioned steps S3 includes：Obtain the coordinate on all vertex for being judged as angle in the comprehensive label figure
(step S31)；And apply the clustering algorithm to obtain the apex coordinate (step S32) at four angles on the coordinate.
It should be noted that above-mentioned clustering algorithm can be kmeans algorithms, still, those skilled in the art should recognize
It arrives, clustering algorithm herein is without being limited thereto.
Moreover it is preferred that above-mentioned steps S4 includes：It is right for each coordinate in four acquired apex coordinates
Its transverse and longitudinal coordinate is summed to obtain four coordinates and (step S41) with four vertex correspondences；By four coordinates and
In minimum value and maximum value corresponding to coordinate be identified as quadrangle lower-left angular vertex and upper right angular vertex coordinate
(step S42)；The abscissa of remaining two coordinates is compared (step S43)；According to the comparison result of step S43, determine
The correspondence (step S44) of the coordinate of the top left corner apex and bottom right angular vertex of the two coordinates and quadrangle；And according to
The step S42, the step S44 result correspondingly adjust the sequence (step S45) of four apex coordinates.
Preferably, above-mentioned steps S5 includes：Based on after the adjustment four apex coordinates and predetermined correction
Representative points coordinate determines perspective transform operator (step S51)；And using the perspective transform operator to the scene graph
Piece carries out perspective transform to extract the quadrangle Target Photo and be corrected (step S52) to it.
In addition, as described above, the present invention can also be implemented as a kind of recording medium, it is stored with wherein for making calculating
Machine executes the program of the method for quadrangle Target Photo to be extracted and corrected from scene picture shown in Fig. 1.
Here, as recording medium, disk class (for example, disk, CD etc.), card class can be used (for example, storage card, light-card
Deng), semiconductor memory class (for example, ROM, nonvolatile memory etc.), band class (for example, tape, cassette tape etc.) etc. it is each
The recording medium of kind mode.
By in these recording mediums record make computer execute the above embodiment in slave scene picture extraction and
It corrects the computer program of the method for quadrangle Target Photo or computer is made to realize the slave scene picture in the above embodiment
The computer program of the function of the device of extraction and correction quadrangle Target Photo simultaneously makes its circulation, so as to make the cheap of cost
Change and portability, versatility improve.
Moreover, loading aforementioned recording medium on computers, the computer recorded in the recording medium is read by computer
Program simultaneously stores in memory, the processor (CPU that computer has：Central Processing Unit (centres
Manage unit), MPU：Micro Processing Unit (microprocessing unit)) it reads the computer program from memory and executes,
The method of the extraction of the slave scene picture in the above embodiment and correction quadrangle Target Photo can be executed as a result, and can be realized
State the function of the slave scene picture extraction in embodiment and the device of correction quadrangle Target Photo.
For those of ordinary skill in the art it is to be appreciated that the present invention is not limited to above-mentioned embodiment, the present invention can be
Without departing from its spirit in range in the form of many other implement.Therefore, the example shown is considered as showing with embodiment
Meaning property and not restrictive, in the case where not departing from the spirit and scope of the present invention as defined in appended claims,
The present invention may cover various modification and replacement.
Claims (18)
1. a kind of method for being extracted from scene picture and correcting quadrangle Target Photo, which is characterized in that including：
Step S1：Scene picture is predicted using multiple neural network models for customizing training to generate multiple target marks
Label figure；
Step S2：The vertex at four angles that can cover the quadrangle Target Photo is obtained based on the multiple target labels figure
Comprehensive label figure；
Step S3：The apex coordinate at four angles is obtained from the comprehensive label figure using clustering algorithm；
Step S4：The correspondence for determining four angles of four acquired apex coordinates and quadrangle, described in adjustment
The sequence of four apex coordinates；And
Step S5：Perspective transform is carried out to extract the quadrangle to the scene picture based on four apex coordinates after adjustment
Target Photo simultaneously corrects it.
2. according to the method described in claim 1, it is characterized in that, further including：
Step S00：Using random background picture and training training input picture and training are generated with quadrangle Target Photo
With label figure；
Step S01：Different parameters are set to establish the multiple neural network model；And
Step S02：The multiple neural network mould is trained with label figure with input picture and the training using the training
Type.
3. method according to claim 1 or 2, which is characterized in that the neural network model is HED models.
4. method according to claim 1 or 2, which is characterized in that the step S2 includes：
Step S21：Retain all the points on the vertex in the multiple target labels figure while being predicted to be angle；
Step S22：The matrix core of certain size is taken to execute corrosion and expansive working to all the points；And
Step S23：The comprehensive label figure is obtained based on the result of the step S21 and S22.
5. method according to claim 1 or 2, which is characterized in that the step S3 includes：
Step S31：Obtain the coordinate on all vertex for being judged as angle in the comprehensive label figure；And
Step S32：To the coordinate using the clustering algorithm to obtain the apex coordinate at four angles.
6. according to the method described in claim 5, it is characterized in that, the clustering algorithm is kmeans algorithms.
7. method according to claim 1 or 2, which is characterized in that the step S4 includes：
Step S41：For each coordinate in four acquired apex coordinates, sum its transverse and longitudinal coordinate to obtain
With four coordinates of four vertex correspondences and；
Step S42：Coordinate corresponding to minimum value and maximum value of four coordinates in is identified as quadrangle
The coordinate of lower-left angular vertex and upper right angular vertex；
Step S43：The abscissa of remaining two coordinates is compared；
Step S44：According to the comparison result of step S43, the top left corner apex and the lower right corner of the two coordinates and quadrangle are determined
The correspondence of the coordinate on vertex；And
Step S45：The sequence of four apex coordinates is correspondingly adjusted according to the result of the step S42, the step S44.
8. method according to claim 1 or 2, which is characterized in that the step S5 includes：
Step S51：Based on after the adjustment four apex coordinates and predetermined correction representative points coordinate determine
Perspective transform operator；And
Step S52：Perspective transform is carried out to extract the quadrangle mesh to the scene picture using the perspective transform operator
It marks on a map and piece and it is corrected.
9. a kind of device for quadrangle Target Photo to be extracted and corrected from scene picture, which is characterized in that including：
Unit the 1st is used to predict scene picture using multiple neural network models for customizing training more to generate
A target labels figure；
Unit the 2nd is used to obtain four angles that can cover the quadrangle Target Photo based on the multiple target labels figure
Vertex synthesis label figure；
Unit the 3rd is used to obtain the apex coordinate at four angles from the comprehensive label figure using clustering algorithm；
Unit the 4th is used to determine the correspondence at four angles of four acquired apex coordinates and quadrangle, to adjust
The sequence of whole four apex coordinates；And
Unit the 5th, four apex coordinates after being used for based on adjustment carry out perspective transform to extract to the scene picture
It states quadrangle Target Photo and it is corrected.
10. device according to claim 9, which is characterized in that further include：
For being marked with quadrangle Target Photo to generate training input picture and training using random background picture and training
Sign the unit of figure；
The unit of the multiple neural network model is established for different parameters to be arranged；And
For training the multiple neural network model with label figure with input picture and the training using the training
Unit.
11. device according to claim 9 or 10, which is characterized in that the neural network model is HED models.
12. device according to claim 9 or 10, which is characterized in that Unit the 2nd includes：
2A units are used to retain all the points on the vertex in the multiple target labels figure while being predicted to be angle；
2B units are used to take the matrix core of certain size to execute corrosion and expansive working to all the points；And
2C units are used to obtain the comprehensive label figure based on the result in the 2A units and 2B units.
13. device according to claim 9 or 10, which is characterized in that Unit the 3rd includes：
3A units are used to obtain the coordinate on all vertex for being judged as angle in the comprehensive label figure；And
3B units are used for the coordinate using the clustering algorithm to obtain the apex coordinate at four angles.
14. device according to claim 13, which is characterized in that the clustering algorithm is kmeans algorithms.
15. device according to claim 9 or 10, which is characterized in that Unit the 4th includes：
4A units are used to sum for each coordinate pair its transverse and longitudinal coordinate in four acquired apex coordinates
With obtain with four coordinates of four vertex correspondences and；
4B units are used for the coordinate corresponding to minimum value and maximum value by four coordinates in and are identified as four
The coordinate of the lower-left angular vertex and upper right angular vertex of side shape；
The abscissa of remaining two coordinates is compared by 4C units；
4D units are used to determine the upper left of the two coordinates and quadrangle according to the comparison result in the 4C units
The correspondence of the coordinate of angular vertex and bottom right angular vertex；And
4E units are used to be sat correspondingly to adjust four vertex according to the result of the 4B units, the 4D units
Target sequence.
16. device according to claim 9 or 10, which is characterized in that Unit the 5th includes：
5A units：It is used for based on after the adjustment four apex coordinates and predetermined correction representative points coordinate
To determine perspective transform operator；And
5B units：Perspective transform is carried out to extract the quadrangle mesh to the scene picture using the perspective transform operator
It marks on a map and piece and it is corrected.
17. a kind of computer equipment, including memory, processor and storage are on a memory and the meter that can run on a processor
Calculation machine program, which is characterized in that the processor is realized when executing described program according to any one of claim 1 to 8 institute
The step of method stated.
18. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by computer
The step of executing to realize the method according to any one of claim 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711483213.4A CN108320290B (en) | 2017-12-29 | 2017-12-29 | Target picture extraction and correction method and device, computer equipment and recording medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711483213.4A CN108320290B (en) | 2017-12-29 | 2017-12-29 | Target picture extraction and correction method and device, computer equipment and recording medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108320290A true CN108320290A (en) | 2018-07-24 |
| CN108320290B CN108320290B (en) | 2021-10-22 |
Family
ID=62893505
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711483213.4A Active CN108320290B (en) | 2017-12-29 | 2017-12-29 | Target picture extraction and correction method and device, computer equipment and recording medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108320290B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325415A (en) * | 2018-08-22 | 2019-02-12 | 吴昌议 | A method of all target areas are predicted based on image column alignment feature |
| CN109464148A (en) * | 2018-11-12 | 2019-03-15 | 深圳码隆科技有限公司 | Measure the apparatus and system of spinal curvature |
| CN109544496A (en) * | 2018-11-19 | 2019-03-29 | 南京旷云科技有限公司 | Generation method, the training method and device of object detection model of training data |
| CN109635743A (en) * | 2018-12-13 | 2019-04-16 | 深源恒际科技有限公司 | A kind of text detection deep learning method and system of combination STN module |
| CN110047061A (en) * | 2019-04-26 | 2019-07-23 | 杭州智趣智能信息技术有限公司 | A kind of image interfusion method, device and the medium of the more backgrounds of multi-angle |
| CN110414511A (en) * | 2019-07-30 | 2019-11-05 | 深圳市普渡科技有限公司 | Cooperate sign and system |
| CN111080694A (en) * | 2019-12-20 | 2020-04-28 | 上海眼控科技股份有限公司 | Training and positioning method, device, equipment and storage medium of positioning model |
| CN111353422A (en) * | 2020-02-27 | 2020-06-30 | 维沃移动通信有限公司 | Information extraction method and device and electronic equipment |
| CN111445566A (en) * | 2020-03-27 | 2020-07-24 | 腾讯科技（深圳）有限公司 | Information processing method and device and computer readable storage medium |
| CN112433662A (en) * | 2020-11-26 | 2021-03-02 | 北京达佳互联信息技术有限公司 | Image correction method, image correction device, electronic device and storage medium |
| CN112464852A (en) * | 2020-12-09 | 2021-03-09 | 重庆大学 | Self-adaptive correction and identification method for vehicle driving license picture |
| CN113221897A (en) * | 2020-02-06 | 2021-08-06 | 马上消费金融股份有限公司 | Image correction method, image text recognition method, identity verification method and device |
| CN113743492A (en) * | 2021-08-30 | 2021-12-03 | 许继集团有限公司 | Method and device for sequencing row positions and column positions of pressing plates |
| WO2022028247A1 (en) * | 2020-08-06 | 2022-02-10 | 杭州睿琪软件有限公司 | Object edge identification and processing method and system, and computer-readable storage medium |
| US11250291B2 (en) | 2018-09-04 | 2022-02-15 | Advanced New Technologies, Co., Ltd. | Information detection method, apparatus, and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005011070A (en) * | 2003-06-19 | 2005-01-13 | Victor Co Of Japan Ltd | Image synthesis device |
| US20170124433A1 (en) * | 2015-11-04 | 2017-05-04 | Nec Laboratories America, Inc. | Unsupervised matching in fine-grained datasets for single-view object reconstruction |
| CN107169493A (en) * | 2017-05-31 | 2017-09-15 | 北京小米移动软件有限公司 | information identifying method and device |
| CN107506765A (en) * | 2017-10-13 | 2017-12-22 | 厦门大学 | A kind of method of the license plate sloped correction based on neutral net |
-
2017
- 2017-12-29 CN CN201711483213.4A patent/CN108320290B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005011070A (en) * | 2003-06-19 | 2005-01-13 | Victor Co Of Japan Ltd | Image synthesis device |
| US20170124433A1 (en) * | 2015-11-04 | 2017-05-04 | Nec Laboratories America, Inc. | Unsupervised matching in fine-grained datasets for single-view object reconstruction |
| CN107169493A (en) * | 2017-05-31 | 2017-09-15 | 北京小米移动软件有限公司 | information identifying method and device |
| CN107506765A (en) * | 2017-10-13 | 2017-12-22 | 厦门大学 | A kind of method of the license plate sloped correction based on neutral net |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325415A (en) * | 2018-08-22 | 2019-02-12 | 吴昌议 | A method of all target areas are predicted based on image column alignment feature |
| US11250291B2 (en) | 2018-09-04 | 2022-02-15 | Advanced New Technologies, Co., Ltd. | Information detection method, apparatus, and device |
| CN109464148B (en) * | 2018-11-12 | 2021-09-14 | 深圳码隆科技有限公司 | Device and system for measuring spinal curvature |
| CN109464148A (en) * | 2018-11-12 | 2019-03-15 | 深圳码隆科技有限公司 | Measure the apparatus and system of spinal curvature |
| CN109544496A (en) * | 2018-11-19 | 2019-03-29 | 南京旷云科技有限公司 | Generation method, the training method and device of object detection model of training data |
| CN109635743A (en) * | 2018-12-13 | 2019-04-16 | 深源恒际科技有限公司 | A kind of text detection deep learning method and system of combination STN module |
| CN110047061A (en) * | 2019-04-26 | 2019-07-23 | 杭州智趣智能信息技术有限公司 | A kind of image interfusion method, device and the medium of the more backgrounds of multi-angle |
| CN110414511A (en) * | 2019-07-30 | 2019-11-05 | 深圳市普渡科技有限公司 | Cooperate sign and system |
| CN110414511B (en) * | 2019-07-30 | 2022-05-03 | 深圳市普渡科技有限公司 | Cooperative sign recognition method and system for robot |
| CN111080694A (en) * | 2019-12-20 | 2020-04-28 | 上海眼控科技股份有限公司 | Training and positioning method, device, equipment and storage medium of positioning model |
| CN113221897A (en) * | 2020-02-06 | 2021-08-06 | 马上消费金融股份有限公司 | Image correction method, image text recognition method, identity verification method and device |
| CN111353422B (en) * | 2020-02-27 | 2023-08-22 | 维沃移动通信有限公司 | Information extraction method and device and electronic equipment |
| CN111353422A (en) * | 2020-02-27 | 2020-06-30 | 维沃移动通信有限公司 | Information extraction method and device and electronic equipment |
| CN111445566B (en) * | 2020-03-27 | 2022-05-06 | 腾讯科技（深圳）有限公司 | Information processing method and device and computer readable storage medium |
| CN111445566A (en) * | 2020-03-27 | 2020-07-24 | 腾讯科技（深圳）有限公司 | Information processing method and device and computer readable storage medium |
| WO2022028247A1 (en) * | 2020-08-06 | 2022-02-10 | 杭州睿琪软件有限公司 | Object edge identification and processing method and system, and computer-readable storage medium |
| CN112433662B (en) * | 2020-11-26 | 2022-08-12 | 北京达佳互联信息技术有限公司 | Image correction method, image correction device, electronic device and storage medium |
| CN112433662A (en) * | 2020-11-26 | 2021-03-02 | 北京达佳互联信息技术有限公司 | Image correction method, image correction device, electronic device and storage medium |
| CN112464852A (en) * | 2020-12-09 | 2021-03-09 | 重庆大学 | Self-adaptive correction and identification method for vehicle driving license picture |
| CN112464852B (en) * | 2020-12-09 | 2023-12-05 | 重庆大学 | Vehicle driving license picture self-adaptive correction and identification method |
| CN113743492A (en) * | 2021-08-30 | 2021-12-03 | 许继集团有限公司 | Method and device for sequencing row positions and column positions of pressing plates |
| CN113743492B (en) * | 2021-08-30 | 2024-04-19 | 许继集团有限公司 | Method and device for ordering positions of rows and columns of pressing plates |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108320290B (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108320290A (en) | Target Photo extracts antidote and device, computer equipment and recording medium | |
| JP6754619B2 (en) | Face recognition method and device | |
| US9754192B2 (en) | Object detection utilizing geometric information fused with image data | |
| CN105184249A (en) | Method and device for processing face image | |
| CN102087703B (en) | The method determining the facial pose in front | |
| US11610373B2 (en) | Method of generating three-dimensional model data of object | |
| JP5227629B2 (en) | Object detection method, object detection apparatus, and object detection program | |
| CN104978764A (en) | Three-dimensional face mesh model processing method and three-dimensional face mesh model processing equipment | |
| JP2019117577A (en) | Program, learning processing method, learning model, data structure, learning device and object recognition device | |
| CN105184735A (en) | Portrait deformation method and device | |
| JP2018022360A (en) | Image analysis device, image analysis method and program | |
| JP5772572B2 (en) | Image processing apparatus, image processing method, and program | |
| CN110288715B (en) | Virtual necklace try-on method and device, electronic equipment and storage medium | |
| CN104680512A (en) | Image processing device and method | |
| CN108573251A (en) | Character area localization method and device | |
| CN112381061B (en) | Facial expression recognition method and system | |
| CN108846385A (en) | Based on convolution-deconvolution neural network image recognition, bearing calibration and device | |
| CN107844742A (en) | Facial image glasses minimizing technology, device and storage medium | |
| JP2017092947A (en) | Learning support program, learning support device, learning tool formation notebook, and learning tool formation method | |
| CN103839066A (en) | Feature extraction method based on biological vision | |
| CN111488753A (en) | Two-dimensional code identification method and device, electronic equipment and readable storage medium | |
| CN110807459A (en) | License plate correction method and device and readable storage medium | |
| CN111353069A (en) | Character scene video generation method, system, device and storage medium | |
| EP3121788B1 (en) | Image feature estimation method and device | |
| JP2009230704A (en) | Object detection method, object detection device, and object detection program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |