CN107424131B - Image based on deep learning removes grid method and device - Google Patents
Image based on deep learning removes grid method and device Download PDFInfo
- Publication number
- CN107424131B CN107424131B CN201710575071.8A CN201710575071A CN107424131B CN 107424131 B CN107424131 B CN 107424131B CN 201710575071 A CN201710575071 A CN 201710575071A CN 107424131 B CN107424131 B CN 107424131B
- Authority
- CN
- China
- Prior art keywords
- grid
- image
- network
- data
- trained
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013135 deep learning Methods 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000012549 training Methods 0.000 claims abstract description 90
- 238000012545 processing Methods 0.000 claims abstract description 61
- 238000012216 screening Methods 0.000 claims description 24
- 238000013507 mapping Methods 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 17
- 230000015572 biosynthetic process Effects 0.000 claims description 11
- 238000010276 construction Methods 0.000 claims description 6
- 235000013399 edible fruits Nutrition 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 17
- 230000005284 excitation Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 239000000284 extract Substances 0.000 description 5
- 238000003475 lamination Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of images based on deep learning to remove grid method and device, grid image is constructed online by net template, multi-class grid data corresponding with net template is generated, and sorter network and full convolutional network is respectively trained using multi-class grid data as training data；Classification processing, and handling as a result, carrying out grid to the grid image to be removed classified using trained full convolutional network according to classification processing are carried out to grid image to be removed using trained sorter network.Image provided by the invention based on deep learning removes grid method and device, using National Federation of Trade Unions's convolutional network as training network, carries out grid to grid image to be removed by trained total convolutional network and handles, learns relevant parameter；And classification processing is carried out to grid image to be removed in advance by trained sorter network, then accordingly grid is gone to handle the grid image to be removed progress classified by corresponding trained total convolutional network, go grid effect good.
Description
Technical field
The present invention relates to field of image processings, particularly, be related to a kind of image based on deep learning go grid method and
Device.
Background technique
Biological identification technology has gradually been applied in the various scenes of authentication.Wherein most widely used is that face is known
Other technology.In many business scenarios, the certificate photograph of the true man's human face photo and he or she that need to arrive collection in worksite is carried out
Recognition of face processing.The certificate photograph obtained in business scenario is by adding grid to handle, so firstly the need of to band net
The certificate photo of lattice carries out grid operations.
Certificate photo goes the technology of grid to change domain algorithm and image processing method at present.Transform-domain algorithm uses DCT
(Discrete Cosine Transform, discrete cosine transform) or DWT (Discrete Wavelet Transform, from
Dissipate wavelet transformation) changing image spatial domain come do grid removal processing, this algorithm process effect is general, and to picture noise pole
It is sensitive.Using the algorithm of traditional image procossing, such as the operation such as smothing filtering and refinement, image lattice is needed to be aligned, and
The algorithm robustness is too poor.
Therefore, the grid poor removal effect in the presence of the method for existing removal grid, is a skill urgently to be resolved
Art problem.
Summary of the invention
The present invention provides a kind of images based on deep learning to remove grid method and device, to solve the removal grid having
Method in the presence of grid poor removal effect the technical issues of.
The technical solution adopted by the invention is as follows:
According to an aspect of the present invention, a kind of image based on deep learning is provided and removes grid method, comprising steps of
Grid of different sizes in multiple original mesh images is extracted, a variety of nets corresponding with the type of grid are constructed
Grid template；
Grid image is constructed online using a variety of net templates, generates multi-class grid number corresponding with net template
According to；
Sorter network and full convolutional network are respectively trained as training data for the multi-class grid data of generation；
Classification processing is carried out to grid image to be removed using trained sorter network；And according to the knot of classification processing
Fruit carries out grid to the grid image to be removed classified using trained full convolutional network and handles.
Further, grid of different sizes in multiple original mesh images is extracted, is constructed opposite with the type of grid
The step of a variety of net templates answered includes:
The different grids in multiple the original mesh images collected are extracted, the pixel size of the grid of extraction is obtained；
According to the pixel size of the grid of acquisition, classify to grid, by grid dividing at multiple types, and construct with
The corresponding net template of the type of grid.
Further, grid image is constructed online using a variety of net templates, generate multiclass corresponding with net template
The step of other grid data includes:
A variety of net templates are superimposed in non-screening table images, construct corresponding grid image online；
Non- screening table images and grid image are integrated to form training image mapping pair, and the training image of formation is mapped
To as multi-class grid data.
Further, sorter network includes two sorter networks, and multi-class grid data includes refined net data and coarse grid
Data, the step of sorter network and full convolutional network are respectively trained as training data for the multi-class grid data of generation packet
It includes:
Refined net data and coarse grid data are trained into two sorter networks as training data, trained two are classified
Network is as two sorter network models；
Refined net data and coarse grid data are trained into full convolutional network as training data, by trained full convolution
Network is as corresponding full convolutional network model.
Further, include: to the step of grid image to be removed progress classification processing using trained sorter network
Classification processing is carried out to grid image to be removed using sorter network, judges the network mould of grid image to be removed
Formula；
According to the network mode judged, grid image to be removed is transported in corresponding full convolutional network model, with
Grid image is removed after obtaining grid.
According to another aspect of the present invention, a kind of image based on deep learning is also provided and goes mesh device, comprising:
It constructs module and constructs the type with grid for extracting grid of different sizes in multiple original mesh images
Corresponding a variety of net templates；
Generation module generates corresponding with net template for constructing grid image online using a variety of net templates
Multi-class grid data；
Training template, for using generate multi-class grid data be respectively trained as training data sorter network and entirely
Convolutional network；
Grid processing module, for carrying out classification processing to grid image to be removed using trained sorter network；And
According to classification processing as a result, being carried out at grid using trained full convolutional network to the grid image to be removed classified
Reason.
Further, building module includes extraction unit and construction unit,
Extraction unit obtains the grid of extraction for extracting the different grids in multiple the original mesh images collected
Pixel size；
Construction unit classifies to grid, for the pixel size according to the grid of acquisition by grid dividing at a variety of
Type, and construct net template corresponding with the type of grid.
Further, generation module includes structural unit and integral unit,
Structural unit constructs corresponding grid chart for a variety of net templates to be superimposed in non-screening table images online
Picture；
Integral unit for integrating non-screening table images and grid image to form training image mapping pair, and will be formed
Training image map to as multi-class grid data.
Further, sorter network includes two sorter networks, and multi-class grid data includes refined net data and coarse grid
Data, training template include the first training unit and the second training unit,
First training unit, for refined net data and coarse grid data to be trained two classification nets as training data
Network, using trained two sorter network as two sorter network models；
Second training unit, for refined net data and coarse grid data to be trained full convolution net as training data
Network, using trained full convolutional network as corresponding full convolutional network model.
Further, grid processing module includes judging unit and supply unit,
Judging unit judges grid to be removed for carrying out classification processing to grid image to be removed using sorter network
The network mode of image；
Supply unit, for according to the network mode judged, grid image to be removed to be transported to corresponding full convolution
In network model, to obtain removing grid image after grid.
The invention has the following advantages:
Image provided by the invention based on deep learning removes grid method and device, constructs net online by net template
Table images generate multi-class grid data corresponding with net template, and using multi-class grid data as training data point
It Xun Lian not sorter network and full convolutional network；Classification processing is carried out to grid image to be removed using trained sorter network,
And according to classification processing as a result, carrying out grid to the grid image to be removed classified using trained full convolutional network
Processing.Image provided by the invention based on deep learning removes grid method and device, using National Federation of Trade Unions's convolutional network as training net
Network carries out grid to grid image to be removed by trained total convolutional network and handles, with the grid to be removed with grid
Image is exported without the removal grid image of grid as target as being originally inputted, and learns relevant parameter；And pass through instruction
The sorter network perfected carries out classification processing to grid image to be removed in advance, then by corresponding trained total convolutional network pair
The grid image to be removed progress classified accordingly goes grid to handle, and goes the effect of grid good.
Other than objects, features and advantages described above, there are also other objects, features and advantages by the present invention.
Below with reference to figure, the present invention is described in further detail.
Detailed description of the invention
The attached drawing constituted part of this application is used to provide further understanding of the present invention, schematic reality of the invention
It applies example and its explanation is used to explain the present invention, do not constitute improper limitations of the present invention.In the accompanying drawings:
Fig. 1 is that the present invention is based on the flow diagrams that the image of deep learning goes grid method preferred embodiment；
Fig. 2 is that grid of different sizes in multiple original mesh images is extracted in Fig. 1, is constructed opposite with the type of grid
The refinement flow diagram of the step of a variety of net templates answered；
Fig. 3 is the image schematic diagram for the original image collected in the present invention；
Fig. 4 is the net template schematic diagram according to a variety of net templates of the original image building in Fig. 3；
Fig. 5 is to construct grid image online using a variety of net templates in Fig. 1, generates multiclass corresponding with net template
The refinement flow diagram of the step of other grid data；
Fig. 6 is the image schematic diagram of the non-screening table images of the present invention；
Fig. 7 is that the corresponding grid image constructed after net template is superimposed on the basis of the non-screening table images of Fig. 6
Image schematic diagram；
Fig. 8 is that sorter network and full volume is respectively trained in Fig. 1 using the multi-class grid data of generation as training data
The refinement flow diagram of the step of product network；
Fig. 9 is the schematic network structure of sorter network in the present invention；
Figure 10 is the schematic network structure of full convolutional network in the present invention；
Figure 11 is thin the step of carrying out classification processing to grid image to be removed using trained sorter network in Fig. 1
Change flow diagram；
Figure 12 is that the present invention is based on the functional block diagrams that the image of deep learning goes mesh device preferred embodiment；
Figure 13 is the functional block diagram that module is constructed in Figure 12；
Figure 14 is the functional block diagram of generation module in Figure 12；
Figure 15 is the functional block diagram of training module in Figure 12；
Figure 16 is the functional block diagram of grid processing module in Figure 12.
Drawing reference numeral explanation:
10, module is constructed；20, generation module；30, training template；40, grid processing module；11, extraction unit；12, structure
Build unit；21, structural unit；22, integral unit；31, the first training unit；32, the second training unit；41, judging unit；
42, supply unit.
Specific embodiment
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The present invention will be described in detail below with reference to the accompanying drawings and embodiments.
Referring to Fig.1, the preferred embodiment of the present invention provides a kind of image based on deep learning and removes grid method, including
Step:
Step S100, grid of different sizes in multiple original mesh images is extracted, is constructed opposite with the type of grid
The a variety of net templates answered.
As shown in Figure 3 and Figure 4, the different original mesh image of multiple grids is collected, size in original mesh image is extracted
Different grids constructs a variety of net templates corresponding with the type of grid.
Step S200, grid image is constructed online using a variety of net templates, generate multiclass corresponding with net template
Other grid data.
Grid image is constructed online using a variety of net templates, based on not band grid image, generates corresponding grid
Image band grid image and corresponding grid image will not combine to form training image mapping pair, and the training of formation will be schemed
As mapping is to as multi-class grid data corresponding with net template.And the corresponding grid image generated is subjected to data and is added
By force, a variety of noises are added, to guarantee the robustness of data, and the grid image generated are made to be more nearly true grid image.
Step S300, sorter network and full convolution are respectively trained as training data for the multi-class grid data of generation
Network.
According to the thickness of grid, the multi-class grid data of generation is divided into two classes, one kind is coarse grid data, another kind of
For refined net data.The coarse grid data of generation and refined net data are trained into a sorter network as training data,
Corresponding FCN (Fully Convolutional Networks, full convolutional network) is trained to every a kind of grid simultaneously.
Step S400, classification processing is carried out to grid image to be removed using trained sorter network；And according to classification
Processing is handled as a result, carrying out grid to the grid image to be removed classified using trained full convolutional network.
In test phase, classification processing is carried out to grid image to be removed using trained sorter network；And according to point
Handling as a result, carrying out grid to the grid image to be removed classified using trained FCN for class processing, obtains grid
Afterwards remove grid image.
Image provided in this embodiment based on deep learning removes grid method, constructs grid chart online by net template
Picture generates multi-class grid data corresponding with net template, and instructs respectively using multi-class grid data as training data
Practice sorter network and full convolutional network；Classification processing, and root are carried out to grid image to be removed using trained sorter network
According to classification processing as a result, being carried out at grid using trained full convolutional network to the grid image to be removed classified
Reason.Image provided in this embodiment based on deep learning removes grid method, using National Federation of Trade Unions's convolutional network as training network, passes through
Trained total convolutional network carries out grid to grid image to be removed and handles, using the grid image to be removed with grid as
It is originally inputted, is exported without the removal grid image of grid as target, learn relevant parameter；And pass through trained point
Class network carries out classification processing to grid image to be removed in advance, then by corresponding trained total convolutional network to having classified
Grid image progress to be removed accordingly goes grid to handle, and goes the effect of grid good.
Preferably, as shown in Fig. 2, the image provided in this embodiment based on deep learning removes grid method, step S100
Include:
Step S110, the different grids in multiple the original mesh images collected are extracted, the picture of the grid of extraction is obtained
Plain size.
See Fig. 3 and Fig. 4, the different original mesh image of multiple grids is collected, extracts multiple original meshes collected
Different grids in image, obtain the pixel size of the grid of extraction.
Step S120, according to the pixel size of the grid of acquisition, classify to grid, by grid dividing at multiple types
Type, and construct net template corresponding with the type of grid.
The pixel size for obtaining grid, classifies to grid according to the pixel size of the grid of acquisition, by grid dividing
At multiple types, and construct net template corresponding with the type of grid.Specifically, in the present embodiment, by grid dividing
At two class of refined net and coarse grid, classification foundation is that less than 5 pixels of net lattice control width are refined net in net template, greatly
In 5 pixels be coarse grid (referring to fig. 4, the first row secondary series and the second row first are classified as refined net, remaining is coarse grid).
Image provided in this embodiment based on deep learning removes grid method, extracts multiple the original mesh figures collected
Different grids as in, obtain the pixel size of the grid of extraction；According to the pixel size of the grid of acquisition, grid is divided
Class by grid dividing at multiple types, and constructs net template corresponding with the type of grid.It is provided in this embodiment to be based on
The image of deep learning removes grid method, according to the pixel size of the grid of acquisition, classifies to grid, by grid dividing at
Multiple types, and net template corresponding with the type of grid is constructed, guarantee the diversity of sample, thus the classification made
Network and full convolutional network are preferably trained and are learnt, and the better effect of grid is made, and card is gone after network processes
Grid image is as far as possible close to original image.
Preferably, as shown in figure 5, the image provided in this embodiment based on deep learning removes grid method, step S200
Include:
Step S210, a variety of net templates are superimposed in non-screening table images, construct corresponding grid image online.
As shown in Figure 6 and Figure 7, according to a variety of net templates constructed, based on not band grid image, by a variety of nets
Grid template is superimposed in non-screening table images, constructs corresponding grid image online.
Step S220, non-screening table images and grid image are integrated to form training image mapping pair, and by the instruction of formation
Practice image mapping to as multi-class grid data.
Non- screening table images and corresponding grid image are integrated, form training image mapping pair, and by formation
Training image mapping to as multi-class grid data to be trained respectively to sorter network and full convolutional network.
Image provided in this embodiment based on deep learning removes grid method, by being superimposed on a variety of net templates not
In screening table images, corresponding grid image is constructed online；Non- screening table images and grid image are integrated to form training image
Mapping pair, and the training image of formation is mapped to as multi-class grid data.It is provided in this embodiment to be based on deep learning
Image remove grid method, using training image mapping to sorter network and full convolutional network is trained respectively, make to train
Sorter network and full convolutional network grid is carried out to grid image to be removed and is handled, go the effect of grid good；It is online to generate
Multi-class grid data in a variety of net templates are added, guarantee the diversity of sample.
Preferably, as shown in figure 8, the image provided in this embodiment based on deep learning removes grid method, sorter network
Including two sorter networks, multi-class grid data includes refined net data and coarse grid data, and step S300 includes:
Step S310, refined net data and coarse grid data are trained into two sorter networks as training data, will trained
Two good sorter networks are as two sorter network models.
The refined net data constructed and coarse grid data are trained into two sorter networks as training data, will be instructed
Two sorter networks perfected as two sorter network models to be detected to images to be recognized, classify network such as Fig. 9 institute
Show, sorter network includes the first convolutional layer, the second convolutional layer, third convolutional layer, the first full articulamentum and the second full articulamentum.When
So, the first excitation layer and the first pond layer can also be connect behind the first convolutional layer, can connect the second excitation behind the second convolutional layer
Layer and the second pond layer can connect third excitation layer and third pond layer behind third convolutional layer.The input of first convolutional layer is 112
× 112 × 3 sized images, the convolution kernel for being 5 × 5 followed by 20 scales；The core of first pond layer is 3 × 3, step-length 2.
Second convolutional layer has 40 3 × 3 convolution kernels；Third convolutional layer has 60 3 × 3 convolution kernels.First excitation layer, the second excitation
Layer is identical with the structure of third excitation layer, and the structure of the first pond layer, the second pond layer and the second pond layer is identical.First connects entirely
The characteristic length for connecing layer is 128.Second full articulamentum is divided to refined net and two class of coarse grid to be exported.In the present embodiment, net
It is 112*112*3 that network, which inputs size, after three convolutional layers, carries out the first full articulamentum processing, and formation length is the of 128
One feature vector chart (featureMap) reconnects the second class label that connection corresponds to that length is 2 entirely and exports.
In the present embodiment, classification task is one two classification, and using cross entropy loss function, formula is as follows:
Wherein, piIndicate that the sample is the probability of refined net,Value be 0 or 1, respectively correspond the reality of the sample
Label is refined net or coarse grid.
Step S320, refined net data and coarse grid data are trained into full convolutional network as training data, will trained
Good full convolutional network is as corresponding full convolutional network model.
The refined net data constructed and coarse grid data are trained into FCN network as training data, it will be trained
FCN network is as corresponding full convolutional network model to detect to images to be recognized.FCN network structure is as indicated by 10: FCN
Network includes Volume Four lamination, the 5th convolutional layer and the 6th convolutional layer, and the input of FCN network is 56 × 56 × 3 sized images, the
The size of one convolution kernel is 9 × 9, and the number of core is 64, and the size for mending side (pad) is 6, and treated second for Volume Four lamination
Feature vector chart (featureMap) size is 60 × 60 × 64.Second convolution kernel size is 5 × 5, and the number of core is 32,
Pad is 1, is 32 × 60 × 60 by the third feature vectogram featureMap size that the 5th convolutional layer is handled well.Third
Convolution kernel size be 5 × 5, number 3, pad is 0, by the 6th convolutional layer processing after fourth feature vectogram size be 56 ×
56 × 3, and input it is identical, this method is also called the learning method of end-to-end (end to end).In addition, in Volume Four lamination
Behind connect the first excitation layer (Relu), connect behind the 5th convolutional layer the second excitation layer (Relu) and carry out non-linear place
Reason.Finally using Euclidean distance as loss function, reaction is input to the error of output.In the present embodiment, in full convolution net
In network, without full articulamentum, only convolutional layer and excitation layer.Setting three-layer coil product herein, passes through setting pad and convolution kernel size
Etc. parameters, guarantee three-layer coil product processing after output and input it is equal in magnitude.Using Euclidean distance as loss function, guarantee warp
Image after crossing network processes is as far as possible close to original image (non-screening table images).When training, using lesser learning rate, very much
One of to ten a ten thousandths, guarantee preferable convergence effect.
Image provided in this embodiment based on deep learning removes grid method, by by refined net data and coarse grid number
According to two sorter networks are trained as training data, using trained two sorter network as two sorter network models；By fine-structure mesh
Lattice data and coarse grid data train full convolutional network as training data, using trained full convolutional network as corresponding
Full convolutional network model.Image provided in this embodiment based on deep learning removes grid method, by the FCN net of end to end
Network structure is applied to grid and handles, and goes the effect of grid good.
Preferably, as shown in figure 11, the image provided in this embodiment based on deep learning removes grid method, step S400
Include:
Step S410, classification processing is carried out to grid image to be removed using sorter network, judges grid image to be removed
Network mode.
The grid image to be removed for inputting sorter network is carried out classification processing, judges grid chart to be removed by test phase
It seem to belong to coarse grid mode or refined net mode.
Step S420, according to the network mode judged, grid image to be removed is transported to corresponding full convolutional network
In model, to obtain removing grid image after grid.
The network mode finally judged according to sorter network conveys grid image to be removed for different models
Into corresponding full convolutional network model, to obtain removing grid image after grid.
Image provided in this embodiment based on deep learning removes grid method, by using sorter network to net to be removed
Table images carry out classification processing, judge the network mode of grid image to be removed；It, will be to be removed according to the network mode judged
Grid image is transported in corresponding full convolutional network model, to obtain removing grid image after grid.The present embodiment provides
The image based on deep learning remove grid method, grid image to be removed is divided in advance by trained sorter network
Class processing, then accordingly grid is gone to handle the grid image to be removed progress classified by corresponding full convolutional network model, it goes
The effect of grid is good, and guarantees to go grid image as far as possible close to original image after network processes.
Preferably, as shown in figure 12, the present embodiment also provides a kind of image based on deep learning and goes mesh device, wraps
Include: building module 10 constructs opposite with the type of grid for extracting grid of different sizes in multiple original mesh images
The a variety of net templates answered；Generation module 20 generates and grid mould for constructing grid image online using a variety of net templates
The corresponding multi-class grid data of plate；Training template 30, the multi-class grid data for that will generate is as training data
Sorter network and full convolutional network is respectively trained；Grid processing module 40, for using trained sorter network to be removed
Grid image carries out classification processing；And according to classification processing as a result, using trained full convolutional network to classified to
Removal grid image carries out grid and handles.
As shown in Figure 3 and Figure 4, building module 10 collects the different original mesh image of multiple grids, extracts original mesh
Grid of different sizes in image constructs a variety of net templates corresponding with the type of grid.
Generation module 20 constructs grid image using a variety of net templates online, based on not band grid image, generates
Corresponding grid image band grid image and corresponding grid image will not combine to form training image mapping pair, and by shape
At training image map to as multi-class grid data corresponding with net template.And the grid image generated will be corresponded to
Data reinforcement is carried out, a variety of noises are added, to guarantee the robustness of data, and is more nearly the grid image generated really
Grid image.
According to the thickness of grid, the multi-class grid data of generation is divided into two classes, one kind is coarse grid data, another kind of
For refined net data.The coarse grid data of generation and refined net data are trained one as training data by training template 30
Sorter network, while corresponding FCN (Fully Convolutional Networks, full convolution net are trained to every a kind of grid
Network).
In test phase, grid processing module 40 classifies to grid image to be removed using trained sorter network
Processing；And according to classification processing as a result, being carried out at grid using trained FCN to the grid image to be removed classified
Reason, removes grid image after obtaining grid.
Image provided in this embodiment based on deep learning goes mesh device, constructs grid chart online by net template
Picture generates multi-class grid data corresponding with net template, and instructs respectively using multi-class grid data as training data
Practice sorter network and full convolutional network；Classification processing, and root are carried out to grid image to be removed using trained sorter network
According to classification processing as a result, being carried out at grid using trained full convolutional network to the grid image to be removed classified
Reason.Image provided in this embodiment based on deep learning goes mesh device, using National Federation of Trade Unions's convolutional network as training network, passes through
Trained total convolutional network carries out grid to grid image to be removed and handles, using the grid image to be removed with grid as
It is originally inputted, is exported without the removal grid image of grid as target, learn relevant parameter；And pass through trained point
Class network carries out classification processing to grid image to be removed in advance, then by corresponding trained total convolutional network to having classified
Grid image progress to be removed accordingly goes grid to handle, and goes the effect of grid good.
Preferably, as shown in figure 13, the present embodiment also provides a kind of image based on deep learning and goes mesh device, constructs
Module 10 includes extraction unit 11 and construction unit 12, extraction unit 11, for extracting multiple the original mesh images collected
In different grids, obtain the pixel size of the grid of extraction；Construction unit 12, the pixel for the grid according to acquisition are big
It is small, classify to grid, by grid dividing at multiple types, and constructs net template corresponding with the type of grid.
See Fig. 3 and Fig. 4, the different original mesh image of multiple grids is collected, extracts multiple original meshes collected
Different grids in image, obtain the pixel size of the grid of extraction.
The pixel size for obtaining grid, classifies to grid according to the pixel size of the grid of acquisition, by grid dividing
At multiple types, and construct net template corresponding with the type of grid.Specifically, in the present embodiment, by grid dividing
At two class of refined net and coarse grid, classification foundation is that less than 5 pixels of net lattice control width are refined net in net template, greatly
In 5 pixels be coarse grid (referring to fig. 4, the first row secondary series and the second row first are classified as refined net, remaining is coarse grid).
Image provided in this embodiment based on deep learning goes mesh device, extracts multiple the original mesh figures collected
Different grids as in, obtain the pixel size of the grid of extraction；According to the pixel size of the grid of acquisition, grid is divided
Class by grid dividing at multiple types, and constructs net template corresponding with the type of grid.It is provided in this embodiment to be based on
The image of deep learning goes mesh device, according to the pixel size of the grid of acquisition, classifies to grid, by grid dividing at
Multiple types, and net template corresponding with the type of grid is constructed, guarantee the diversity of sample, thus the classification made
Network and full convolutional network are preferably trained and are learnt, and the better effect of grid is made, and card is gone after network processes
Grid image is as far as possible close to original image.
Preferably, as shown in figure 14, the present embodiment also provides a kind of image based on deep learning and goes mesh device, generates
Module 20 includes structural unit 21 and integral unit 22, structural unit 21, for a variety of net templates to be superimposed on non-screening lattice
In image, corresponding grid image is constructed online；Integral unit 22, for integrating non-screening table images and grid image to be formed
Training image mapping pair, and the training image of formation is mapped to as multi-class grid data.
As shown in Figure 6 and Figure 7, structural unit 21 is according to a variety of net templates constructed, using not band grid image as base
A variety of net templates are superimposed in non-screening table images, construct corresponding grid image online by plinth.
Integral unit 22 integrates non-screening table images and corresponding grid image, forms training image mapping pair,
And by the training image of formation mapping to as multi-class grid data to be instructed respectively to sorter network and full convolutional network
Practice.
Image provided in this embodiment based on deep learning goes mesh device, by being superimposed on a variety of net templates not
In screening table images, corresponding grid image is constructed online；Non- screening table images and grid image are integrated to form training image
Mapping pair, and the training image of formation is mapped to as multi-class grid data.It is provided in this embodiment to be based on deep learning
Image go mesh device, using training image mapping to sorter network and full convolutional network is trained respectively, make to train
Sorter network and full convolutional network grid is carried out to grid image to be removed and is handled, go the effect of grid good；It is online to generate
Multi-class grid data in a variety of net templates are added, guarantee the diversity of sample.
Preferably, as shown in figure 15, the present embodiment also provides a kind of image based on deep learning and goes mesh device, training
Template 30 includes the first training unit 31 and the second training unit 32, the first training unit 31, for by refined net data and slightly
Grid data trains two sorter networks as training data, using trained two sorter network as two sorter network models；
Second training unit 32 will be instructed for refined net data and coarse grid data to be trained full convolutional network as training data
The full convolutional network perfected is as corresponding full convolutional network model.
The refined net data constructed and coarse grid data are trained one as training data by the first training unit 31
Two sorter networks, using trained two sorter network as two sorter network models to be detected to images to be recognized, classification
Network as shown in figure 9, sorter network include the first convolutional layer, the second convolutional layer, third convolutional layer, the first full articulamentum and
Second full articulamentum.Certainly, the first excitation layer and the first pond layer can also be connect behind the first convolutional layer, behind the second convolutional layer
The second excitation layer and the second pond layer can be connect, third excitation layer and third pond layer can be connect behind third convolutional layer.First
Convolutional layer input is 112 × 112 × 3 sized images, the convolution kernel for being 5 × 5 followed by 20 scales；The core of first pond layer
It is 3 × 3, step-length 2.Second convolutional layer has 40 3 × 3 convolution kernels；Third convolutional layer has 60 3 × 3 convolution kernels.First
Excitation layer, the second excitation layer are identical with the structure of third excitation layer, the knot of the first pond layer, the second pond layer and the second pond layer
Structure is identical.The characteristic length of first full articulamentum is 128.Second full articulamentum is divided to refined net and two class of coarse grid to be exported.
In the present embodiment, network inputs size is 112*112*3, after three convolutional layers, carries out the first full articulamentum processing, shape
The first eigenvector figure (featureMap) for being 128 at length reconnects the second full connection and corresponds to the class label that length is 2
Output.
In the present embodiment, classification task is one two classification, and using cross entropy loss function, formula is as follows:
Wherein, piIndicate that the sample is the probability of refined net,Value be 0 or 1, respectively correspond the reality of the sample
Label is refined net or coarse grid.
The refined net data constructed and coarse grid data are trained FCN net as training data by the second training unit 32
Network, using trained FCN network as corresponding full convolutional network model to be detected to images to be recognized.FCN network knot
Structure is as indicated by 10: FCN network includes Volume Four lamination, the 5th convolutional layer and the 6th convolutional layer, and the input of FCN network is 56 × 56
× 3 sized images, the size of first convolution kernel are 9 × 9, and the number of core is 64, and the size for mending side (pad) is 6, Volume Four product
Layer treated second feature vectogram (featureMap) size is 60 × 60 × 64.Second convolution kernel size is 5 × 5,
The number of core is 32, pad 1, the third feature vectogram featureMap size handled well by the 5th convolutional layer is 32 ×
60×60.Third convolution kernel size is 5 × 5, and number 3, pad is 0, the fourth feature vector after the processing of the 6th convolutional layer
Figure size is 56 × 56 × 3, and input is identical, and this method is also called the learning method of end-to-end (end to end).In addition,
Connected behind Volume Four lamination the first excitation layer (Relu), connect behind the 5th convolutional layer the second excitation layer (Relu) into
Row Nonlinear Processing.Finally using Euclidean distance as loss function, reaction is input to the error of output.In the present embodiment,
In full convolutional network, without full articulamentum, only convolutional layer and excitation layer.Herein setting three-layer coil product, by setting pad and
It is equal in magnitude to guarantee that three-layer coil product processing is output and input later for the parameters such as convolution kernel size.Using Euclidean distance as loss
Function guarantees the image after network processes as far as possible close to original image (non-screening table images).When training, use is lesser
Learning rate, a ten thousandth to ten a ten thousandths guarantee preferable convergence effect.
Image provided in this embodiment based on deep learning goes mesh device, by by refined net data and coarse grid number
According to two sorter networks are trained as training data, using trained two sorter network as two sorter network models；By fine-structure mesh
Lattice data and coarse grid data train full convolutional network as training data, using trained full convolutional network as corresponding
Full convolutional network model.Image provided in this embodiment based on deep learning goes mesh device, by the FCN net of end to end
Network structure is applied to grid and handles, and goes the effect of grid good.
Preferably, as shown in figure 16, the present embodiment also provides a kind of image based on deep learning and removes mesh device, grid
Processing module 40 includes judging unit 41 and supply unit 42, judging unit 41, for using sorter network to grid to be removed
Image carries out classification processing, judges the network mode of grid image to be removed；Supply unit 42, for according to the network judged
Grid image to be removed is transported in corresponding full convolutional network model by mode, to obtain removing grid image after grid.
Test phase, judging unit 41 will input sorter network grid image to be removed carry out classification processing, judge to
Removal grid image belongs to coarse grid mode or refined net mode.
The network mode that supply unit 42 is finally judged according to sorter network, for different models, by net to be removed
Table images are transported in corresponding full convolutional network model, to obtain removing grid image after grid.
Image provided in this embodiment based on deep learning goes mesh device, by using sorter network to net to be removed
Table images carry out classification processing, judge the network mode of grid image to be removed；It, will be to be removed according to the network mode judged
Grid image is transported in corresponding full convolutional network model, to obtain removing grid image after grid.The present embodiment provides
The image based on deep learning go mesh device, grid image to be removed is divided in advance by trained sorter network
Class processing, then accordingly grid is gone to handle the grid image to be removed progress classified by corresponding full convolutional network model, it goes
The effect of grid is good, and guarantees to go grid image as far as possible close to original image after network processes.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (2)
1. a kind of image based on deep learning removes grid method, which is characterized in that comprising steps of
Grid of different sizes in multiple original mesh images is extracted, a variety of nets corresponding with the type of the grid are constructed
Grid template；
Grid image is constructed online using a variety of net templates, generates multi-class grid corresponding with the net template
Data；
Sorter network and full convolutional network are respectively trained as training data for the multi-class grid data of generation；
Classification processing is carried out to grid image to be removed using the trained sorter network；And according to the knot of the classification
Fruit carries out grid to the grid image to be removed classified using the trained full convolutional network and handles；
It is described to extract grid of different sizes in multiple original mesh images, it constructs corresponding with the type of the grid more
The step of kind of net template includes:
The different grids in multiple the described original mesh images collected are extracted, the pixel for obtaining the grid of extraction is big
It is small；
According to the pixel size of the grid of acquisition, classify to the grid, by the grid dividing at multiple types,
And construct net template corresponding with the type of the grid；
It is described to construct grid image online using a variety of net templates, it generates corresponding with the net template multi-class
The step of grid data includes:
A variety of net templates are superimposed in non-screening table images, construct corresponding grid image online；
The non-screening table images and the grid image are integrated to form training image mapping pair, and by the training of formation
Image is mapped to as multi-class grid data；
The sorter network includes two sorter networks, and the multi-class grid data includes refined net data and coarse grid data,
The step that the multi-class grid data of generation is respectively trained to sorter network and full convolutional network as training data
Suddenly include:
The refined net data and the coarse grid data are trained into two sorter network as training data, will be trained
Two sorter network as two sorter network models；
The refined net data and the coarse grid data are trained into the full convolutional network as training data, will be trained
The full convolutional network as corresponding full convolutional network model；
It is described that classification processing is carried out to grid image to be removed using the trained sorter network；And according to the classification
As a result, the step of grid processing is carried out to the grid image to be removed classified using the trained full convolutional network
Suddenly include:
Classification processing is carried out to the grid image to be removed using the sorter network, judges the grid image to be removed
Network mode；
According to the network mode judged, the grid image to be removed is transported to corresponding full convolutional network model
In, to obtain removing grid image after grid.
2. a kind of image based on deep learning goes mesh device characterized by comprising
Building module (10) is constructed and the grid for extracting grid of different sizes in multiple original mesh images
The corresponding a variety of net templates of type；
Generation module (20) generates and the net template for constructing grid image online using a variety of net templates
Corresponding multi-class grid data；
Training template (30), for sorter network to be respectively trained as training data in the multi-class grid data generated
With full convolutional network；
Grid processing module (40), for being carried out at classification using the trained sorter network to grid image to be removed
Reason, carries out grid to the grid image to be removed classified using the trained full convolutional network and handles；
The building module (10) includes extraction unit (11) and construction unit (12),
Extraction unit (11) obtains extraction for extracting the different grids in multiple the described original mesh images collected
The pixel size of the grid；
Construction unit (12) classifies to the grid, for the pixel size according to the grid of acquisition by the net
Lattice are divided into multiple types, and construct net template corresponding with the type of the grid；
The generation module (20) includes structural unit (21) and integral unit (22),
Structural unit (21) constructs corresponding net for a variety of net templates to be superimposed in non-screening table images online
Table images；
Integral unit (22), for integrating the non-screening table images and the grid image to form training image mapping pair,
And the training image of formation is mapped to as multi-class grid data；
The sorter network includes two sorter networks, and the multi-class grid data includes refined net data and coarse grid data,
The trained template (30) includes the first training unit (31) and the second training unit (32),
First training unit (31), for using the refined net data and the coarse grid data as training data to train
Two sorter networks are stated, using trained two sorter network as two sorter network models；
Second training unit (32), for using the refined net data and the coarse grid data as training data to train
Full convolutional network is stated, using the trained full convolutional network as corresponding full convolutional network model；
The grid processing module (40) includes judging unit (41) and supply unit (42),
Judging unit (41) judges institute for carrying out classification processing to the grid image to be removed using the sorter network
State the network mode of grid image to be removed；
Supply unit (42), for according to the network mode judged, the grid image to be removed to be transported to correspondence
Full convolutional network model in, to obtain removing grid image after grid.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710575071.8A CN107424131B (en) | 2017-07-14 | 2017-07-14 | Image based on deep learning removes grid method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710575071.8A CN107424131B (en) | 2017-07-14 | 2017-07-14 | Image based on deep learning removes grid method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107424131A CN107424131A (en) | 2017-12-01 |
| CN107424131B true CN107424131B (en) | 2019-11-12 |
Family
ID=60426594
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710575071.8A Active CN107424131B (en) | 2017-07-14 | 2017-07-14 | Image based on deep learning removes grid method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107424131B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108230269B (en) * | 2017-12-28 | 2021-02-09 | 智慧眼科技股份有限公司 | Grid removing method, device and equipment based on depth residual error network and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150658A (en) * | 2007-08-22 | 2008-03-26 | 北京中星微电子有限公司 | A bad point self-adapted grid noise elimination device and method |
| CN101272503A (en) * | 2008-04-30 | 2008-09-24 | 北京中星微电子有限公司 | Gridding noise elimination method and device for remaining image definition |
| CN101352047A (en) * | 2005-12-29 | 2009-01-21 | 安泰科技有限公司 | Method and device for removing grid noise |
| CN105125206A (en) * | 2015-09-15 | 2015-12-09 | 中山大学 | Intelligent electrocardio monitoring method and device |
| CN106709866A (en) * | 2016-11-18 | 2017-05-24 | 北京智慧眼科技股份有限公司 | Method and device for removing mesh watermark from identification photo, face verification method and device |
| CN106875360A (en) * | 2017-02-17 | 2017-06-20 | 广州智能装备研究院有限公司 | It is a kind of to eliminate the fuzzy method and device of image motion |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7840046B2 (en) * | 2006-06-27 | 2010-11-23 | Siemens Medical Solutions Usa, Inc. | System and method for detection of breast masses and calcifications using the tomosynthesis projection and reconstructed images |
-
2017
- 2017-07-14 CN CN201710575071.8A patent/CN107424131B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101352047A (en) * | 2005-12-29 | 2009-01-21 | 安泰科技有限公司 | Method and device for removing grid noise |
| CN101150658A (en) * | 2007-08-22 | 2008-03-26 | 北京中星微电子有限公司 | A bad point self-adapted grid noise elimination device and method |
| CN101272503A (en) * | 2008-04-30 | 2008-09-24 | 北京中星微电子有限公司 | Gridding noise elimination method and device for remaining image definition |
| CN105125206A (en) * | 2015-09-15 | 2015-12-09 | 中山大学 | Intelligent electrocardio monitoring method and device |
| CN106709866A (en) * | 2016-11-18 | 2017-05-24 | 北京智慧眼科技股份有限公司 | Method and device for removing mesh watermark from identification photo, face verification method and device |
| CN106875360A (en) * | 2017-02-17 | 2017-06-20 | 广州智能装备研究院有限公司 | It is a kind of to eliminate the fuzzy method and device of image motion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107424131A (en) | 2017-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108537743B (en) | Face image enhancement method based on generation countermeasure network | |
| CN104834933B (en) | A kind of detection method and device in saliency region | |
| CN108399362A (en) | A kind of rapid pedestrian detection method and device | |
| CN107506740A (en) | A kind of Human bodys' response method based on Three dimensional convolution neutral net and transfer learning model | |
| CN108682007B (en) | JPEG image resampling automatic detection method based on depth random forest | |
| CN106372648A (en) | Multi-feature-fusion-convolutional-neural-network-based plankton image classification method | |
| CN106934408A (en) | Identity card picture sorting technique based on convolutional neural networks | |
| Wu et al. | Steganalysis via deep residual network | |
| CN106682628B (en) | Face attribute classification method based on multilayer depth feature information | |
| CN109117897A (en) | Image processing method, device and readable storage medium storing program for executing based on convolutional neural networks | |
| CN108573243A (en) | A kind of comparison method of the low quality face based on depth convolutional neural networks | |
| CN104268593A (en) | Multiple-sparse-representation face recognition method for solving small sample size problem | |
| CN109446985B (en) | Multi-angle plant identification method based on vector neural network | |
| CN106203628B (en) | A kind of optimization method and system enhancing deep learning algorithm robustness | |
| CN107729901A (en) | Method for building up, device and the image processing method and system of image processing model | |
| Chen et al. | Densely connected convolutional neural network for multi-purpose image forensics under anti-forensic attacks | |
| Chen et al. | Automated design of neural network architectures with reinforcement learning for detection of global manipulations | |
| CN107180241A (en) | A kind of animal classification method of the profound neutral net based on Gabor characteristic with fractal structure | |
| CN108564166A (en) | Based on the semi-supervised feature learning method of the convolutional neural networks with symmetrical parallel link | |
| CN103984954B (en) | Image combining method based on multi-feature fusion | |
| CN108537747A (en) | A kind of image repair method based on the convolutional neural networks with symmetrical parallel link | |
| Jin et al. | Remote sensing image fusion method in CIELab color space using nonsubsampled shearlet transform and pulse coupled neural networks | |
| CN104881635A (en) | Image texture extraction and identification method by non-Gauss two-dimension Gabor filter | |
| CN105825228B (en) | Image-recognizing method and device | |
| CN113298024A (en) | Unmanned aerial vehicle ground small target identification method based on lightweight neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100097 Beijing Haidian District Kunming Hunan Road 51 C block two floor 207. Applicant after: BEIJING ATHENA EYES SCIENCE & TECHNOLOGY CO.,LTD. Address before: 100193 4, 403, block A, 14 building, 10 East North Road, Haidian District, Beijing. Applicant before: BEIJING ATHENA EYES SCIENCE & TECHNOLOGY CO.,LTD. |
|
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 410205 14 Changsha Zhongdian Software Park Phase I, 39 Jianshan Road, Changsha High-tech Development Zone, Yuelu District, Changsha City, Hunan Province Applicant after: Wisdom Eye Technology Co.,Ltd. Address before: 100097 2nd Floor 207, Block C, 51 Hunan Road, Kunming, Haidian District, Beijing Applicant before: BEIJING ATHENA EYES SCIENCE & TECHNOLOGY CO.,LTD. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Image de grid method and device based on depth learning Effective date of registration: 20221205 Granted publication date: 20191112 Pledgee: Agricultural Bank of China Limited Hunan Xiangjiang New Area Branch Pledgor: Wisdom Eye Technology Co.,Ltd. Registration number: Y2022430000107 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20231220 Granted publication date: 20191112 Pledgee: Agricultural Bank of China Limited Hunan Xiangjiang New Area Branch Pledgor: Wisdom Eye Technology Co.,Ltd. Registration number: Y2022430000107 |
|
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: No. 205, Building B1, Huigu Science and Technology Industrial Park, No. 336 Bachelor Road, Bachelor Street, Yuelu District, Changsha City, Hunan Province, 410000 Patentee after: Wisdom Eye Technology Co.,Ltd. Address before: 410205 building 14, phase I, Changsha Zhongdian Software Park, No. 39, Jianshan Road, Changsha high tech Development Zone, Yuelu District, Changsha City, Hunan Province Patentee before: Wisdom Eye Technology Co.,Ltd. |