CN107424131B - Image based on deep learning removes grid method and device - Google Patents

Image based on deep learning removes grid method and device Download PDF

Info

Publication number
CN107424131B
CN107424131B CN201710575071.8A CN201710575071A CN107424131B CN 107424131 B CN107424131 B CN 107424131B CN 201710575071 A CN201710575071 A CN 201710575071A CN 107424131 B CN107424131 B CN 107424131B
Authority
CN
China
Prior art keywords
grid
image
network
data
trained
Prior art date
Application number
CN201710575071.8A
Other languages
Chinese (zh)
Other versions
CN107424131A (en
Inventor
丁建华
杨东
Original Assignee
智慧眼科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 智慧眼科技股份有限公司 filed Critical 智慧眼科技股份有限公司
Priority to CN201710575071.8A priority Critical patent/CN107424131B/en
Publication of CN107424131A publication Critical patent/CN107424131A/en
Application granted granted Critical
Publication of CN107424131B publication Critical patent/CN107424131B/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/001Image restoration
    • G06T5/005Retouching; Inpainting; Scratch removal
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62Methods or arrangements for recognition using electronic means
    • G06K9/6267Classification techniques
    • G06K9/6268Classification techniques relating to the classification paradigm, e.g. parametric or non-parametric approaches

Abstract

The invention discloses a kind of images based on deep learning to remove grid method and device, grid image is constructed online by net template, multi-class grid data corresponding with net template is generated, and sorter network and full convolutional network is respectively trained using multi-class grid data as training data;Classification processing, and handling as a result, carrying out grid to the grid image to be removed classified using trained full convolutional network according to classification processing are carried out to grid image to be removed using trained sorter network.Image provided by the invention based on deep learning removes grid method and device, using National Federation of Trade Unions's convolutional network as training network, carries out grid to grid image to be removed by trained total convolutional network and handles, learns relevant parameter;And classification processing is carried out to grid image to be removed in advance by trained sorter network, then accordingly grid is gone to handle the grid image to be removed progress classified by corresponding trained total convolutional network, go grid effect good.

Description

Image based on deep learning removes grid method and device

Technical field

The present invention relates to field of image processings, particularly, be related to a kind of image based on deep learning go grid method and Device.

Background technique

Biological identification technology has gradually been applied in the various scenes of authentication.Wherein most widely used is that face is known Other technology.In many business scenarios, the certificate photograph of the true man's human face photo and he or she that need to arrive collection in worksite is carried out Recognition of face processing.The certificate photograph obtained in business scenario is by adding grid to handle, so firstly the need of to band net The certificate photo of lattice carries out grid operations.

Certificate photo goes the technology of grid to change domain algorithm and image processing method at present.Transform-domain algorithm uses DCT (Discrete Cosine Transform, discrete cosine transform) or DWT (Discrete Wavelet Transform, from Dissipate wavelet transformation) changing image spatial domain come do grid removal processing, this algorithm process effect is general, and to picture noise pole It is sensitive.Using the algorithm of traditional image procossing, such as the operation such as smothing filtering and refinement, image lattice is needed to be aligned, and The algorithm robustness is too poor.

Therefore, the grid poor removal effect in the presence of the method for existing removal grid, is a skill urgently to be resolved Art problem.

Summary of the invention

The present invention provides a kind of images based on deep learning to remove grid method and device, to solve the removal grid having Method in the presence of grid poor removal effect the technical issues of.

The technical solution adopted by the invention is as follows:

According to an aspect of the present invention, a kind of image based on deep learning is provided and removes grid method, comprising steps of

Grid of different sizes in multiple original mesh images is extracted, a variety of nets corresponding with the type of grid are constructed Grid template;

Grid image is constructed online using a variety of net templates, generates multi-class grid number corresponding with net template According to;

Sorter network and full convolutional network are respectively trained as training data for the multi-class grid data of generation;

Classification processing is carried out to grid image to be removed using trained sorter network;And according to the knot of classification processing Fruit carries out grid to the grid image to be removed classified using trained full convolutional network and handles.

Further, grid of different sizes in multiple original mesh images is extracted, is constructed opposite with the type of grid The step of a variety of net templates answered includes:

The different grids in multiple the original mesh images collected are extracted, the pixel size of the grid of extraction is obtained;

According to the pixel size of the grid of acquisition, classify to grid, by grid dividing at multiple types, and construct with The corresponding net template of the type of grid.

Further, grid image is constructed online using a variety of net templates, generate multiclass corresponding with net template The step of other grid data includes:

A variety of net templates are superimposed in non-screening table images, construct corresponding grid image online;

Non- screening table images and grid image are integrated to form training image mapping pair, and the training image of formation is mapped To as multi-class grid data.

Further, sorter network includes two sorter networks, and multi-class grid data includes refined net data and coarse grid Data, the step of sorter network and full convolutional network are respectively trained as training data for the multi-class grid data of generation packet It includes:

Refined net data and coarse grid data are trained into two sorter networks as training data, trained two are classified Network is as two sorter network models;

Refined net data and coarse grid data are trained into full convolutional network as training data, by trained full convolution Network is as corresponding full convolutional network model.

Further, include: to the step of grid image to be removed progress classification processing using trained sorter network

Classification processing is carried out to grid image to be removed using sorter network, judges the network mould of grid image to be removed Formula;

According to the network mode judged, grid image to be removed is transported in corresponding full convolutional network model, with Grid image is removed after obtaining grid.

According to another aspect of the present invention, a kind of image based on deep learning is also provided and goes mesh device, comprising:

It constructs module and constructs the type with grid for extracting grid of different sizes in multiple original mesh images Corresponding a variety of net templates;

Generation module generates corresponding with net template for constructing grid image online using a variety of net templates Multi-class grid data;

Training template, for using generate multi-class grid data be respectively trained as training data sorter network and entirely Convolutional network;

Grid processing module, for carrying out classification processing to grid image to be removed using trained sorter network;And According to classification processing as a result, being carried out at grid using trained full convolutional network to the grid image to be removed classified Reason.

Further, building module includes extraction unit and construction unit,

Extraction unit obtains the grid of extraction for extracting the different grids in multiple the original mesh images collected Pixel size;

Construction unit classifies to grid, for the pixel size according to the grid of acquisition by grid dividing at a variety of Type, and construct net template corresponding with the type of grid.

Further, generation module includes structural unit and integral unit,

Structural unit constructs corresponding grid chart for a variety of net templates to be superimposed in non-screening table images online Picture;

Integral unit for integrating non-screening table images and grid image to form training image mapping pair, and will be formed Training image map to as multi-class grid data.

Further, sorter network includes two sorter networks, and multi-class grid data includes refined net data and coarse grid Data, training template include the first training unit and the second training unit,

First training unit, for refined net data and coarse grid data to be trained two classification nets as training data Network, using trained two sorter network as two sorter network models;

Second training unit, for refined net data and coarse grid data to be trained full convolution net as training data Network, using trained full convolutional network as corresponding full convolutional network model.

Further, grid processing module includes judging unit and supply unit,

Judging unit judges grid to be removed for carrying out classification processing to grid image to be removed using sorter network The network mode of image;

Supply unit, for according to the network mode judged, grid image to be removed to be transported to corresponding full convolution In network model, to obtain removing grid image after grid.

The invention has the following advantages:

Image provided by the invention based on deep learning removes grid method and device, constructs net online by net template Table images generate multi-class grid data corresponding with net template, and using multi-class grid data as training data point It Xun Lian not sorter network and full convolutional network;Classification processing is carried out to grid image to be removed using trained sorter network, And according to classification processing as a result, carrying out grid to the grid image to be removed classified using trained full convolutional network Processing.Image provided by the invention based on deep learning removes grid method and device, using National Federation of Trade Unions's convolutional network as training net Network carries out grid to grid image to be removed by trained total convolutional network and handles, with the grid to be removed with grid Image is exported without the removal grid image of grid as target as being originally inputted, and learns relevant parameter;And pass through instruction The sorter network perfected carries out classification processing to grid image to be removed in advance, then by corresponding trained total convolutional network pair The grid image to be removed progress classified accordingly goes grid to handle, and goes the effect of grid good.

Other than objects, features and advantages described above, there are also other objects, features and advantages by the present invention. Below with reference to figure, the present invention is described in further detail.

Detailed description of the invention

The attached drawing constituted part of this application is used to provide further understanding of the present invention, schematic reality of the invention It applies example and its explanation is used to explain the present invention, do not constitute improper limitations of the present invention.In the accompanying drawings:

Fig. 1 is that the present invention is based on the flow diagrams that the image of deep learning goes grid method preferred embodiment;

Fig. 2 is that grid of different sizes in multiple original mesh images is extracted in Fig. 1, is constructed opposite with the type of grid The refinement flow diagram of the step of a variety of net templates answered;

Fig. 3 is the image schematic diagram for the original image collected in the present invention;

Fig. 4 is the net template schematic diagram according to a variety of net templates of the original image building in Fig. 3;

Fig. 5 is to construct grid image online using a variety of net templates in Fig. 1, generates multiclass corresponding with net template The refinement flow diagram of the step of other grid data;

Fig. 6 is the image schematic diagram of the non-screening table images of the present invention;

Fig. 7 is that the corresponding grid image constructed after net template is superimposed on the basis of the non-screening table images of Fig. 6 Image schematic diagram;

Fig. 8 is that sorter network and full volume is respectively trained in Fig. 1 using the multi-class grid data of generation as training data The refinement flow diagram of the step of product network;

Fig. 9 is the schematic network structure of sorter network in the present invention;

Figure 10 is the schematic network structure of full convolutional network in the present invention;

Figure 11 is thin the step of carrying out classification processing to grid image to be removed using trained sorter network in Fig. 1 Change flow diagram;

Figure 12 is that the present invention is based on the functional block diagrams that the image of deep learning goes mesh device preferred embodiment;

Figure 13 is the functional block diagram that module is constructed in Figure 12;

Figure 14 is the functional block diagram of generation module in Figure 12;

Figure 15 is the functional block diagram of training module in Figure 12;

Figure 16 is the functional block diagram of grid processing module in Figure 12.

Drawing reference numeral explanation:

10, module is constructed;20, generation module;30, training template;40, grid processing module;11, extraction unit;12, structure Build unit;21, structural unit;22, integral unit;31, the first training unit;32, the second training unit;41, judging unit; 42, supply unit.

Specific embodiment

It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase Mutually combination.The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

Referring to Fig.1, the preferred embodiment of the present invention provides a kind of image based on deep learning and removes grid method, including Step:

Step S100, grid of different sizes in multiple original mesh images is extracted, is constructed opposite with the type of grid The a variety of net templates answered.

As shown in Figure 3 and Figure 4, the different original mesh image of multiple grids is collected, size in original mesh image is extracted Different grids constructs a variety of net templates corresponding with the type of grid.

Step S200, grid image is constructed online using a variety of net templates, generate multiclass corresponding with net template Other grid data.

Grid image is constructed online using a variety of net templates, based on not band grid image, generates corresponding grid Image band grid image and corresponding grid image will not combine to form training image mapping pair, and the training of formation will be schemed As mapping is to as multi-class grid data corresponding with net template.And the corresponding grid image generated is subjected to data and is added By force, a variety of noises are added, to guarantee the robustness of data, and the grid image generated are made to be more nearly true grid image.

Step S300, sorter network and full convolution are respectively trained as training data for the multi-class grid data of generation Network.

According to the thickness of grid, the multi-class grid data of generation is divided into two classes, one kind is coarse grid data, another kind of For refined net data.The coarse grid data of generation and refined net data are trained into a sorter network as training data, Corresponding FCN (Fully Convolutional Networks, full convolutional network) is trained to every a kind of grid simultaneously.

Step S400, classification processing is carried out to grid image to be removed using trained sorter network;And according to classification Processing is handled as a result, carrying out grid to the grid image to be removed classified using trained full convolutional network.

In test phase, classification processing is carried out to grid image to be removed using trained sorter network;And according to point Handling as a result, carrying out grid to the grid image to be removed classified using trained FCN for class processing, obtains grid Afterwards remove grid image.

Image provided in this embodiment based on deep learning removes grid method, constructs grid chart online by net template Picture generates multi-class grid data corresponding with net template, and instructs respectively using multi-class grid data as training data Practice sorter network and full convolutional network;Classification processing, and root are carried out to grid image to be removed using trained sorter network According to classification processing as a result, being carried out at grid using trained full convolutional network to the grid image to be removed classified Reason.Image provided in this embodiment based on deep learning removes grid method, using National Federation of Trade Unions's convolutional network as training network, passes through Trained total convolutional network carries out grid to grid image to be removed and handles, using the grid image to be removed with grid as It is originally inputted, is exported without the removal grid image of grid as target, learn relevant parameter;And pass through trained point Class network carries out classification processing to grid image to be removed in advance, then by corresponding trained total convolutional network to having classified Grid image progress to be removed accordingly goes grid to handle, and goes the effect of grid good.

Preferably, as shown in Fig. 2, the image provided in this embodiment based on deep learning removes grid method, step S100 Include:

Step S110, the different grids in multiple the original mesh images collected are extracted, the picture of the grid of extraction is obtained Plain size.

See Fig. 3 and Fig. 4, the different original mesh image of multiple grids is collected, extracts multiple original meshes collected Different grids in image, obtain the pixel size of the grid of extraction.

Step S120, according to the pixel size of the grid of acquisition, classify to grid, by grid dividing at multiple types Type, and construct net template corresponding with the type of grid.

The pixel size for obtaining grid, classifies to grid according to the pixel size of the grid of acquisition, by grid dividing At multiple types, and construct net template corresponding with the type of grid.Specifically, in the present embodiment, by grid dividing At two class of refined net and coarse grid, classification foundation is that less than 5 pixels of net lattice control width are refined net in net template, greatly In 5 pixels be coarse grid (referring to fig. 4, the first row secondary series and the second row first are classified as refined net, remaining is coarse grid).

Image provided in this embodiment based on deep learning removes grid method, extracts multiple the original mesh figures collected Different grids as in, obtain the pixel size of the grid of extraction;According to the pixel size of the grid of acquisition, grid is divided Class by grid dividing at multiple types, and constructs net template corresponding with the type of grid.It is provided in this embodiment to be based on The image of deep learning removes grid method, according to the pixel size of the grid of acquisition, classifies to grid, by grid dividing at Multiple types, and net template corresponding with the type of grid is constructed, guarantee the diversity of sample, thus the classification made Network and full convolutional network are preferably trained and are learnt, and the better effect of grid is made, and card is gone after network processes Grid image is as far as possible close to original image.

Preferably, as shown in figure 5, the image provided in this embodiment based on deep learning removes grid method, step S200 Include:

Step S210, a variety of net templates are superimposed in non-screening table images, construct corresponding grid image online.

As shown in Figure 6 and Figure 7, according to a variety of net templates constructed, based on not band grid image, by a variety of nets Grid template is superimposed in non-screening table images, constructs corresponding grid image online.

Step S220, non-screening table images and grid image are integrated to form training image mapping pair, and by the instruction of formation Practice image mapping to as multi-class grid data.

Non- screening table images and corresponding grid image are integrated, form training image mapping pair, and by formation Training image mapping to as multi-class grid data to be trained respectively to sorter network and full convolutional network.

Image provided in this embodiment based on deep learning removes grid method, by being superimposed on a variety of net templates not In screening table images, corresponding grid image is constructed online;Non- screening table images and grid image are integrated to form training image Mapping pair, and the training image of formation is mapped to as multi-class grid data.It is provided in this embodiment to be based on deep learning Image remove grid method, using training image mapping to sorter network and full convolutional network is trained respectively, make to train Sorter network and full convolutional network grid is carried out to grid image to be removed and is handled, go the effect of grid good;It is online to generate Multi-class grid data in a variety of net templates are added, guarantee the diversity of sample.

Preferably, as shown in figure 8, the image provided in this embodiment based on deep learning removes grid method, sorter network Including two sorter networks, multi-class grid data includes refined net data and coarse grid data, and step S300 includes:

Step S310, refined net data and coarse grid data are trained into two sorter networks as training data, will trained Two good sorter networks are as two sorter network models.

The refined net data constructed and coarse grid data are trained into two sorter networks as training data, will be instructed Two sorter networks perfected as two sorter network models to be detected to images to be recognized, classify network such as Fig. 9 institute Show, sorter network includes the first convolutional layer, the second convolutional layer, third convolutional layer, the first full articulamentum and the second full articulamentum.When So, the first excitation layer and the first pond layer can also be connect behind the first convolutional layer, can connect the second excitation behind the second convolutional layer Layer and the second pond layer can connect third excitation layer and third pond layer behind third convolutional layer.The input of first convolutional layer is 112 × 112 × 3 sized images, the convolution kernel for being 5 × 5 followed by 20 scales;The core of first pond layer is 3 × 3, step-length 2. Second convolutional layer has 40 3 × 3 convolution kernels;Third convolutional layer has 60 3 × 3 convolution kernels.First excitation layer, the second excitation Layer is identical with the structure of third excitation layer, and the structure of the first pond layer, the second pond layer and the second pond layer is identical.First connects entirely The characteristic length for connecing layer is 128.Second full articulamentum is divided to refined net and two class of coarse grid to be exported.In the present embodiment, net It is 112*112*3 that network, which inputs size, after three convolutional layers, carries out the first full articulamentum processing, and formation length is the of 128 One feature vector chart (featureMap) reconnects the second class label that connection corresponds to that length is 2 entirely and exports.

In the present embodiment, classification task is one two classification, and using cross entropy loss function, formula is as follows:

Wherein, piIndicate that the sample is the probability of refined net,Value be 0 or 1, respectively correspond the reality of the sample Label is refined net or coarse grid.

Step S320, refined net data and coarse grid data are trained into full convolutional network as training data, will trained Good full convolutional network is as corresponding full convolutional network model.

The refined net data constructed and coarse grid data are trained into FCN network as training data, it will be trained FCN network is as corresponding full convolutional network model to detect to images to be recognized.FCN network structure is as indicated by 10: FCN Network includes Volume Four lamination, the 5th convolutional layer and the 6th convolutional layer, and the input of FCN network is 56 × 56 × 3 sized images, the The size of one convolution kernel is 9 × 9, and the number of core is 64, and the size for mending side (pad) is 6, and treated second for Volume Four lamination Feature vector chart (featureMap) size is 60 × 60 × 64.Second convolution kernel size is 5 × 5, and the number of core is 32, Pad is 1, is 32 × 60 × 60 by the third feature vectogram featureMap size that the 5th convolutional layer is handled well.Third Convolution kernel size be 5 × 5, number 3, pad is 0, by the 6th convolutional layer processing after fourth feature vectogram size be 56 × 56 × 3, and input it is identical, this method is also called the learning method of end-to-end (end to end).In addition, in Volume Four lamination Behind connect the first excitation layer (Relu), connect behind the 5th convolutional layer the second excitation layer (Relu) and carry out non-linear place Reason.Finally using Euclidean distance as loss function, reaction is input to the error of output.In the present embodiment, in full convolution net In network, without full articulamentum, only convolutional layer and excitation layer.Setting three-layer coil product herein, passes through setting pad and convolution kernel size Etc. parameters, guarantee three-layer coil product processing after output and input it is equal in magnitude.Using Euclidean distance as loss function, guarantee warp Image after crossing network processes is as far as possible close to original image (non-screening table images).When training, using lesser learning rate, very much One of to ten a ten thousandths, guarantee preferable convergence effect.

Image provided in this embodiment based on deep learning removes grid method, by by refined net data and coarse grid number According to two sorter networks are trained as training data, using trained two sorter network as two sorter network models;By fine-structure mesh Lattice data and coarse grid data train full convolutional network as training data, using trained full convolutional network as corresponding Full convolutional network model.Image provided in this embodiment based on deep learning removes grid method, by the FCN net of end to end Network structure is applied to grid and handles, and goes the effect of grid good.

Preferably, as shown in figure 11, the image provided in this embodiment based on deep learning removes grid method, step S400 Include:

Step S410, classification processing is carried out to grid image to be removed using sorter network, judges grid image to be removed Network mode.

The grid image to be removed for inputting sorter network is carried out classification processing, judges grid chart to be removed by test phase It seem to belong to coarse grid mode or refined net mode.

Step S420, according to the network mode judged, grid image to be removed is transported to corresponding full convolutional network In model, to obtain removing grid image after grid.

The network mode finally judged according to sorter network conveys grid image to be removed for different models Into corresponding full convolutional network model, to obtain removing grid image after grid.

Image provided in this embodiment based on deep learning removes grid method, by using sorter network to net to be removed Table images carry out classification processing, judge the network mode of grid image to be removed;It, will be to be removed according to the network mode judged Grid image is transported in corresponding full convolutional network model, to obtain removing grid image after grid.The present embodiment provides The image based on deep learning remove grid method, grid image to be removed is divided in advance by trained sorter network Class processing, then accordingly grid is gone to handle the grid image to be removed progress classified by corresponding full convolutional network model, it goes The effect of grid is good, and guarantees to go grid image as far as possible close to original image after network processes.

Preferably, as shown in figure 12, the present embodiment also provides a kind of image based on deep learning and goes mesh device, wraps Include: building module 10 constructs opposite with the type of grid for extracting grid of different sizes in multiple original mesh images The a variety of net templates answered;Generation module 20 generates and grid mould for constructing grid image online using a variety of net templates The corresponding multi-class grid data of plate;Training template 30, the multi-class grid data for that will generate is as training data Sorter network and full convolutional network is respectively trained;Grid processing module 40, for using trained sorter network to be removed Grid image carries out classification processing;And according to classification processing as a result, using trained full convolutional network to classified to Removal grid image carries out grid and handles.

As shown in Figure 3 and Figure 4, building module 10 collects the different original mesh image of multiple grids, extracts original mesh Grid of different sizes in image constructs a variety of net templates corresponding with the type of grid.

Generation module 20 constructs grid image using a variety of net templates online, based on not band grid image, generates Corresponding grid image band grid image and corresponding grid image will not combine to form training image mapping pair, and by shape At training image map to as multi-class grid data corresponding with net template.And the grid image generated will be corresponded to Data reinforcement is carried out, a variety of noises are added, to guarantee the robustness of data, and is more nearly the grid image generated really Grid image.

According to the thickness of grid, the multi-class grid data of generation is divided into two classes, one kind is coarse grid data, another kind of For refined net data.The coarse grid data of generation and refined net data are trained one as training data by training template 30 Sorter network, while corresponding FCN (Fully Convolutional Networks, full convolution net are trained to every a kind of grid Network).

In test phase, grid processing module 40 classifies to grid image to be removed using trained sorter network Processing;And according to classification processing as a result, being carried out at grid using trained FCN to the grid image to be removed classified Reason, removes grid image after obtaining grid.

Image provided in this embodiment based on deep learning goes mesh device, constructs grid chart online by net template Picture generates multi-class grid data corresponding with net template, and instructs respectively using multi-class grid data as training data Practice sorter network and full convolutional network;Classification processing, and root are carried out to grid image to be removed using trained sorter network According to classification processing as a result, being carried out at grid using trained full convolutional network to the grid image to be removed classified Reason.Image provided in this embodiment based on deep learning goes mesh device, using National Federation of Trade Unions's convolutional network as training network, passes through Trained total convolutional network carries out grid to grid image to be removed and handles, using the grid image to be removed with grid as It is originally inputted, is exported without the removal grid image of grid as target, learn relevant parameter;And pass through trained point Class network carries out classification processing to grid image to be removed in advance, then by corresponding trained total convolutional network to having classified Grid image progress to be removed accordingly goes grid to handle, and goes the effect of grid good.

Preferably, as shown in figure 13, the present embodiment also provides a kind of image based on deep learning and goes mesh device, constructs Module 10 includes extraction unit 11 and construction unit 12, extraction unit 11, for extracting multiple the original mesh images collected In different grids, obtain the pixel size of the grid of extraction;Construction unit 12, the pixel for the grid according to acquisition are big It is small, classify to grid, by grid dividing at multiple types, and constructs net template corresponding with the type of grid.

See Fig. 3 and Fig. 4, the different original mesh image of multiple grids is collected, extracts multiple original meshes collected Different grids in image, obtain the pixel size of the grid of extraction.

The pixel size for obtaining grid, classifies to grid according to the pixel size of the grid of acquisition, by grid dividing At multiple types, and construct net template corresponding with the type of grid.Specifically, in the present embodiment, by grid dividing At two class of refined net and coarse grid, classification foundation is that less than 5 pixels of net lattice control width are refined net in net template, greatly In 5 pixels be coarse grid (referring to fig. 4, the first row secondary series and the second row first are classified as refined net, remaining is coarse grid).

Image provided in this embodiment based on deep learning goes mesh device, extracts multiple the original mesh figures collected Different grids as in, obtain the pixel size of the grid of extraction;According to the pixel size of the grid of acquisition, grid is divided Class by grid dividing at multiple types, and constructs net template corresponding with the type of grid.It is provided in this embodiment to be based on The image of deep learning goes mesh device, according to the pixel size of the grid of acquisition, classifies to grid, by grid dividing at Multiple types, and net template corresponding with the type of grid is constructed, guarantee the diversity of sample, thus the classification made Network and full convolutional network are preferably trained and are learnt, and the better effect of grid is made, and card is gone after network processes Grid image is as far as possible close to original image.

Preferably, as shown in figure 14, the present embodiment also provides a kind of image based on deep learning and goes mesh device, generates Module 20 includes structural unit 21 and integral unit 22, structural unit 21, for a variety of net templates to be superimposed on non-screening lattice In image, corresponding grid image is constructed online;Integral unit 22, for integrating non-screening table images and grid image to be formed Training image mapping pair, and the training image of formation is mapped to as multi-class grid data.

As shown in Figure 6 and Figure 7, structural unit 21 is according to a variety of net templates constructed, using not band grid image as base A variety of net templates are superimposed in non-screening table images, construct corresponding grid image online by plinth.

Integral unit 22 integrates non-screening table images and corresponding grid image, forms training image mapping pair, And by the training image of formation mapping to as multi-class grid data to be instructed respectively to sorter network and full convolutional network Practice.

Image provided in this embodiment based on deep learning goes mesh device, by being superimposed on a variety of net templates not In screening table images, corresponding grid image is constructed online;Non- screening table images and grid image are integrated to form training image Mapping pair, and the training image of formation is mapped to as multi-class grid data.It is provided in this embodiment to be based on deep learning Image go mesh device, using training image mapping to sorter network and full convolutional network is trained respectively, make to train Sorter network and full convolutional network grid is carried out to grid image to be removed and is handled, go the effect of grid good;It is online to generate Multi-class grid data in a variety of net templates are added, guarantee the diversity of sample.

Preferably, as shown in figure 15, the present embodiment also provides a kind of image based on deep learning and goes mesh device, training Template 30 includes the first training unit 31 and the second training unit 32, the first training unit 31, for by refined net data and slightly Grid data trains two sorter networks as training data, using trained two sorter network as two sorter network models; Second training unit 32 will be instructed for refined net data and coarse grid data to be trained full convolutional network as training data The full convolutional network perfected is as corresponding full convolutional network model.

The refined net data constructed and coarse grid data are trained one as training data by the first training unit 31 Two sorter networks, using trained two sorter network as two sorter network models to be detected to images to be recognized, classification Network as shown in figure 9, sorter network include the first convolutional layer, the second convolutional layer, third convolutional layer, the first full articulamentum and Second full articulamentum.Certainly, the first excitation layer and the first pond layer can also be connect behind the first convolutional layer, behind the second convolutional layer The second excitation layer and the second pond layer can be connect, third excitation layer and third pond layer can be connect behind third convolutional layer.First Convolutional layer input is 112 × 112 × 3 sized images, the convolution kernel for being 5 × 5 followed by 20 scales;The core of first pond layer It is 3 × 3, step-length 2.Second convolutional layer has 40 3 × 3 convolution kernels;Third convolutional layer has 60 3 × 3 convolution kernels.First Excitation layer, the second excitation layer are identical with the structure of third excitation layer, the knot of the first pond layer, the second pond layer and the second pond layer Structure is identical.The characteristic length of first full articulamentum is 128.Second full articulamentum is divided to refined net and two class of coarse grid to be exported. In the present embodiment, network inputs size is 112*112*3, after three convolutional layers, carries out the first full articulamentum processing, shape The first eigenvector figure (featureMap) for being 128 at length reconnects the second full connection and corresponds to the class label that length is 2 Output.

In the present embodiment, classification task is one two classification, and using cross entropy loss function, formula is as follows:

Wherein, piIndicate that the sample is the probability of refined net,Value be 0 or 1, respectively correspond the reality of the sample Label is refined net or coarse grid.

The refined net data constructed and coarse grid data are trained FCN net as training data by the second training unit 32 Network, using trained FCN network as corresponding full convolutional network model to be detected to images to be recognized.FCN network knot Structure is as indicated by 10: FCN network includes Volume Four lamination, the 5th convolutional layer and the 6th convolutional layer, and the input of FCN network is 56 × 56 × 3 sized images, the size of first convolution kernel are 9 × 9, and the number of core is 64, and the size for mending side (pad) is 6, Volume Four product Layer treated second feature vectogram (featureMap) size is 60 × 60 × 64.Second convolution kernel size is 5 × 5, The number of core is 32, pad 1, the third feature vectogram featureMap size handled well by the 5th convolutional layer is 32 × 60×60.Third convolution kernel size is 5 × 5, and number 3, pad is 0, the fourth feature vector after the processing of the 6th convolutional layer Figure size is 56 × 56 × 3, and input is identical, and this method is also called the learning method of end-to-end (end to end).In addition, Connected behind Volume Four lamination the first excitation layer (Relu), connect behind the 5th convolutional layer the second excitation layer (Relu) into Row Nonlinear Processing.Finally using Euclidean distance as loss function, reaction is input to the error of output.In the present embodiment, In full convolutional network, without full articulamentum, only convolutional layer and excitation layer.Herein setting three-layer coil product, by setting pad and It is equal in magnitude to guarantee that three-layer coil product processing is output and input later for the parameters such as convolution kernel size.Using Euclidean distance as loss Function guarantees the image after network processes as far as possible close to original image (non-screening table images).When training, use is lesser Learning rate, a ten thousandth to ten a ten thousandths guarantee preferable convergence effect.

Image provided in this embodiment based on deep learning goes mesh device, by by refined net data and coarse grid number According to two sorter networks are trained as training data, using trained two sorter network as two sorter network models;By fine-structure mesh Lattice data and coarse grid data train full convolutional network as training data, using trained full convolutional network as corresponding Full convolutional network model.Image provided in this embodiment based on deep learning goes mesh device, by the FCN net of end to end Network structure is applied to grid and handles, and goes the effect of grid good.

Preferably, as shown in figure 16, the present embodiment also provides a kind of image based on deep learning and removes mesh device, grid Processing module 40 includes judging unit 41 and supply unit 42, judging unit 41, for using sorter network to grid to be removed Image carries out classification processing, judges the network mode of grid image to be removed;Supply unit 42, for according to the network judged Grid image to be removed is transported in corresponding full convolutional network model by mode, to obtain removing grid image after grid.

Test phase, judging unit 41 will input sorter network grid image to be removed carry out classification processing, judge to Removal grid image belongs to coarse grid mode or refined net mode.

The network mode that supply unit 42 is finally judged according to sorter network, for different models, by net to be removed Table images are transported in corresponding full convolutional network model, to obtain removing grid image after grid.

Image provided in this embodiment based on deep learning goes mesh device, by using sorter network to net to be removed Table images carry out classification processing, judge the network mode of grid image to be removed;It, will be to be removed according to the network mode judged Grid image is transported in corresponding full convolutional network model, to obtain removing grid image after grid.The present embodiment provides The image based on deep learning go mesh device, grid image to be removed is divided in advance by trained sorter network Class processing, then accordingly grid is gone to handle the grid image to be removed progress classified by corresponding full convolutional network model, it goes The effect of grid is good, and guarantees to go grid image as far as possible close to original image after network processes.

The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.

Claims (2)

1. a kind of image based on deep learning removes grid method, which is characterized in that comprising steps of
Grid of different sizes in multiple original mesh images is extracted, a variety of nets corresponding with the type of the grid are constructed Grid template;
Grid image is constructed online using a variety of net templates, generates multi-class grid corresponding with the net template Data;
Sorter network and full convolutional network are respectively trained as training data for the multi-class grid data of generation;
Classification processing is carried out to grid image to be removed using the trained sorter network;And according to the knot of the classification Fruit carries out grid to the grid image to be removed classified using the trained full convolutional network and handles;
It is described to extract grid of different sizes in multiple original mesh images, it constructs corresponding with the type of the grid more The step of kind of net template includes:
The different grids in multiple the described original mesh images collected are extracted, the pixel for obtaining the grid of extraction is big It is small;
According to the pixel size of the grid of acquisition, classify to the grid, by the grid dividing at multiple types, And construct net template corresponding with the type of the grid;
It is described to construct grid image online using a variety of net templates, it generates corresponding with the net template multi-class The step of grid data includes:
A variety of net templates are superimposed in non-screening table images, construct corresponding grid image online;
The non-screening table images and the grid image are integrated to form training image mapping pair, and by the training of formation Image is mapped to as multi-class grid data;
The sorter network includes two sorter networks, and the multi-class grid data includes refined net data and coarse grid data, The step that the multi-class grid data of generation is respectively trained to sorter network and full convolutional network as training data Suddenly include:
The refined net data and the coarse grid data are trained into two sorter network as training data, will be trained Two sorter network as two sorter network models;
The refined net data and the coarse grid data are trained into the full convolutional network as training data, will be trained The full convolutional network as corresponding full convolutional network model;
It is described that classification processing is carried out to grid image to be removed using the trained sorter network;And according to the classification As a result, the step of grid processing is carried out to the grid image to be removed classified using the trained full convolutional network Suddenly include:
Classification processing is carried out to the grid image to be removed using the sorter network, judges the grid image to be removed Network mode;
According to the network mode judged, the grid image to be removed is transported to corresponding full convolutional network model In, to obtain removing grid image after grid.
2. a kind of image based on deep learning goes mesh device characterized by comprising
Building module (10) is constructed and the grid for extracting grid of different sizes in multiple original mesh images The corresponding a variety of net templates of type;
Generation module (20) generates and the net template for constructing grid image online using a variety of net templates Corresponding multi-class grid data;
Training template (30), for sorter network to be respectively trained as training data in the multi-class grid data generated With full convolutional network;
Grid processing module (40), for being carried out at classification using the trained sorter network to grid image to be removed Reason, carries out grid to the grid image to be removed classified using the trained full convolutional network and handles;
The building module (10) includes extraction unit (11) and construction unit (12),
Extraction unit (11) obtains extraction for extracting the different grids in multiple the described original mesh images collected The pixel size of the grid;
Construction unit (12) classifies to the grid, for the pixel size according to the grid of acquisition by the net Lattice are divided into multiple types, and construct net template corresponding with the type of the grid;
The generation module (20) includes structural unit (21) and integral unit (22),
Structural unit (21) constructs corresponding net for a variety of net templates to be superimposed in non-screening table images online Table images;
Integral unit (22), for integrating the non-screening table images and the grid image to form training image mapping pair, And the training image of formation is mapped to as multi-class grid data;
The sorter network includes two sorter networks, and the multi-class grid data includes refined net data and coarse grid data, The trained template (30) includes the first training unit (31) and the second training unit (32),
First training unit (31), for using the refined net data and the coarse grid data as training data to train Two sorter networks are stated, using trained two sorter network as two sorter network models;
Second training unit (32), for using the refined net data and the coarse grid data as training data to train Full convolutional network is stated, using the trained full convolutional network as corresponding full convolutional network model;
The grid processing module (40) includes judging unit (41) and supply unit (42),
Judging unit (41) judges institute for carrying out classification processing to the grid image to be removed using the sorter network State the network mode of grid image to be removed;
Supply unit (42), for according to the network mode judged, the grid image to be removed to be transported to correspondence Full convolutional network model in, to obtain removing grid image after grid.
CN201710575071.8A 2017-07-14 2017-07-14 Image based on deep learning removes grid method and device CN107424131B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710575071.8A CN107424131B (en) 2017-07-14 2017-07-14 Image based on deep learning removes grid method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710575071.8A CN107424131B (en) 2017-07-14 2017-07-14 Image based on deep learning removes grid method and device

Publications (2)

Publication Number Publication Date
CN107424131A CN107424131A (en) 2017-12-01
CN107424131B true CN107424131B (en) 2019-11-12

Family

ID=60426594

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710575071.8A CN107424131B (en) 2017-07-14 2017-07-14 Image based on deep learning removes grid method and device

Country Status (1)

Country Link
CN (1) CN107424131B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108230269A (en) * 2017-12-28 2018-06-29 北京智慧眼科技股份有限公司 Grid method, device, equipment and storage medium are gone based on depth residual error network

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101150658A (en) * 2007-08-22 2008-03-26 北京中星微电子有限公司 A bad point self-adapted grid noise elimination device and method
CN101272503A (en) * 2008-04-30 2008-09-24 北京中星微电子有限公司 Gridding noise elimination method and device for remaining image definition
CN101352047A (en) * 2005-12-29 2009-01-21 安泰科技有限公司 Method and device for removing grid noise
CN105125206A (en) * 2015-09-15 2015-12-09 中山大学 Intelligent electrocardio monitoring method and device
CN106709866A (en) * 2016-11-18 2017-05-24 北京智慧眼科技股份有限公司 Method and device for removing mesh watermark from identification photo, face verification method and device
CN106875360A (en) * 2017-02-17 2017-06-20 广州智能装备研究院有限公司 It is a kind of to eliminate the fuzzy method and device of image motion

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7840046B2 (en) * 2006-06-27 2010-11-23 Siemens Medical Solutions Usa, Inc. System and method for detection of breast masses and calcifications using the tomosynthesis projection and reconstructed images

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101352047A (en) * 2005-12-29 2009-01-21 安泰科技有限公司 Method and device for removing grid noise
CN101150658A (en) * 2007-08-22 2008-03-26 北京中星微电子有限公司 A bad point self-adapted grid noise elimination device and method
CN101272503A (en) * 2008-04-30 2008-09-24 北京中星微电子有限公司 Gridding noise elimination method and device for remaining image definition
CN105125206A (en) * 2015-09-15 2015-12-09 中山大学 Intelligent electrocardio monitoring method and device
CN106709866A (en) * 2016-11-18 2017-05-24 北京智慧眼科技股份有限公司 Method and device for removing mesh watermark from identification photo, face verification method and device
CN106875360A (en) * 2017-02-17 2017-06-20 广州智能装备研究院有限公司 It is a kind of to eliminate the fuzzy method and device of image motion

Also Published As

Publication number Publication date
CN107424131A (en) 2017-12-01

Similar Documents

Publication Publication Date Title
Ye et al. Unsupervised feature learning framework for no-reference image quality assessment
Fang et al. Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum
Rao et al. A deep learning approach to detection of splicing and copy-move forgeries in images
Khirade et al. Plant disease detection using image processing
Zhang et al. Image super-resolution using very deep residual channel attention networks
Gu et al. Leaf recognition based on the combination of wavelet transform and gaussian interpolation
Saeedi et al. Infrared and visible image fusion using fuzzy logic and population-based optimization
CN105938559B (en) Use the Digital Image Processing of convolutional neural networks
Thai et al. Image classification using support vector machine and artificial neural network
CN105160310A (en) 3D (three-dimensional) convolutional neural network based human body behavior recognition method
Krisshna et al. Face recognition using transform domain feature extraction and PSO-based feature selection
CN106204449B (en) A kind of single image super resolution ratio reconstruction method based on symmetrical depth network
CN105320965A (en) Hyperspectral image classification method based on spectral-spatial cooperation of deep convolutional neural network
Chierchia et al. A nonlocal structure tensor-based approach for multicomponent image recovery problems
Qian et al. Learning and transferring representations for image steganalysis using convolutional neural network
Zhao et al. Passive image-splicing detection by a 2-D noncausal Markov model
CN103116763B (en) A kind of living body faces detection method based on hsv color Spatial Statistical Character
CN105719188B (en) The anti-method cheated of settlement of insurance claim and server are realized based on plurality of pictures uniformity
CN103914847B (en) Based on phase equalization and the SAR image registration method of SIFT
CN101916365B (en) Intelligent video identifying method for cheat in test
CN102184557B (en) Salient region detection method for complex scene
CN104778481B (en) A kind of construction method and device of extensive face pattern analysis sample storehouse
CN101847210B (en) Multi-group image classification method based on two-dimensional empirical modal decomposition and wavelet denoising
CN104268521A (en) Image recognition method based on convolutional neural network in non-finite category
CN104392228B (en) Unmanned plane image object class detection method based on conditional random field models

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 100097 Beijing Haidian District Kunming Hunan Road 51 C block two floor 207.

Applicant after: ATHENA EYES SCIENCE & TECHNOLOGY CO., LTD.

Address before: 100193 4, 403, block A, 14 building, 10 East North Road, Haidian District, Beijing.

Applicant before: ATHENA EYES SCIENCE & TECHNOLOGY CO., LTD.

CB02 Change of applicant information
CB02 Change of applicant information

Address after: 410205 14 Changsha Zhongdian Software Park Phase I, 39 Jianshan Road, Changsha High-tech Development Zone, Yuelu District, Changsha City, Hunan Province

Applicant after: Wisdom Eye Technology Co., Ltd.

Address before: 100097 2nd Floor 207, Block C, 51 Hunan Road, Kunming, Haidian District, Beijing

Applicant before: ATHENA EYES SCIENCE & TECHNOLOGY CO., LTD.

GR01 Patent grant
GR01 Patent grant