CN107424131A - Image based on deep learning removes grid method and device - Google Patents

Image based on deep learning removes grid method and device Download PDF

Info

Publication number
CN107424131A
CN107424131A CN201710575071.8A CN201710575071A CN107424131A CN 107424131 A CN107424131 A CN 107424131A CN 201710575071 A CN201710575071 A CN 201710575071A CN 107424131 A CN107424131 A CN 107424131A
Authority
CN
China
Prior art keywords
grid
image
data
network
net
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710575071.8A
Other languages
Chinese (zh)
Other versions
CN107424131B (en
Inventor
丁建华
杨东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Athena Eyes Co Ltd
Original Assignee
Beijing Athena Eyes Science & Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Athena Eyes Science & Technology Co ltd filed Critical Beijing Athena Eyes Science & Technology Co ltd
Priority to CN201710575071.8A priority Critical patent/CN107424131B/en
Publication of CN107424131A publication Critical patent/CN107424131A/en
Application granted granted Critical
Publication of CN107424131B publication Critical patent/CN107424131B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/77Retouching; Inpainting; Scratch removal
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a kind of image based on deep learning to remove grid method and device, grid image is constructed by net template online, the generation multi-class grid data corresponding with net template, and sorter network and full convolutional network is respectively trained using multi-class grid data as training data;Classification processing is carried out to grid image to be removed using the sorter network trained, and according to the result of classification processing, grid is carried out to the grid image to be removed classified using the full convolutional network trained and handled.Image provided by the invention based on deep learning removes grid method and device, using National Federation of Trade Unions's convolutional network as training network, carries out grid to grid image to be removed by the total convolutional network trained and handles, learn relevant parameter;And the sorter network by training in advance carries out grid image to be removed classification processing, then the grid image to be removed classified is carried out accordingly going grid to handle by the corresponding total convolutional network trained, go grid effect good.

Description

Image based on deep learning removes grid method and device
Technical field
The present invention relates to image processing field, especially, be related to a kind of image based on deep learning go grid method and Device.
Background technology
Biological identification technology is gradually applied in the various scenes of authentication.Wherein most widely used is that face is known Other technology., it is necessary to which true man's human face photo and the certificate photograph of he or she that collection in worksite is arrived are carried out in many business scenarios Recognition of face is handled.The certificate photograph obtained in business scenario is by adding grid to handle, so firstly the need of to band net The certificate photo of lattice carries out grid operations.
Certificate photo goes the technology of grid to change domain algorithm and image processing method at present.Transform-domain algorithm uses DCT (Discrete Cosine Transform, discrete cosine transform) or DWT (Discrete Wavelet Transform, from Dissipating wavelet transformation) changing image spatial domain does grid removal processing, and this algorithm process effect is general, and to picture noise pole It is sensitive.Using the algorithm of traditional image procossing, such as the operation such as smothing filtering and refinement is, it is necessary to image lattice aligns, and The algorithm robustness is too poor.
Therefore, the grid poor removal effect in the presence of the existing method for removing grid, is a skill urgently to be resolved hurrily Art problem.
The content of the invention
The invention provides a kind of image based on deep learning to remove grid method and device, to solve the removal grid having Method in the presence of grid poor removal effect technical problem.
The technical solution adopted by the present invention is as follows:
According to an aspect of the present invention, there is provided a kind of image based on deep learning goes grid method, including step:
Grid of different sizes in multiple original mesh images is extracted, constructs a variety of nets corresponding with the type of grid Grid template;
Grid image, the generation multi-class grid number corresponding with net template are constructed online using a variety of net templates According to;
Sorter network and full convolutional network is respectively trained using the multi-class grid data of generation as training data;
Classification processing is carried out to grid image to be removed using the sorter network trained;And according to the knot of classification processing Fruit, grid is carried out to the grid image to be removed classified using the full convolutional network trained and handled.
Further, grid of different sizes in multiple original mesh images is extracted, is constructed relative with the type of grid The step of a variety of net templates answered, includes:
The different grids in multiple the original mesh images collected are extracted, obtain the pixel size of the grid of extraction;
According to the pixel size of the grid of acquisition, grid is classified, by mesh generation into polytype, and build with The corresponding net template of the type of grid.
Further, grid image, the generation multiclass corresponding with net template are constructed online using a variety of net templates The step of other grid data, includes:
A variety of net templates are superimposed in non-screening table images, grid image corresponding to online construction;
Non- screening table images and grid image are integrated to form training image mapping pair, and the training image of formation is mapped To as multi-class grid data.
Further, sorter network includes two sorter networks, and multi-class grid data includes refined net data and coarse grid Data, wrapped using the multi-class grid data of generation as training data the step of sorter network and full convolutional network is respectively trained Include:
Two sorter networks are trained using refined net data and coarse grid data as training data, two classification that will be trained Network is as two sorter network models;
Full convolutional network is trained using refined net data and coarse grid data as training data, the full convolution that will be trained Full convolutional network model corresponding to network conduct.
Further, being carried out the step of classification processing to grid image to be removed using the sorter network trained is included:
Classification processing is carried out to grid image to be removed using sorter network, judges the network mould of grid image to be removed Formula;
According to the network mode judged, grid image to be removed is transported in corresponding full convolutional network model, with Obtain removing grid image after grid.
According to another aspect of the present invention, a kind of image based on deep learning is also provided and goes mesh device, including:
Module is built, for extracting grid of different sizes in multiple original mesh images, constructs the type with grid Corresponding a variety of net templates;
Generation module, for constructing grid image online using a variety of net templates, generation is corresponding with net template Multi-class grid data;
Train template, for using the multi-class grid data of generation as training data come be respectively trained sorter network and entirely Convolutional network;
Grid processing module, for carrying out classification processing to grid image to be removed using the sorter network trained;And According to the result of classification processing, the grid image to be removed classified is carried out at grid using the full convolutional network trained Reason.
Further, structure module includes extraction unit and construction unit,
Extraction unit, for extracting the different grids in multiple the original mesh images collected, obtain the grid of extraction Pixel size;
Construction unit, for the pixel size of the grid according to acquisition, grid is classified, by mesh generation into a variety of Type, and build the net template corresponding with the type of grid.
Further, generation module includes structural unit and integral unit,
Structural unit, for a variety of net templates to be superimposed in non-screening table images, grid chart corresponding to online construction Picture;
Integral unit, for integrating non-screening table images and grid image to form training image mapping pair, and it will be formed Training image map to as multi-class grid data.
Further, sorter network includes two sorter networks, and multi-class grid data includes refined net data and coarse grid Data, training template include the first training unit and the second training unit,
First training unit, for training two classification nets using refined net data and coarse grid data as training data Network, using two sorter networks trained as two sorter network models;
Second training unit, for training full convolution net using refined net data and coarse grid data as training data Network, using the full convolutional network trained as corresponding to full convolutional network model.
Further, grid processing module includes judging unit and supply unit,
Judging unit, for carrying out classification processing to grid image to be removed using sorter network, judge grid to be removed The network mode of image;
Supply unit, for according to the network mode judged, grid image to be removed to be transported into corresponding full convolution In network model, to obtain removing grid image after grid.
The invention has the advantages that:
Image provided by the invention based on deep learning removes grid method and device, and net is constructed online by net template Table images, the generation multi-class grid data corresponding with net template, and using multi-class grid data as training data point Xun Lian not sorter network and full convolutional network;Classification processing is carried out to grid image to be removed using the sorter network trained, And according to the result of classification processing, grid is carried out to the grid image to be removed classified using the full convolutional network trained Processing.Image provided by the invention based on deep learning removes grid method and device, using National Federation of Trade Unions's convolutional network as training net Network, grid is carried out to grid image to be removed by the total convolutional network trained and handled, with the grid to be removed with grid Image is exported as being originally inputted without the removal grid image of grid as target, learns relevant parameter;And pass through instruction The sorter network perfected carries out classification processing to grid image to be removed in advance, then by the corresponding total convolutional network pair trained The grid image to be removed classified carries out accordingly going grid to handle, and goes the effect of grid good.
In addition to objects, features and advantages described above, the present invention also has other objects, features and advantages. Below with reference to figure, the present invention is further detailed explanation.
Brief description of the drawings
The accompanying drawing for forming the part of the application is used for providing a further understanding of the present invention, schematic reality of the invention Apply example and its illustrate to be used to explain the present invention, do not form inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is the schematic flow sheet that the image of the invention based on deep learning removes grid method preferred embodiment;
Fig. 2 is that grid of different sizes in multiple original mesh images is extracted in Fig. 1, is constructed relative with the type of grid The refinement schematic flow sheet of the step of a variety of net templates answered;
Fig. 3 is the image schematic diagram for the original image collected in the present invention;
Fig. 4 is the net template schematic diagram of a variety of net templates of the original image structure in Fig. 3;
Fig. 5 is to construct grid image, the generation multiclass corresponding with net template online using a variety of net templates in Fig. 1 The refinement schematic flow sheet of the step of other grid data;
Fig. 6 is the image schematic diagram of the non-screening table images of the present invention;
Fig. 7 is that the corresponding grid image constructed after net template is superimposed on the basis of Fig. 6 non-screening table images Image schematic diagram;
Fig. 8 is that sorter network and full volume is respectively trained using the multi-class grid data of generation as training data in Fig. 1 The refinement schematic flow sheet of the step of product network;
Fig. 9 is the schematic network structure of sorter network in the present invention;
Figure 10 is the schematic network structure of full convolutional network in the present invention;
Figure 11 is thin the step of carrying out classification processing in Fig. 1 to grid image to be removed using the sorter network trained Change schematic flow sheet;
Figure 12 is the functional block diagram that the image of the invention based on deep learning removes mesh device preferred embodiment;
Figure 13 is the high-level schematic functional block diagram that module is built in Figure 12;
Figure 14 is the high-level schematic functional block diagram of generation module in Figure 12;
Figure 15 is the high-level schematic functional block diagram of training module in Figure 12;
Figure 16 is the high-level schematic functional block diagram of grid processing module in Figure 12.
Drawing reference numeral explanation:
10th, module is built;20th, generation module;30th, template is trained;40th, grid processing module;11st, extraction unit;12nd, structure Build unit;21st, structural unit;22nd, integral unit;31st, the first training unit;32nd, the second training unit;41st, judging unit; 42nd, supply unit.
Embodiment
It should be noted that in the case where not conflicting, the feature in embodiment and embodiment in the application can phase Mutually combination.Describe the present invention in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Reference picture 1, the preferred embodiments of the present invention provide a kind of image based on deep learning and remove grid method, including Step:
Step S100, grid of different sizes in multiple original mesh images is extracted, is constructed relative with the type of grid The a variety of net templates answered.
As shown in Figure 3 and Figure 4, the different original mesh image of multiple grids is collected, extracts size in original mesh image Different grids, construct a variety of net templates corresponding with the type of grid.
Step S200, grid image, the generation multiclass corresponding with net template are constructed online using a variety of net templates Other grid data.
Grid image is constructed online using a variety of net templates, based on not band grid image, grid corresponding to generation Image, band grid image and corresponding grid image it will not be combined to form training image mapping pair, and by the training figure of formation As mapping is to as the multi-class grid data corresponding with net template.And the grid image of corresponding generation is subjected to data and added By force, a variety of noises are added, to ensure the robustness of data, and the grid image of generation is more nearly real grid image.
Step S300, sorter network and full convolution is respectively trained using the multi-class grid data of generation as training data Network.
According to the thickness of grid, the multi-class grid data of generation is divided into two classes, one kind is coarse grid data, another kind of For refined net data.A sorter network is trained using the coarse grid data of generation and refined net data as training data, Simultaneously to FCN (Fully Convolutional Networks, full convolutional network) corresponding to every a kind of grid training.
Step S400, classification processing is carried out to grid image to be removed using the sorter network trained;And according to classification The result of processing, grid is carried out to the grid image to be removed classified using the full convolutional network trained and handled.
In test phase, classification processing is carried out to grid image to be removed using the sorter network trained;And according to point The result of class processing, carries out grid to the grid image to be removed classified using the FCN trained and handles, obtain grid Afterwards remove grid image.
The image based on deep learning that the present embodiment provides removes grid method, and grid chart is constructed online by net template Picture, the generation multi-class grid data corresponding with net template, and instructed respectively using multi-class grid data as training data Practice sorter network and full convolutional network;Classification processing, and root are carried out to grid image to be removed using the sorter network trained According to the result of classification processing, the grid image to be removed classified is carried out at grid using the full convolutional network trained Reason.The image based on deep learning that the present embodiment provides removes grid method, using National Federation of Trade Unions's convolutional network as training network, passes through The total convolutional network trained carries out grid to grid image to be removed and handled, using the grid image to be removed with grid as It is originally inputted, is exported without the removal grid image of grid as target, learns relevant parameter;And point by training Class network carries out classification processing to grid image to be removed in advance, then by the corresponding total convolutional network trained to having classified Grid image to be removed carries out accordingly going grid to handle, and goes the effect of grid good.
Preferably, as shown in Fig. 2 the image based on deep learning that the present embodiment provides removes grid method, step S100 Including:
Step S110, the different grids in multiple original mesh images that extraction has been collected, the picture of the grid of extraction is obtained Plain size.
See Fig. 3 and Fig. 4, the different original mesh image of multiple grids is collected, extracts multiple original meshes collected Different grids in image, obtain the pixel size of the grid of extraction.
Step S120, according to the pixel size of the grid of acquisition, grid is classified, by mesh generation into multiple types Type, and build the net template corresponding with the type of grid.
The pixel size of grid is obtained, grid is classified according to the pixel size of the grid of acquisition, by mesh generation Into polytype, and build the net template corresponding with the type of grid.Specifically, in the present embodiment, by mesh generation Into refined net and the class of coarse grid two, classification foundation be in net template net lattice control width be less than 5 pixels for refined net, greatly In 5 pixels be coarse grid (referring to Fig. 4, the first row secondary series and the second row first are classified as refined net, and remaining is coarse grid).
The image based on deep learning that the present embodiment provides removes grid method, extracts multiple the original mesh figures collected Different grids as in, obtain the pixel size of the grid of extraction;According to the pixel size of the grid of acquisition, grid is divided Class, by mesh generation into polytype, and build the net template corresponding with the type of grid.The present embodiment provide based on The image of deep learning removes grid method, and according to the pixel size of the grid of acquisition, grid is classified, by mesh generation into Polytype, and the net template corresponding with the type of grid is built, ensure the diversity of sample, so as to make obtained classification Network and full convolutional network are preferably trained and learnt, and are made the better of grid, are demonstrate,proved and gone after network processes Grid image is as far as possible close to original image.
Preferably, as shown in figure 5, the image based on deep learning that the present embodiment provides removes grid method, step S200 Including:
Step S210, a variety of net templates are superimposed in non-screening table images, grid image corresponding to online construction.
As shown in Figure 6 and Figure 7, according to a variety of net templates constructed, based on not band grid image, by a variety of nets Grid template is superimposed in non-screening table images, grid image corresponding to online construction.
Step S220, non-screening table images and grid image are integrated to form training image mapping pair, and by the instruction of formation Practice image mapping to as multi-class grid data.
Non- screening table images and corresponding grid image are integrated, form training image mapping pair, and by formation Training image is mapped to being trained respectively to sorter network and full convolutional network as multi-class grid data.
The image based on deep learning that the present embodiment provides removes grid method, by the way that a variety of net templates are superimposed on not In screening table images, grid image corresponding to online construction;Non- screening table images and grid image are integrated to form training image Mapping pair, and the training image of formation is mapped to as multi-class grid data.The present embodiment provide based on deep learning Image remove grid method, using training image mapping to training sorter network and full convolutional network respectively, make to train Sorter network and full convolutional network grid is carried out to grid image to be removed and handled, go the effect of grid good;Online generation Multi-class grid data in add a variety of net templates, ensure the diversity of sample.
Preferably, as shown in figure 8, the image based on deep learning that the present embodiment provides removes grid method, sorter network Including two sorter networks, multi-class grid data includes refined net data and coarse grid data, and step S300 includes:
Step S310, two sorter networks are trained using refined net data and coarse grid data as training data, will trained Two good sorter networks are as two sorter network models.
Two sorter networks are trained using the refined net data constructed and coarse grid data as training data, will be instructed Two sorter networks perfected as two sorter network models to be detected to images to be recognized, classification network such as Fig. 9 institutes Show, sorter network includes the first convolutional layer, the second convolutional layer, the 3rd convolutional layer, the first full articulamentum and the second full articulamentum.When So, the first excitation layer and the first pond layer can also be connect behind the first convolutional layer, the second excitation can be connect behind the second convolutional layer Layer and the second pond layer, the 3rd excitation layer and the 3rd pond layer can be connect behind the 3rd convolutional layer.The input of first convolutional layer is 112 × 112 × 3 sized images, followed by the convolution kernel that 20 yardsticks are 5 × 5;The core of first pond layer is 3 × 3, step-length 2. Second convolutional layer has the convolution kernel of 40 3 × 3;3rd convolutional layer has the convolution kernel of 60 3 × 3.First excitation layer, the second excitation Layer is identical with the structure of the 3rd excitation layer, and the structure of the first pond layer, the second pond layer and the second pond layer is identical.First connects entirely The characteristic length for connecing layer is 128.Second full articulamentum is divided to refined net and the class of coarse grid two to be exported.In the present embodiment, net Network input size be 112*112*3, after three convolutional layers, carries out the first full articulamentum processing, formed length for 128 the One feature vector chart (featureMap), reconnect second and connect the class label output for corresponding to that length is 2 entirely.
In the present embodiment, classification task is one two classification, as follows using cross entropy loss function, its formula:
Wherein, piIt is the probability of refined net to represent the sample,Value be 0 or 1, respectively to should sample reality Label is refined net or coarse grid.
Step S320, full convolutional network is trained using refined net data and coarse grid data as training data, will trained Full convolutional network model corresponding to good full convolutional network conduct.
FCN networks are trained using the refined net data constructed and coarse grid data as training data, by what is trained FCN networks be used as corresponding to full convolutional network model to be detected to images to be recognized.FCN network structures are as indicated by 10:FCN Network includes Volume Four lamination, the 5th convolutional layer and the 6th convolutional layer, and the input of FCN networks is 56 × 56 × 3 sized images, the The size of one convolution kernel is 9 × 9, and the number of core is 64, and the size for mending side (pad) is 6, second after the processing of Volume Four lamination Feature vector chart (featureMap) size is 60 × 60 × 64.Second convolution kernel size is 5 × 5, and the number of core is 32, Pad is 1, and the third feature vectogram featureMap sizes handled well by the 5th convolutional layer are 32 × 60 × 60.3rd Convolution kernel size be 5 × 5, number 3, pad is 0, by the 6th convolutional layer processing after fourth feature vectogram size be 56 × 56 × 3, and input it is identical, this method is also called the learning method of end-to-end (end to end).In addition, in Volume Four lamination Behind connect the first excitation layer (Relu), connect behind the 5th convolutional layer the second excitation layer (Relu) and carry out non-linear place Reason.The error of output is finally input to as loss function, reaction using Euclidean distance.In the present embodiment, in full convolution net In network, without full articulamentum, only convolutional layer and excitation layer.Three-layer coil is set to accumulate herein, by setting pad and convolution kernel size Etc. parameter, input is equal with output size after ensureing three-layer coil product processing.Using Euclidean distance as loss function, ensure warp The image crossed after network processes is as far as possible close to original image (non-screening table images).During training, using less learning rate, very much One of to ten a ten thousandths, ensure preferably convergence effect.
The image based on deep learning that the present embodiment provides removes grid method, by by refined net data and coarse grid number According to two sorter networks are trained as training data, using two sorter networks trained as two sorter network models;By fine-structure mesh Lattice data and coarse grid data train full convolutional network as training data, using the full convolutional network trained as corresponding to Full convolutional network model.The image based on deep learning that the present embodiment provides removes grid method, by end to end FCN nets Network structure is applied to grid and handled, and goes the effect of grid good.
Preferably, as shown in figure 11, the image based on deep learning that the present embodiment provides removes grid method, step S400 Including:
Step S410, classification processing is carried out to grid image to be removed using sorter network, judges grid image to be removed Network mode.
Test phase, the grid image to be removed for inputting sorter network is subjected to classification processing, judges grid chart to be removed Seem to belong to coarse grid pattern or refined net pattern.
Step S420, according to the network mode judged, grid image to be removed is transported to corresponding full convolutional network In model, to obtain removing grid image after grid.
The network mode finally judged according to sorter network, for different models, grid image to be removed is conveyed Into corresponding full convolutional network model, to obtain removing grid image after grid.
The image based on deep learning that the present embodiment provides removes grid method, by using sorter network to net to be removed Table images carry out classification processing, judge the network mode of grid image to be removed;, will be to be removed according to the network mode judged Grid image is transported in corresponding full convolutional network model, to obtain removing grid image after grid.The present embodiment provides The image based on deep learning remove grid method, grid image to be removed is divided in advance by the sorter network trained Class processing, then the grid image to be removed classified is carried out accordingly going grid to handle by corresponding full convolutional network model, go The effect of grid is good, and ensures to go grid image as far as possible close to original image after network processes.
Preferably, as shown in figure 12, the present embodiment also provides a kind of image based on deep learning and goes mesh device, bag Include:Module 10 is built, for extracting grid of different sizes in multiple original mesh images, is constructed relative with the type of grid The a variety of net templates answered;Generation module 20, for constructing grid image, generation and grid mould online using a variety of net templates The corresponding multi-class grid data of plate;Template 30 is trained, for using the multi-class grid data of generation as training data Sorter network and full convolutional network is respectively trained;Grid processing module 40, for using the sorter network trained to be removed Grid image carries out classification processing;And according to the result of classification processing, treated using the full convolutional network trained to what is classified Removal grid image carries out grid and handled.
As shown in Figure 3 and Figure 4, build module 10 and collect the different original mesh image of multiple grids, extract original mesh Grid of different sizes in image, construct a variety of net templates corresponding with the type of grid.
Generation module 20 constructs grid image online using a variety of net templates, based on not band grid image, generation Corresponding grid image, band grid image and corresponding grid image it will not be combined to form training image mapping pair, and by shape Into training image map to as the multi-class grid data corresponding with net template.And the grid image that generation will be corresponded to Data reinforcement is carried out, a variety of noises is added, to ensure the robustness of data, and the grid image of generation is more nearly really Grid image.
According to the thickness of grid, the multi-class grid data of generation is divided into two classes, one kind is coarse grid data, another kind of For refined net data.Template 30 is trained using the coarse grid data of generation and refined net data as training data to train one Sorter network, while to FCN (Fully Convolutional Networks, full convolution net corresponding to every a kind of grid training Network).
In test phase, grid processing module 40 is classified using the sorter network trained to grid image to be removed Processing;And according to the result of classification processing, the grid image to be removed classified is carried out at grid using the FCN trained Reason, obtains removing grid image after grid.
The image based on deep learning that the present embodiment provides goes mesh device, and grid chart is constructed online by net template Picture, the generation multi-class grid data corresponding with net template, and instructed respectively using multi-class grid data as training data Practice sorter network and full convolutional network;Classification processing, and root are carried out to grid image to be removed using the sorter network trained According to the result of classification processing, the grid image to be removed classified is carried out at grid using the full convolutional network trained Reason.The image based on deep learning that the present embodiment provides goes mesh device, using National Federation of Trade Unions's convolutional network as training network, passes through The total convolutional network trained carries out grid to grid image to be removed and handled, using the grid image to be removed with grid as It is originally inputted, is exported without the removal grid image of grid as target, learns relevant parameter;And point by training Class network carries out classification processing to grid image to be removed in advance, then by the corresponding total convolutional network trained to having classified Grid image to be removed carries out accordingly going grid to handle, and goes the effect of grid good.
Preferably, as shown in figure 13, the present embodiment also provides a kind of image based on deep learning and goes mesh device, structure Module 10 includes extraction unit 11 and construction unit 12, extraction unit 11, for extracting multiple the original mesh images collected In different grids, obtain the pixel size of the grid of extraction;Construction unit 12, the pixel for the grid according to acquisition are big It is small, grid is classified, by mesh generation into polytype, and builds the net template corresponding with the type of grid.
See Fig. 3 and Fig. 4, the different original mesh image of multiple grids is collected, extracts multiple original meshes collected Different grids in image, obtain the pixel size of the grid of extraction.
The pixel size of grid is obtained, grid is classified according to the pixel size of the grid of acquisition, by mesh generation Into polytype, and build the net template corresponding with the type of grid.Specifically, in the present embodiment, by mesh generation Into refined net and the class of coarse grid two, classification foundation be in net template net lattice control width be less than 5 pixels for refined net, greatly In 5 pixels be coarse grid (referring to Fig. 4, the first row secondary series and the second row first are classified as refined net, and remaining is coarse grid).
The image based on deep learning that the present embodiment provides goes mesh device, extracts multiple the original mesh figures collected Different grids as in, obtain the pixel size of the grid of extraction;According to the pixel size of the grid of acquisition, grid is divided Class, by mesh generation into polytype, and build the net template corresponding with the type of grid.The present embodiment provide based on The image of deep learning goes mesh device, and according to the pixel size of the grid of acquisition, grid is classified, by mesh generation into Polytype, and the net template corresponding with the type of grid is built, ensure the diversity of sample, so as to make obtained classification Network and full convolutional network are preferably trained and learnt, and are made the better of grid, are demonstrate,proved and gone after network processes Grid image is as far as possible close to original image.
Preferably, as shown in figure 14, the present embodiment also provides a kind of image based on deep learning and goes mesh device, generation Module 20 includes structural unit 21 and integral unit 22, structural unit 21, for a variety of net templates to be superimposed on into non-screening lattice In image, grid image corresponding to online construction;Integral unit 22, for integrating non-screening table images and grid image to be formed Training image mapping pair, and the training image of formation is mapped to as multi-class grid data.
As shown in Figure 6 and Figure 7, structural unit 21 is according to a variety of net templates constructed, using not band grid image as base Plinth, a variety of net templates are superimposed in non-screening table images, grid image corresponding to online construction.
Integral unit 22 is integrated non-screening table images and corresponding grid image, forms training image mapping pair, And the training image of formation is mapped to being instructed respectively to sorter network and full convolutional network as multi-class grid data Practice.
The image based on deep learning that the present embodiment provides goes mesh device, by the way that a variety of net templates are superimposed on not In screening table images, grid image corresponding to online construction;Non- screening table images and grid image are integrated to form training image Mapping pair, and the training image of formation is mapped to as multi-class grid data.The present embodiment provide based on deep learning Image go mesh device, using training image mapping to training sorter network and full convolutional network respectively, make to train Sorter network and full convolutional network grid is carried out to grid image to be removed and handled, go the effect of grid good;Online generation Multi-class grid data in add a variety of net templates, ensure the diversity of sample.
Preferably, as shown in figure 15, the present embodiment also provides a kind of image based on deep learning and goes mesh device, training Template 30 includes the first training unit 31 and the second training unit 32, the first training unit 31, for by refined net data and slightly Grid data trains two sorter networks as training data, using two sorter networks trained as two sorter network models; Second training unit 32, for training full convolutional network using refined net data and coarse grid data as training data, it will instruct Full convolutional network model corresponding to the full convolutional network conduct perfected.
First training unit 31 trains one using the refined net data constructed and coarse grid data as training data Two sorter networks, using two sorter networks trained as two sorter network models to be detected to images to be recognized, classify Network as shown in figure 9, sorter network include the first convolutional layer, the second convolutional layer, the 3rd convolutional layer, the first full articulamentum and Second full articulamentum.Certainly, the first excitation layer and the first pond layer can also be connect behind the first convolutional layer, behind the second convolutional layer The second excitation layer and the second pond layer can be connect, the 3rd excitation layer and the 3rd pond layer can be connect behind the 3rd convolutional layer.First Convolutional layer input is 112 × 112 × 3 sized images, followed by the convolution kernel that 20 yardsticks are 5 × 5;The core of first pond layer For 3 × 3, step-length 2.Second convolutional layer has the convolution kernel of 40 3 × 3;3rd convolutional layer has the convolution kernel of 60 3 × 3.First The structure of excitation layer, the second excitation layer and the 3rd excitation layer is identical, the knot of the first pond layer, the second pond layer and the second pond layer Structure is identical.The characteristic length of first full articulamentum is 128.Second full articulamentum is divided to refined net and the class of coarse grid two to be exported. In the present embodiment, network inputs size is 112*112*3, after three convolutional layers, carries out the first full articulamentum processing, shape Into the first eigenvector figure (featureMap) that length is 128, reconnect second and connect the class label for corresponding to that length is 2 entirely Output.
In the present embodiment, classification task is one two classification, as follows using cross entropy loss function, its formula:
Wherein, piIt is the probability of refined net to represent the sample,Value be 0 or 1, respectively to should sample actual mark Sign is refined net or coarse grid.
Second training unit 32 trains FCN nets using the refined net data constructed and coarse grid data as training data Network, using the FCN networks trained as corresponding to full convolutional network model to be detected to images to be recognized.FCN network knots Structure is as indicated by 10:FCN networks include Volume Four lamination, the 5th convolutional layer and the 6th convolutional layer, and the input of FCN networks is 56 × 56 × 3 sized images, the size of first convolution kernel is 9 × 9, and the number of core is 64, and the size for mending side (pad) is 6, Volume Four product Second feature vectogram (featureMap) size after layer processing is 60 × 60 × 64.Second convolution kernel size is 5 × 5, The number of core is 32, pad 1, the third feature vectogram featureMap sizes handled well by the 5th convolutional layer for 32 × 60×60.3rd convolution kernel size is 5 × 5, and number 3, pad is 0, the fourth feature vector after the processing of the 6th convolutional layer Figure size is 56 × 56 × 3, and input is identical, and this method is also called the learning method of end-to-end (end to end).In addition, The first excitation layer (Relu) is connected behind Volume Four lamination, the second excitation layer (Relu) is connected behind the 5th convolutional layer and enters Row Nonlinear Processing.The error of output is finally input to as loss function, reaction using Euclidean distance.In the present embodiment, In full convolutional network, without full articulamentum, only convolutional layer and excitation layer.Herein set three-layer coil accumulate, by set pad and The parameters such as convolution kernel size, input is equal with output size after ensureing three-layer coil product processing.Using Euclidean distance as loss Function, ensure the image after network processes as far as possible close to original image (non-screening table images).During training, use is less Learning rate, a ten thousandth to ten a ten thousandths, ensure preferably convergence effect.
The image based on deep learning that the present embodiment provides goes mesh device, by by refined net data and coarse grid number According to two sorter networks are trained as training data, using two sorter networks trained as two sorter network models;By fine-structure mesh Lattice data and coarse grid data train full convolutional network as training data, using the full convolutional network trained as corresponding to Full convolutional network model.The image based on deep learning that the present embodiment provides goes mesh device, by end to end FCN nets Network structure is applied to grid and handled, and goes the effect of grid good.
Preferably, as shown in figure 16, the present embodiment also provides a kind of image based on deep learning and removes mesh device, grid Processing module 40 includes judging unit 41 and supply unit 42, judging unit 41, for using sorter network to grid to be removed Image carries out classification processing, judges the network mode of grid image to be removed;Supply unit 42, for according to the network judged Pattern, grid image to be removed is transported in corresponding full convolutional network model, to obtain removing grid image after grid.
The grid image to be removed for inputting sorter network is carried out classification processing, judges to treat by test phase, judging unit 41 Remove grid image and belong to coarse grid pattern or refined net pattern.
The network mode that supply unit 42 is finally judged according to sorter network, for different models, by net to be removed Table images are transported in corresponding full convolutional network model, to obtain removing grid image after grid.
The image based on deep learning that the present embodiment provides goes mesh device, by using sorter network to net to be removed Table images carry out classification processing, judge the network mode of grid image to be removed;, will be to be removed according to the network mode judged Grid image is transported in corresponding full convolutional network model, to obtain removing grid image after grid.The present embodiment provides The image based on deep learning go mesh device, grid image to be removed is divided in advance by the sorter network trained Class processing, then the grid image to be removed classified is carried out accordingly going grid to handle by corresponding full convolutional network model, go The effect of grid is good, and ensures to go grid image as far as possible close to original image after network processes.
The preferred embodiments of the present invention are the foregoing is only, are not intended to limit the invention, for the skill of this area For art personnel, the present invention can have various modifications and variations.Within the spirit and principles of the invention, that is made any repaiies Change, equivalent substitution, improvement etc., should be included in the scope of the protection.

Claims (10)

1. a kind of image based on deep learning removes grid method, it is characterised in that including step:
Grid of different sizes in multiple original mesh images is extracted, constructs a variety of nets corresponding with the type of the grid Grid template;
Grid image, the generation multi-class grid corresponding with the net template are constructed online using a variety of net templates Data;
Sorter network and full convolutional network is respectively trained using the multi-class grid data of generation as training data;
Classification processing is carried out to grid image to be removed using the sorter network trained;And according to the knot of the classification Fruit, grid is carried out to the grid image to be removed classified using the full convolutional network trained and handled.
2. the image according to claim 1 based on deep learning removes grid method, it is characterised in that
It is described to extract grid of different sizes in multiple original mesh images, construct corresponding with the type of the grid more The step of kind net template, includes:
The different grids in multiple the described original mesh images collected are extracted, the pixel for obtaining the grid of extraction is big It is small;
According to the pixel size of the grid of acquisition, the grid is classified, by the mesh generation into polytype, And build the net template corresponding with the type of the grid.
3. the image according to claim 1 based on deep learning removes grid method, it is characterised in that
It is described to construct grid image online using a variety of net templates, multi-class corresponding with the net template of generation The step of grid data, includes:
A variety of net templates are superimposed in non-screening table images, grid image corresponding to online construction;
The non-screening table images and the grid image are integrated to form training image mapping pair, and by the training of formation Image is mapped to as multi-class grid data.
4. the image according to claim 1 based on deep learning removes grid method, it is characterised in that
The sorter network includes two sorter networks, and the multi-class grid data includes refined net data and coarse grid data, It is described that the step of sorter network and full convolutional network is respectively trained using the multi-class grid data of generation as training data Suddenly include:
Two sorter network is trained using the refined net data and the coarse grid data as training data, will be trained Two sorter network as two sorter network models;
The full convolutional network is trained using the refined net data and the coarse grid data as training data, will be trained The full convolutional network be used as corresponding to full convolutional network model.
5. the image according to claim 4 based on deep learning removes grid method, it is characterised in that
It is described that classification processing is carried out to grid image to be removed using the sorter network trained;And according to the classification As a result, the step of grid processing is carried out to the grid image to be removed classified using the full convolutional network trained Suddenly include:
Classification processing is carried out to the grid image to be removed using the sorter network, judges the grid image to be removed Network mode;
According to the network mode judged, the grid image to be removed is transported to corresponding full convolutional network model In, to obtain removing grid image after grid.
6. a kind of image based on deep learning goes mesh device, it is characterised in that including:
Module (10) is built, for extracting grid of different sizes in multiple original mesh images, is constructed and the grid The corresponding a variety of net templates of type;
Generation module (20), for constructing grid image, generation and the net template online using a variety of net templates Corresponding multi-class grid data;
Template (30) is trained, for sorter network to be respectively trained using the multi-class grid data of generation as training data With full convolutional network;
Grid processing module (40), for being carried out using the sorter network trained to grid image to be removed at classification Reason, grid is carried out to the grid image to be removed classified using the full convolutional network trained and handled.
7. the image according to claim 6 based on deep learning goes mesh device, it is characterised in that
The structure module (10) includes extraction unit (11) and construction unit (12),
Extraction unit (11), for extracting the different grids in multiple the described original mesh images collected, obtain extraction The pixel size of the grid;
Construction unit (12), for the pixel size of the grid according to acquisition, the grid is classified, by the net Lattice are divided into polytype, and build the net template corresponding with the type of the grid.
8. the image according to claim 6 based on deep learning goes mesh device, it is characterised in that
The generation module (20) includes structural unit (21) and integral unit (22),
Structural unit (21), for a variety of net templates to be superimposed in non-screening table images, net corresponding to online construction Table images;
Integral unit (22), for integrating the non-screening table images and the grid image to form training image mapping pair, And the training image of formation is mapped to as multi-class grid data.
9. the image according to claim 6 based on deep learning goes mesh device, it is characterised in that
The sorter network includes two sorter networks, and the multi-class grid data includes refined net data and coarse grid data, The training template (30) includes the first training unit (31) and the second training unit (32),
First training unit (31), for using the refined net data and the coarse grid data as training data to train Two sorter networks are stated, using two sorter network trained as two sorter network models;
Second training unit (32), for using the refined net data and the coarse grid data as training data to train Full convolutional network is stated, the full convolutional network full convolutional network model as corresponding to that will be trained.
10. the image according to claim 9 based on deep learning goes mesh device, it is characterised in that
The grid processing module (40) includes judging unit (41) and supply unit (42),
Judging unit (41), for carrying out classification processing to the grid image to be removed using the sorter network, judge institute State the network mode of grid image to be removed;
Supply unit (42), for according to the network mode judged, the grid image to be removed to be transported to correspondingly Full convolutional network model in, to obtain removing grid image after grid.
CN201710575071.8A 2017-07-14 2017-07-14 Image based on deep learning removes grid method and device Active CN107424131B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710575071.8A CN107424131B (en) 2017-07-14 2017-07-14 Image based on deep learning removes grid method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710575071.8A CN107424131B (en) 2017-07-14 2017-07-14 Image based on deep learning removes grid method and device

Publications (2)

Publication Number Publication Date
CN107424131A true CN107424131A (en) 2017-12-01
CN107424131B CN107424131B (en) 2019-11-12

Family

ID=60426594

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710575071.8A Active CN107424131B (en) 2017-07-14 2017-07-14 Image based on deep learning removes grid method and device

Country Status (1)

Country Link
CN (1) CN107424131B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108230269A (en) * 2017-12-28 2018-06-29 北京智慧眼科技股份有限公司 Grid method, device, equipment and storage medium are gone based on depth residual error network

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080025592A1 (en) * 2006-06-27 2008-01-31 Siemens Medical Solutions Usa, Inc. System and Method for Detection of Breast Masses and Calcifications Using the Tomosynthesis Projection and Reconstructed Images
CN101150658A (en) * 2007-08-22 2008-03-26 北京中星微电子有限公司 A bad point self-adapted grid noise elimination device and method
CN101272503A (en) * 2008-04-30 2008-09-24 北京中星微电子有限公司 Gridding noise elimination method and device for remaining image definition
CN101352047A (en) * 2005-12-29 2009-01-21 安泰科技有限公司 Method and device for removing grid noise
CN105125206A (en) * 2015-09-15 2015-12-09 中山大学 Intelligent electrocardio monitoring method and device
CN106709866A (en) * 2016-11-18 2017-05-24 北京智慧眼科技股份有限公司 Method and device for removing mesh watermark from identification photo, face verification method and device
CN106875360A (en) * 2017-02-17 2017-06-20 广州智能装备研究院有限公司 It is a kind of to eliminate the fuzzy method and device of image motion

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101352047A (en) * 2005-12-29 2009-01-21 安泰科技有限公司 Method and device for removing grid noise
US20080025592A1 (en) * 2006-06-27 2008-01-31 Siemens Medical Solutions Usa, Inc. System and Method for Detection of Breast Masses and Calcifications Using the Tomosynthesis Projection and Reconstructed Images
CN101150658A (en) * 2007-08-22 2008-03-26 北京中星微电子有限公司 A bad point self-adapted grid noise elimination device and method
CN101272503A (en) * 2008-04-30 2008-09-24 北京中星微电子有限公司 Gridding noise elimination method and device for remaining image definition
CN105125206A (en) * 2015-09-15 2015-12-09 中山大学 Intelligent electrocardio monitoring method and device
CN106709866A (en) * 2016-11-18 2017-05-24 北京智慧眼科技股份有限公司 Method and device for removing mesh watermark from identification photo, face verification method and device
CN106875360A (en) * 2017-02-17 2017-06-20 广州智能装备研究院有限公司 It is a kind of to eliminate the fuzzy method and device of image motion

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108230269A (en) * 2017-12-28 2018-06-29 北京智慧眼科技股份有限公司 Grid method, device, equipment and storage medium are gone based on depth residual error network
CN108230269B (en) * 2017-12-28 2021-02-09 智慧眼科技股份有限公司 Grid removing method, device and equipment based on depth residual error network and storage medium

Also Published As

Publication number Publication date
CN107424131B (en) 2019-11-12

Similar Documents

Publication Publication Date Title
CN107944442A (en) Based on the object test equipment and method for improving convolutional neural networks
CN104834933B (en) A kind of detection method and device in saliency region
CN104966085B (en) A kind of remote sensing images region of interest area detecting method based on the fusion of more notable features
CN108399362A (en) A kind of rapid pedestrian detection method and device
CN107506740A (en) A kind of Human bodys' response method based on Three dimensional convolution neutral net and transfer learning model
CN107103277B (en) Gait recognition method based on depth camera and 3D convolutional neural network
CN107784654A (en) Image partition method, device and full convolutional network system
CN106934408A (en) Identity card picture sorting technique based on convolutional neural networks
CN106228177A (en) Daily life subject image recognition methods based on convolutional neural networks
CN106682628B (en) Face attribute classification method based on multilayer depth feature information
CN107230205A (en) A kind of transmission line of electricity bolt detection method based on convolutional neural networks
CN103235947B (en) A kind of Handwritten Numeral Recognition Method and device
CN107766890A (en) The improved method that identification segment learns in a kind of fine granularity identification
CN106529484A (en) Combined spectrum and laser radar data classification method based on class-fixed multinucleated learning
CN107180241A (en) A kind of animal classification method of the profound neutral net based on Gabor characteristic with fractal structure
CN108710893A (en) A kind of digital image cameras source model sorting technique of feature based fusion
CN113298024A (en) Unmanned aerial vehicle ground small target identification method based on lightweight neural network
CN109685772B (en) No-reference stereo image quality evaluation method based on registration distortion representation
CN105825228B (en) Image-recognizing method and device
CN103984954B (en) Image combining method based on multi-feature fusion
CN104881635A (en) Image texture extraction and identification method by non-Gauss two-dimension Gabor filter
CN107886065A (en) A kind of Serial No. recognition methods of mixing script
Majumder et al. A tale of a deep learning approach to image forgery detection
CN105718934A (en) Method for pest image feature learning and identification based on low-rank sparse coding technology
CN109558880B (en) Contour detection method based on visual integral and local feature fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 100097 Beijing Haidian District Kunming Hunan Road 51 C block two floor 207.

Applicant after: BEIJING ATHENA EYES SCIENCE & TECHNOLOGY CO.,LTD.

Address before: 100193 4, 403, block A, 14 building, 10 East North Road, Haidian District, Beijing.

Applicant before: BEIJING ATHENA EYES SCIENCE & TECHNOLOGY CO.,LTD.

CB02 Change of applicant information
CB02 Change of applicant information

Address after: 410205 14 Changsha Zhongdian Software Park Phase I, 39 Jianshan Road, Changsha High-tech Development Zone, Yuelu District, Changsha City, Hunan Province

Applicant after: Wisdom Eye Technology Co.,Ltd.

Address before: 100097 2nd Floor 207, Block C, 51 Hunan Road, Kunming, Haidian District, Beijing

Applicant before: BEIJING ATHENA EYES SCIENCE & TECHNOLOGY CO.,LTD.

GR01 Patent grant
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: Image de grid method and device based on depth learning

Effective date of registration: 20221205

Granted publication date: 20191112

Pledgee: Agricultural Bank of China Limited Hunan Xiangjiang New Area Branch

Pledgor: Wisdom Eye Technology Co.,Ltd.

Registration number: Y2022430000107

PC01 Cancellation of the registration of the contract for pledge of patent right
PC01 Cancellation of the registration of the contract for pledge of patent right

Date of cancellation: 20231220

Granted publication date: 20191112

Pledgee: Agricultural Bank of China Limited Hunan Xiangjiang New Area Branch

Pledgor: Wisdom Eye Technology Co.,Ltd.

Registration number: Y2022430000107

CP03 Change of name, title or address
CP03 Change of name, title or address

Address after: No. 205, Building B1, Huigu Science and Technology Industrial Park, No. 336 Bachelor Road, Bachelor Street, Yuelu District, Changsha City, Hunan Province, 410000

Patentee after: Wisdom Eye Technology Co.,Ltd.

Address before: 410205 building 14, phase I, Changsha Zhongdian Software Park, No. 39, Jianshan Road, Changsha high tech Development Zone, Yuelu District, Changsha City, Hunan Province

Patentee before: Wisdom Eye Technology Co.,Ltd.