CN109101975A - Image, semantic dividing method based on full convolutional neural networks - Google Patents
Image, semantic dividing method based on full convolutional neural networks Download PDFInfo
- Publication number
- CN109101975A CN109101975A CN201810947884.XA CN201810947884A CN109101975A CN 109101975 A CN109101975 A CN 109101975A CN 201810947884 A CN201810947884 A CN 201810947884A CN 109101975 A CN109101975 A CN 109101975A
- Authority
- CN
- China
- Prior art keywords
- image
- semantic
- layer
- semantic segmentation
- details
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 17
- 230000011218 segmentation Effects 0.000 claims abstract description 49
- 238000012549 training Methods 0.000 claims abstract description 16
- 238000012937 correction Methods 0.000 claims abstract description 15
- 238000005070 sampling Methods 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims description 18
- 238000010606 normalization Methods 0.000 claims description 6
- 230000000644 propagated effect Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims 1
- 238000003709 image segmentation Methods 0.000 abstract description 3
- 238000013135 deep learning Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 10
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The present invention discloses a kind of image, semantic dividing method based on full convolutional neural networks, is related to image, semantic segmentation and deep learning field, includes the following steps: to select training dataset；It constructs and trains by the disaggregated model of image to class label, and as semantic segmentation model front network；The characteristic pattern of each piece of front network output passes through details respectively, and to retain pond layer down-sampled at unified size, then this four output characteristic patterns is connected, and by the heavy correction module of feature, after re-calibrating characteristic pattern, by the incoming back-end network of obtained characteristic pattern；Back-end network is to be mainly responsible for picture up-sampling, by up-sampling and then carrying out error back propagation finally with the semantic tagger image of training dataset calculating cross entropy by the global pool of a variable weight.The present invention solves the problems, such as that image segmentation accuracy rate in the prior art is lower.
Description
Technical field
The present invention relates to image, semantic segmentation and deep learning fields, more particularly to the image based on full convolutional neural networks
Semantic segmentation method.
Background technique
Semantic segmentation is an important problem in computer vision field.Image, semantic segmentation is to give each pixel
A different label (classification) is assigned, therefore is considered an intensive classification problem.
In recent years, most current optimal image, semantic dividing methods are all based on full convolutional neural networks.Allusion quotation
The semantic segmentation network structure of type is coder-decoder structure, and encoder is an image drop sampling process, is responsible for extracting figure
As coarse semantic feature, it is followed by a decoder, decoder is a picture up-sampling process, is responsible for down-sampled
Obtained characteristics of image carries out up-sampling and is restored to the original dimension of input picture.
Although pond is in the process a crucial component part in the down-sampled of convolutional neural networks, can be used to reduce
The scale of parameter enhances the invariance to certain distortions, while increasing receptive field.But because of pond, itself is a damage
The process of consumption makes semantic point so it will lead to the loss of image, semantic information during the image drop sampling of semantic segmentation
The precision for cutting result is relatively low.
In depth convolutional neural networks, pond layer is replaced commonly using the convolution that strides (strided convolutions)
Have the function that down-sampled, the convolution that strides only considers a node of the fixation position of each local neighborhood, activates without considering
Importance.From the angle of image drop sampling, such down-sampled mode equally also results in the distortion of feature.Full convolutional Neural
Network is the state-of-the-art image, semantic partitioning algorithm of a large amount of application programming, and wherein the innovation of network structure is mainly concentrated
It encodes or is connected to the network to promote gradient current in room for improvement.
Summary of the invention
It is existing to solve it is an object of the invention to design a kind of image, semantic dividing method based on full convolutional neural networks
The problem for having the image segmentation accuracy rate in technology lower.
Technical scheme is as follows:
Image, semantic dividing method based on full convolutional neural networks, includes the following steps:
Step 1: selection training dataset.
Step 2: constructing and train the disaggregated model by image to class label, and as semantic segmentation model front end
Network；
The structure of semantic segmentation model front network includes Conv1, Conv2_x, Conv3_x and Conv4_x, Conv1,
Conv2_x, Conv3_x and Conv4_x include multiple convolutional layers, Conv4_x, Conv1, Conv2_x, Conv3_x and
It is all connected with a details behind Conv4_x and retains pond layer one.
Step 3: based on trained semantic segmentation model front network, constructing semantic segmentation model back-end network.
The structure of back-end network include details retain pond layer two, feature weigh correction module, convolutional layer, Conv5_x,
Conv6_x, Conv7_x, convolutional layer, variable weight global pool layer and up-sampling layer；The output of Conv1, Conv2_x, Conv3_x point
Tong Guo not be after three details retain and are connected in series after pond layers two with Conv4_x, common input feature vector weighs correction module；Conv5_
X, be all connected with a up-sampling layer before Conv6_x and Conv7_x, Conv5_x, Conv6_x and Conv7_x include convolutional layer,
Batch normalization layer and line rectification unit, Conv5_x, Conv6_x, Conv7_x by jump structure respectively successively with Conv3_
X, the output characteristic pattern series connection of Conv2_x and Conv1.
Step 4: whole image semantic segmentation model is trained.
Step 5: inputting new image, a propagated forward is carried out in trained deep neural network model, hold
The semantic segmentation result of prediction is exported to end.
It specifically, include 33 residual error structures, each residual error structure packet in the semantic segmentation front end model front network
Containing 11 × 1 convolution, 13 × 3 convolution, 11 × 1 convolution sum 1 quick connection.
Specifically, the details after the Conv1 retains down-sampled 8 times of output characteristic pattern of pond layer a pair of Conv1,
Details after Conv2_x retains down-sampled 4 times of output characteristic pattern of pond layer a pair of Conv2_x, and the details after Conv3_x retains
Down-sampled 2 times of the output characteristic pattern of pond layer a pair of Conv3_x.
Specifically, the details retains pond layer one and details retains the detailed process of pond layer two are as follows:
The output P of each position is calculated according to the characteristic pattern I of input:
Wherein,[P] indicates that input feature vector figure retains the value of pond Hua Cenghou output position P by details；
Neighborhood spaceThe space average weights omega of input nodeα,β[p, q] is
Wherein α is biased exponent, and β is reward index.ρβ(·)It is anti-bilateral filtering function, is used in neighborhood space ΩpMeter
The weight of input point is calculated, β reduces the dynamic range of reward function, and β → 0 is exactly that simple field is average.
It is linear-scale reduction factor, specifically:
Wherein F is in neighborhoodOn one can learn, the 2D filter of nonstandardized technique, thisSize be 3 ×
3。
Specifically, the feature weight correction module is to combine space characteristics to correct the network corrected again with channel characteristics again
Module.
Specifically, the process of training whole image semantic segmentation model are as follows:
Step 4.1: the image concentrated to training data pre-processes, and is fixed dimension by image cutting-out.
Step 4.2: whole image semantic segmentation model is initialized.
Step 4.3: the data concentrated to training data are expanded by way of overturning, scaling and rotation.
Step 4.4: using the sum of the intersection entropy loss of each pixel as loss function, reusing random tonsure descent algorithm
Error back propagation is carried out, model parameter is updated, obtains trained semantic segmentation model.
After adopting the above scheme, beneficial effects of the present invention are as follows:
(1) image, semantic parted pattern of the invention introduces details reservation pond layer, during down-sampled, Neng Goubao
Stay more image detail informations.It is a kind of adaptive pond method that details, which retains pond layer, and this method can amplify sky
Between change and retain important CONSTRUCTED SPECIFICATION, it is also important that, its parameter can learn jointly with the rest part of network.
(2) introduced feature weight correction module in the present invention, is carrying out re-graduation just to feature, space characteristics re-graduation just can be more preferable
The importance by same position pixels all in space re-calibrated, and be assigned to corresponding weight, improve semantic segmentation
Accuracy rate, important channel can just be assigned to high weight by channel characteristics re-graduation, prominent importance；In short, feature re-graduation is just
Module can efficiently solve image, semantic segmentation, and accuracy rate is low, during pond the problem of detailed information loss, finally obtains
Preferable semantic segmentation result.
(3) the variable weight global pool described in, since traditional global draw pondization operates, to the same of all feature channels
Position is carried out same operation, i.e. 1 × 1 convolution, cannot protrude the correct class categories of each pixel in semantic segmentation, give
1 × 1 convolution in the average pond of the overall situation adds 1 weight vector, is distributed by standard gaussian and carries out parameter initialization, in error
In back-propagation process, the weight of pixel is constantly updated, can preferably be classified pixel-by-pixel, moreover it is possible to it is convergent to play quickening
Effect.
Detailed description of the invention
Fig. 1 is flow chart of the invention；
Fig. 2 is image, semantic parted pattern structure chart of the invention；
Fig. 3 is residual error structure chart of the invention；
Fig. 4 is the positive function structure chart of feature re-graduation of the invention；
Fig. 5 is the positive function structure chart of channel characteristics re-graduation of the invention；
Fig. 6 is the positive function structure chart of space characteristics re-graduation of the invention.
Specific embodiment
Understand to make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with specific implementation with and it is attached
Figure, invention is further described in detail.
It is proposed by the present invention a kind of based on full convolution to solve the problems, such as that image segmentation accuracy rate in the prior art is lower
The image, semantic dividing method of neural network can be widely applied for the field of General Two-Dimensional image, semantic segmentation.
As shown in Figure 1, the image, semantic dividing method based on full convolutional neural networks, the present invention include the following steps:
Step 1: selection training dataset；Using 21 classes of 2012 data set of VOC in the present embodiment (wherein 1 class is background)
On the basis of scene type, acquires the image in COCO data set comprising above-mentioned 20 class classification target and data set is added, finally obtain
Trained and test data set.
Step 2: constructing and train the disaggregated model by image to class label, and as semantic segmentation model front end
Network.
As shown in Fig. 2, the structure of semantic segmentation model front network includes Conv1, Conv2_x, Conv3_x and Conv4_
X, Conv1, Conv2_x, Conv3_x and Conv4_x include multiple convolutional layers, Conv4_x, Conv1, Conv2_x, Conv3_
It is all connected with a details behind x and Conv4_x and retains pond layer one；After each piece of Conv2_x, Conv3_x and Conv4_x
Face adds a details to retain pond layer, this is a kind of adaptive pool method, can amplify spatial variations and retain important knot
Structure details.
As shown in figure 3, including 33 residual error structures in the model front network of semantic segmentation front end, each residual error structure includes
11 × 1 convolution, 13 × 3 convolution, 11 × 1 convolution sum 1 quick connection (shortcut connection).
For the convenience of description, by the characteristic pattern (having a size of 112 × 112) of Conv1 output, the characteristic pattern of Conv2_x output
(having a size of 56 × 56), the characteristic pattern (having a size of 28 × 28) of Conv3_x output, the characteristic pattern of Conv4_x output is (having a size of 14
× 14) characteristic pattern Res_1, characteristic pattern Res_2, characteristic pattern Res_3 and characteristic pattern Res_4 are denoted as.
Details after Conv1 retains down-sampled 8 times of pond layer a pair of characteristic pattern Res_1, and the details after Conv2_x retains pond
Change down-sampled 4 times of layer a pair of characteristic pattern Res_2, the details after Conv3_x retains pond layer a pair of characteristic pattern Res_3 down-sampled 2
Times.
Step 3: based on trained semantic segmentation model front network, constructing semantic segmentation model back-end network.
As shown in Fig. 2, the structure of back-end network include details retain pond layer two, feature weigh correction module, convolutional layer,
Conv5_x, Conv6_x, Conv7_x, convolutional layer, variable weight global pool layer and up-sampling layer；Characteristic pattern Res_1, characteristic pattern
Res_2, characteristic pattern Res_3 pass through respectively after three details retain and be connected in series after pond layers two with Conv4_x, common input spy
Levy weight correction module；Be all connected with a up-sampling layer before Conv5_x, Conv6_x and Conv7_x, Conv5_x, Conv6_x and
Conv7_x includes that convolutional layer, batch normalization layer and line rectification unit, Conv5_x, Conv6_x, Conv7_x pass through jump
Structure is successively connected with the output characteristic pattern of Conv3_x, Conv2_x and Conv1 respectively.
For in step 2 and step 3, the details retains pond layer one and details retains the detailed process of pond layer two
Are as follows:
The output P of each position is calculated according to the characteristic pattern I of input:
Wherein,[P] indicates that input feature vector figure retains the value of pond Hua Cenghou output position P by details；Neighborhood is empty
BetweenThe space weighted average ω of input nodeα,β[p, q] is
Wherein α is biased exponent, and β is reward index.ρβ() is anti-bilateral filtering function, is used in neighborhood space ΩpMeter
The weight of input point is calculated, β reduces the dynamic range of reward function, and β → 0 is exactly that simple field is average.
It is linear-scale reduction factor, specifically:
Wherein F is in neighborhoodOn one can learn, the 2D filter of nonstandardized technique, thisSize be 3 ×
3。
Specifically, feature weight correction module it is (as shown in Figure 4) be in conjunction with space characteristics re-graduation just with channel characteristics re-graduation just
Network module.
It will separately be illustrated below:
As shown in figure 5, process in space characteristics weight correction module are as follows:
(1) by primitive character figureIt is 1 × 1 by a convolution kernel size, port number is c (each logical
The weight in road is not shared, it is allowed to obtain from study) convolution, obtain a characteristic pattern
(2) one sigmoid layers are passed it through again, by McEach spatial position M ' (i, j), i ∈ { 1,2 ..., H }, j
The importance of ∈ { 1,2 ..., W } re-calibrates, and is assigned to the weight p (i, j) of each spatial position one, obtained p (i, j) with
Primitive character figure McCarry out dot product.
Finally, McThe characteristic pattern just obtained by space characteristics re-graduation are as follows:
Space characteristics re-graduation just can preferably be re-calibrated the importance of same position pixels all in space,
And it is assigned to corresponding weight, improve the accuracy rate of semantic segmentation.
As shown in fig. 6, process in channel characteristics weight correction module are as follows:
(1) by primitive character figureIt is averaged pond by an overall situation, obtains a characteristic patternAgain by M ' and primitive character figure McIt is connected entirely, carries out the integration of characteristic pattern.
(2) characteristic pattern after integrating is modified feature using a linear amending unit.
It (3) is finally H × W using a convolution kernel size to revised characteristic pattern, port number is that the convolution of c obtains
One feature vector
(4) the activation range of feature vector z is limited between [0,1], obtains using one sigmoid layers by characteristic pattern
To a channel weight vectorMcThe characteristic pattern just obtained by channel characteristics re-graduation:
Just by channel characteristics re-graduation, important channel can be assigned to high weight, prominent importance.
Step 4: whole image semantic segmentation model is trained；The process of training whole image semantic segmentation model
For.
Step 4.1: the image concentrated to training data pre-processes, and is fixed dimension 513 × 513 by image cutting-out.
Step 4.2: whole image semantic segmentation model being initialized, i.e., mould is divided with the good image, semantic of pre-training
The parameter value of type is initial value.
Step 4.3: the data concentrated to training data are expanded by way of overturning, scaling and rotation；Specifically,
Overturning is random overturning；In the random zoomed image between 0.5 to 2 times of original image；In original image between -10 to 10 degree,
Random-Rotation image.
Step 4.4: using the sum of the intersection entropy loss of each pixel as loss function, reusing random tonsure descent algorithm
Error back propagation is carried out, with multinomial learning strategy, model parameter is updated, obtains trained semantic segmentation model.It is multinomial
In formula learning strategy, learning rate lr setting are as follows:
Wherein, baselr is initial learning rate, is set as 0.001, power settingization 0.9 here.
Step 5: inputting new image, a propagated forward is carried out in trained deep neural network model, hold
The semantic segmentation result of prediction is exported to end.
The principle of the present invention and process are as follows: using the spy of Conv1 output in image, semantic parted pattern of the invention
The feature of characteristic pattern Res_3, the Conv4_x output of characteristic pattern Res_2, the Conv3_x output of sign figure Res_1, Conv2_x output
Scheme Res_4, the respectively first layer, the second layer of front network (i.e. feature extraction network), third layer and the 4th layer.It then will be special
Sign figure Res_1, which retains pond layer one by details, to carry out retaining down-sampled 8 times of details convolution, and Res_2 passes through to be retained by details
Pond layer one carries out retaining down-sampled 4 times of details pondization, and Res_3 passes through to carry out retaining details volume by details reservation pond layer one
Down-sampled 2 times and Res_1 of product is together in series, and is input to feature weight correction module, just by channel characteristics re-graduation, space characteristics
Re-graduation just can preferably be re-calibrated the importance of same position pixels all in space, and be assigned to weigh accordingly
Value, improves the accuracy rate of semantic segmentation, and important channel can be just assigned to high weight, prominent importance by channel characteristics re-graduation.
Then the characteristic pattern of feature weight correction module output is passed through to 11 × 1 convolutional layer, obtained characteristic pattern De_1, by characteristic pattern
De_1 is up-sampled to 28 × 28, and obtained characteristic pattern passes through 23 × 3 convolution, batch normalization layer and line rectification units, most
It connects afterwards with characteristic pattern Res_3, obtained characteristic pattern De_2；De_1De_2 is up-sampled to 56 × 56, using 23 × 3
Convolution, batch normalization layer and line rectification unit, connect with characteristic pattern Res_2, obtained characteristic pattern De_3, by characteristic pattern De_3
Up-sampling is to 112 × 112, using 23 × 3 convolution, batch normalization layer and line rectification unit, obtained characteristic pattern De_4,
Characteristic pattern De_4 is finally passed through into 1 variable weight global pool, is finally up-sampled to original image size, and marks and calculates with semantic segmentation
Cross entropy is propagated using direction of error, obtains the network model of semantic segmentation.Variable weight global pool, since traditional overall situation is flat
Office's pondization operation, is carried out same operation, i.e. 1 × 1 convolution to the same position in all feature channels, cannot protrude semantic segmentation
In the correct class categories of each pixel pass through mark to 1 × 1 convolution in global average pond plus 1 weight vector
Quasi- Gaussian Profile carries out parameter initialization, in training process, according to backpropagation, is assigned to Gao Quan to the pixel for belonging to target category
Value, can preferably be classified pixel-by-pixel, moreover it is possible to play the role of accelerating convergent.The present invention is in VOC2012 semantic segmentation number
According to achieved on collection mIoU be 76.33% result.
All technology deformations made according to the technique and scheme of the present invention, fall within the scope of protection of the present invention.
Claims (6)
1. the image, semantic dividing method based on full convolutional neural networks, which comprises the steps of:
Step 1: selection training dataset；
Step 2: constructing and train the semantic segmentation model front network by image to class label；
The structure of semantic segmentation model front network includes Conv1, Conv2_x, Conv3_x and Conv4_x, Conv1, Conv2_
X, Conv3_x and Conv4_x includes multiple convolutional layers, Conv4_x, after Conv1, Conv2_x, Conv3_x and Conv4_x
Face is all connected with a details and retains pond layer；
Step 3: based on trained semantic segmentation model front network, constructing semantic segmentation model back-end network；
The structure of back-end network include details retain pond layer two, feature weigh correction module, convolutional layer, Conv5_x, Conv6_x,
Conv7_x, convolutional layer, variable weight global pool layer and up-sampling layer；The output of Conv1, Conv2_x, Conv3_x pass through three respectively
After being connected in series after a details reservation pond layer two with Conv4_x, common input feature vector weight correction module；Conv5_x,Conv6_
A up-sampling layer is all connected with before x and Conv7_x, Conv5_x, Conv6_x and Conv7_x include convolutional layer, batch normalization
Layer and line rectification unit, Conv5_x, Conv6_x, Conv7_x by jump structure respectively successively with Conv3_x, Conv2_x
It connects with the output characteristic pattern of Conv1；
Step 4: whole image semantic segmentation model is trained；
Step 5: new image is inputted, a propagated forward is carried out in trained deep neural network model, it is end-to-end
The semantic segmentation result of ground output prediction.
2. the image, semantic dividing method according to claim 1 based on full convolutional neural networks, which is characterized in that described
It include 33 residual error structures in semantic segmentation front network, each residual error structure includes 11 × 1 convolution, 13 × 3 volume
Product, 11 × 1 convolution sum 1 quick connection.
3. the image, semantic dividing method according to claim 1 based on full convolutional neural networks, which is characterized in that described
Details after Conv1 retains down-sampled 8 times of output characteristic pattern of pond layer a pair of Conv1, and the details after Conv2_x retains pond
Down-sampled 4 times of output characteristic pattern of layer a pair of Conv2_x, the details after Conv3_x retain the output of pond layer a pair of Conv3_x
Down-sampled 2 times of characteristic pattern.
4. the image, semantic dividing method according to claim 1 based on full convolutional neural networks, which is characterized in that
The details retains pond layer one and details retains the detailed process of pond layer two are as follows:
The output P of each position is calculated according to the characteristic pattern I of input:
Wherein,Indicate that input feature vector figure retains the value of pond Hua Cenghou output position P by details；
Neighborhood spaceThe space weighted average ω of input nodeα,β[p, q] is
Wherein α is biased exponent, and β is reward index；ρβ() is anti-bilateral filtering function, is used in neighborhood space ΩpIt calculates defeated
The weight of access point, β reduce the dynamic range of reward function, and β → 0 is exactly simple neighborhood averaging.
It is linear-scale reduction factor, specifically:
Wherein F is in neighborhoodOn one can learn, the 2D filter of nonstandardized technique, thisSize be 3 × 3.
5. the image, semantic dividing method according to claim 1 based on full convolutional neural networks, which is characterized in that described
Feature weight correction module is to combine the space characteristics re-graduation just positive network module with channel characteristics re-graduation.
6. the image, semantic dividing method according to claim 1 based on full convolutional neural networks, which is characterized in that training
The process of whole image semantic segmentation model are as follows:
Step 4.1: the image concentrated to training data pre-processes, and is fixed dimension by image cutting-out；
Step 4.2: whole image semantic segmentation model is initialized；
Step 4.3: the data concentrated to training data are expanded by way of overturning, scaling and rotation；
Step 4.4: using the sum of the intersection entropy loss of each pixel as loss function, reusing random tonsure descent algorithm and carry out
Error back propagation updates model parameter, obtains trained semantic segmentation model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810947884.XA CN109101975B (en) | 2018-08-20 | 2018-08-20 | Image semantic segmentation method based on full convolution neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810947884.XA CN109101975B (en) | 2018-08-20 | 2018-08-20 | Image semantic segmentation method based on full convolution neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109101975A true CN109101975A (en) | 2018-12-28 |
| CN109101975B CN109101975B (en) | 2022-01-25 |
Family
ID=64850450
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810947884.XA Expired - Fee Related CN109101975B (en) | 2018-08-20 | 2018-08-20 | Image semantic segmentation method based on full convolution neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109101975B (en) |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886273A (en) * | 2019-02-26 | 2019-06-14 | 四川大学华西医院 | A kind of CMR classification of image segmentation system |
| CN109886082A (en) * | 2019-01-03 | 2019-06-14 | 南京理工大学 | A kind of Small object enhancing prediction module detection method based on SSD |
| CN109919080A (en) * | 2019-03-05 | 2019-06-21 | 南京航空航天大学 | More full convolutional neural networks of decoder and its corresponding microscopical structure recognition methods |
| CN110084319A (en) * | 2019-05-07 | 2019-08-02 | 上海宝尊电子商务有限公司 | Fashion images clothes collar recognition methods and system based on deep neural network |
| CN110147763A (en) * | 2019-05-20 | 2019-08-20 | 哈尔滨工业大学 | Video semanteme dividing method based on convolutional neural networks |
| CN110189334A (en) * | 2019-05-28 | 2019-08-30 | 南京邮电大学 | The medical image cutting method of the full convolutional neural networks of residual error type based on attention mechanism |
| CN110225342A (en) * | 2019-04-10 | 2019-09-10 | 中国科学技术大学 | The bit distribution system and method for Video coding based on semantic distortion metrics |
| CN110232693A (en) * | 2019-06-12 | 2019-09-13 | 桂林电子科技大学 | A kind of combination thermodynamic chart channel and the image partition method for improving U-Net |
| CN110263706A (en) * | 2019-06-19 | 2019-09-20 | 南京邮电大学 | A kind of haze weather Vehicular video Detection dynamic target and know method for distinguishing |
| CN110276267A (en) * | 2019-05-28 | 2019-09-24 | 江苏金海星导航科技有限公司 | Method for detecting lane lines based on Spatial-LargeFOV deep learning network |
| CN110363134A (en) * | 2019-07-10 | 2019-10-22 | 电子科技大学 | A kind of face blocked area localization method based on semantic segmentation |
| CN110428009A (en) * | 2019-08-02 | 2019-11-08 | 南京航空航天大学 | A kind of full convolutional neural networks and corresponding microscopical structure recognition methods |
| CN110517272A (en) * | 2019-08-29 | 2019-11-29 | 电子科技大学 | Blood cell segmentation method based on deep learning |
| CN110689020A (en) * | 2019-10-10 | 2020-01-14 | 湖南师范大学 | Segmentation method of mineral flotation froth image and electronic equipment |
| CN110782023A (en) * | 2019-11-04 | 2020-02-11 | 华南理工大学 | Reduction residual module porous convolution architecture network and rapid semantic segmentation method |
| CN110930409A (en) * | 2019-10-18 | 2020-03-27 | 电子科技大学 | Salt body semantic segmentation method based on deep learning and semantic segmentation model |
| CN111091550A (en) * | 2019-12-12 | 2020-05-01 | 创新奇智(北京)科技有限公司 | Multi-size self-adaptive PCB solder paste area detection system and detection method |
| CN111192248A (en) * | 2019-12-30 | 2020-05-22 | 山东大学 | Multi-task relation learning method for positioning, identifying and segmenting vertebral body in nuclear magnetic resonance imaging |
| CN111209972A (en) * | 2020-01-09 | 2020-05-29 | 中国科学院计算技术研究所 | Image classification method and system based on hybrid connectivity deep convolution neural network |
| CN111259906A (en) * | 2020-01-17 | 2020-06-09 | 陕西师范大学 | Method for generating and resisting remote sensing image target segmentation under condition containing multilevel channel attention |
| CN111275712A (en) * | 2020-01-15 | 2020-06-12 | 浙江工业大学 | Residual semantic network training method oriented to large-scale image data |
| CN111489364A (en) * | 2020-04-08 | 2020-08-04 | 重庆邮电大学 | Medical image segmentation method based on lightweight full convolution neural network |
| CN111612740A (en) * | 2020-04-16 | 2020-09-01 | 深圳大学 | Pathological image processing method and device |
| CN111862190A (en) * | 2020-07-10 | 2020-10-30 | 北京农业生物技术研究中心 | Method and device for automatically measuring area of isolated plant soft rot disease spot |
| CN111860386A (en) * | 2020-07-27 | 2020-10-30 | 山东大学 | Video semantic segmentation method based on ConvLSTM convolutional neural network |
| WO2020238734A1 (en) * | 2019-05-27 | 2020-12-03 | 腾讯科技（深圳）有限公司 | Image segmentation model training method and apparatus, computer device, and storage medium |
| CN112364878A (en) * | 2020-09-25 | 2021-02-12 | 江苏师范大学 | Power line classification method based on deep learning under complex background |
| CN112766279A (en) * | 2020-12-31 | 2021-05-07 | 中国船舶重工集团公司第七0九研究所 | Image feature extraction method based on combined attention mechanism |
| CN113487524A (en) * | 2021-04-07 | 2021-10-08 | 北京百度网讯科技有限公司 | Image format conversion method, device, equipment, storage medium and program product |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107480726A (en) * | 2017-08-25 | 2017-12-15 | 电子科技大学 | A kind of Scene Semantics dividing method based on full convolution and shot and long term mnemon |
| CN107784654A (en) * | 2016-08-26 | 2018-03-09 | 杭州海康威视数字技术股份有限公司 | Image partition method, device and full convolutional network system |
| US20180137338A1 (en) * | 2016-11-16 | 2018-05-17 | The Governing Council Of The University Of Toronto | System and method for classifying and segmenting microscopy images with deep multiple instance learning |
| CN108062756A (en) * | 2018-01-29 | 2018-05-22 | 重庆理工大学 | Image, semantic dividing method based on the full convolutional network of depth and condition random field |
-
2018
- 2018-08-20 CN CN201810947884.XA patent/CN109101975B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107784654A (en) * | 2016-08-26 | 2018-03-09 | 杭州海康威视数字技术股份有限公司 | Image partition method, device and full convolutional network system |
| US20180137338A1 (en) * | 2016-11-16 | 2018-05-17 | The Governing Council Of The University Of Toronto | System and method for classifying and segmenting microscopy images with deep multiple instance learning |
| CN107480726A (en) * | 2017-08-25 | 2017-12-15 | 电子科技大学 | A kind of Scene Semantics dividing method based on full convolution and shot and long term mnemon |
| CN108062756A (en) * | 2018-01-29 | 2018-05-22 | 重庆理工大学 | Image, semantic dividing method based on the full convolutional network of depth and condition random field |
Non-Patent Citations (4)
| Title |
|---|
| ABHIJIT GUHA ROY等: "Concurrent Spatial and Channel ‘Squeeze & Excitation" in Fully Convolutional Networks", 《ARXIV:1803.02579V2 [CS.CV]》 * |
| FARAZ SAEEDAN等: "Detail-Preserving Pooling in Deep Networks", 《ARXIV:1804.04076V1 [》 * |
| OZAN OKTAY等: "Attention U-Net:Learning Where to Look for the Pancreas", 《ARXIV:1804.03999V1 [CS.CV]》 * |
| 辛鹏等: "全卷积网络多层特征融合的飞机快速检测", 《光学学报》 * |
Cited By (45)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886082A (en) * | 2019-01-03 | 2019-06-14 | 南京理工大学 | A kind of Small object enhancing prediction module detection method based on SSD |
| CN109886273A (en) * | 2019-02-26 | 2019-06-14 | 四川大学华西医院 | A kind of CMR classification of image segmentation system |
| CN109919080A (en) * | 2019-03-05 | 2019-06-21 | 南京航空航天大学 | More full convolutional neural networks of decoder and its corresponding microscopical structure recognition methods |
| CN109919080B (en) * | 2019-03-05 | 2019-10-11 | 南京航空航天大学 | More full convolutional neural networks of decoder and its corresponding microscopical structure recognition methods |
| CN110225342A (en) * | 2019-04-10 | 2019-09-10 | 中国科学技术大学 | The bit distribution system and method for Video coding based on semantic distortion metrics |
| CN110225342B (en) * | 2019-04-10 | 2021-03-09 | 中国科学技术大学 | Video coding bit distribution system and method based on semantic distortion measurement |
| CN110084319A (en) * | 2019-05-07 | 2019-08-02 | 上海宝尊电子商务有限公司 | Fashion images clothes collar recognition methods and system based on deep neural network |
| CN110084319B (en) * | 2019-05-07 | 2023-06-30 | 上海宝尊电子商务有限公司 | Fashion image clothing collar type recognition method and system based on deep neural network |
| CN110147763A (en) * | 2019-05-20 | 2019-08-20 | 哈尔滨工业大学 | Video semanteme dividing method based on convolutional neural networks |
| CN110147763B (en) * | 2019-05-20 | 2023-02-24 | 哈尔滨工业大学 | Video semantic segmentation method based on convolutional neural network |
| US11961233B2 (en) | 2019-05-27 | 2024-04-16 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for training image segmentation model, computer device, and storage medium |
| WO2020238734A1 (en) * | 2019-05-27 | 2020-12-03 | 腾讯科技（深圳）有限公司 | Image segmentation model training method and apparatus, computer device, and storage medium |
| CN110189334A (en) * | 2019-05-28 | 2019-08-30 | 南京邮电大学 | The medical image cutting method of the full convolutional neural networks of residual error type based on attention mechanism |
| CN110276267A (en) * | 2019-05-28 | 2019-09-24 | 江苏金海星导航科技有限公司 | Method for detecting lane lines based on Spatial-LargeFOV deep learning network |
| CN110189334B (en) * | 2019-05-28 | 2022-08-09 | 南京邮电大学 | Medical image segmentation method of residual error type full convolution neural network based on attention mechanism |
| CN110232693A (en) * | 2019-06-12 | 2019-09-13 | 桂林电子科技大学 | A kind of combination thermodynamic chart channel and the image partition method for improving U-Net |
| CN110263706B (en) * | 2019-06-19 | 2021-07-27 | 南京邮电大学 | Method for detecting and identifying dynamic target of vehicle-mounted video in haze weather |
| CN110263706A (en) * | 2019-06-19 | 2019-09-20 | 南京邮电大学 | A kind of haze weather Vehicular video Detection dynamic target and know method for distinguishing |
| CN110363134A (en) * | 2019-07-10 | 2019-10-22 | 电子科技大学 | A kind of face blocked area localization method based on semantic segmentation |
| CN110363134B (en) * | 2019-07-10 | 2021-06-08 | 电子科技大学 | Human face shielding area positioning method based on semantic segmentation |
| CN110428009B (en) * | 2019-08-02 | 2020-06-16 | 南京航空航天大学 | Full convolution neural network and corresponding mesoscopic structure identification method |
| CN110428009A (en) * | 2019-08-02 | 2019-11-08 | 南京航空航天大学 | A kind of full convolutional neural networks and corresponding microscopical structure recognition methods |
| CN110517272A (en) * | 2019-08-29 | 2019-11-29 | 电子科技大学 | Blood cell segmentation method based on deep learning |
| CN110517272B (en) * | 2019-08-29 | 2022-03-25 | 电子科技大学 | Deep learning-based blood cell segmentation method |
| CN110689020A (en) * | 2019-10-10 | 2020-01-14 | 湖南师范大学 | Segmentation method of mineral flotation froth image and electronic equipment |
| CN110930409A (en) * | 2019-10-18 | 2020-03-27 | 电子科技大学 | Salt body semantic segmentation method based on deep learning and semantic segmentation model |
| CN110782023B (en) * | 2019-11-04 | 2023-04-07 | 华南理工大学 | Reduction residual module porous convolution architecture network and rapid semantic segmentation method |
| CN110782023A (en) * | 2019-11-04 | 2020-02-11 | 华南理工大学 | Reduction residual module porous convolution architecture network and rapid semantic segmentation method |
| CN111091550A (en) * | 2019-12-12 | 2020-05-01 | 创新奇智(北京)科技有限公司 | Multi-size self-adaptive PCB solder paste area detection system and detection method |
| CN111192248B (en) * | 2019-12-30 | 2023-05-05 | 山东大学 | Multi-task relation learning method for positioning, identifying and segmenting vertebral body in nuclear magnetic resonance imaging |
| CN111192248A (en) * | 2019-12-30 | 2020-05-22 | 山东大学 | Multi-task relation learning method for positioning, identifying and segmenting vertebral body in nuclear magnetic resonance imaging |
| CN111209972A (en) * | 2020-01-09 | 2020-05-29 | 中国科学院计算技术研究所 | Image classification method and system based on hybrid connectivity deep convolution neural network |
| CN111275712A (en) * | 2020-01-15 | 2020-06-12 | 浙江工业大学 | Residual semantic network training method oriented to large-scale image data |
| CN111259906A (en) * | 2020-01-17 | 2020-06-09 | 陕西师范大学 | Method for generating and resisting remote sensing image target segmentation under condition containing multilevel channel attention |
| CN111489364B (en) * | 2020-04-08 | 2022-05-03 | 重庆邮电大学 | Medical image segmentation method based on lightweight full convolution neural network |
| CN111489364A (en) * | 2020-04-08 | 2020-08-04 | 重庆邮电大学 | Medical image segmentation method based on lightweight full convolution neural network |
| CN111612740A (en) * | 2020-04-16 | 2020-09-01 | 深圳大学 | Pathological image processing method and device |
| CN111612740B (en) * | 2020-04-16 | 2023-07-25 | 深圳大学 | Pathological image processing method and device |
| CN111862190B (en) * | 2020-07-10 | 2024-04-05 | 北京农业生物技术研究中心 | Method and device for automatically measuring area of soft rot disease spots of isolated plants |
| CN111862190A (en) * | 2020-07-10 | 2020-10-30 | 北京农业生物技术研究中心 | Method and device for automatically measuring area of isolated plant soft rot disease spot |
| CN111860386B (en) * | 2020-07-27 | 2022-04-08 | 山东大学 | Video semantic segmentation method based on ConvLSTM convolutional neural network |
| CN111860386A (en) * | 2020-07-27 | 2020-10-30 | 山东大学 | Video semantic segmentation method based on ConvLSTM convolutional neural network |
| CN112364878A (en) * | 2020-09-25 | 2021-02-12 | 江苏师范大学 | Power line classification method based on deep learning under complex background |
| CN112766279A (en) * | 2020-12-31 | 2021-05-07 | 中国船舶重工集团公司第七0九研究所 | Image feature extraction method based on combined attention mechanism |
| CN113487524A (en) * | 2021-04-07 | 2021-10-08 | 北京百度网讯科技有限公司 | Image format conversion method, device, equipment, storage medium and program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109101975B (en) | 2022-01-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109101975A (en) | Image, semantic dividing method based on full convolutional neural networks | |
| CN111259905B (en) | Feature fusion remote sensing image semantic segmentation method based on downsampling | |
| CN110458165B (en) | Natural scene text detection method introducing attention mechanism | |
| CN104599275B (en) | The RGB-D scene understanding methods of imparametrization based on probability graph model | |
| CN110111335A (en) | A kind of the urban transportation Scene Semantics dividing method and system of adaptive confrontation study | |
| CN108985269A (en) | Converged network driving environment sensor model based on convolution sum cavity convolutional coding structure | |
| CN110427937A (en) | A kind of correction of inclination license plate and random length licence plate recognition method based on deep learning | |
| CN110111366A (en) | A kind of end-to-end light stream estimation method based on multistage loss amount | |
| CN108288035A (en) | The human motion recognition method of multichannel image Fusion Features based on deep learning | |
| CN110414387A (en) | A kind of lane line multi-task learning detection method based on lane segmentation | |
| CN113627228B (en) | Lane line detection method based on key point regression and multi-scale feature fusion | |
| CN105631415A (en) | Video pedestrian recognition method based on convolution neural network | |
| CN108021889A (en) | A kind of binary channels infrared behavior recognition methods based on posture shape and movable information | |
| CN112232351B (en) | License plate recognition system based on deep neural network | |
| CN113160062B (en) | Infrared image target detection method, device, equipment and storage medium | |
| CN109711401A (en) | A kind of Method for text detection in natural scene image based on Faster Rcnn | |
| CN111882620A (en) | Road drivable area segmentation method based on multi-scale information | |
| CN111161244B (en) | Industrial product surface defect detection method based on FCN + FC-WXGboost | |
| CN116051953A (en) | Small target detection method based on selectable convolution kernel network and weighted bidirectional feature pyramid | |
| CN110517272A (en) | Blood cell segmentation method based on deep learning | |
| CN110555461A (en) | scene classification method and system based on multi-structure convolutional neural network feature fusion | |
| CN109886082A (en) | A kind of Small object enhancing prediction module detection method based on SSD | |
| CN113743417A (en) | Semantic segmentation method and semantic segmentation device | |
| CN113205103A (en) | Lightweight tattoo detection method | |
| Zhaosheng et al. | Rapid detection of wheat ears in orthophotos from unmanned aerial vehicles in fields based on YOLOX |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220125 |