CN107016415B - A kind of color image Color Semantic classification method based on full convolutional network - Google Patents
A kind of color image Color Semantic classification method based on full convolutional network Download PDFInfo
- Publication number
- CN107016415B CN107016415B CN201710236763.XA CN201710236763A CN107016415B CN 107016415 B CN107016415 B CN 107016415B CN 201710236763 A CN201710236763 A CN 201710236763A CN 107016415 B CN107016415 B CN 107016415B
- Authority
- CN
- China
- Prior art keywords
- color
- characteristic pattern
- color image
- image
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The color image Color Semantic classification method based on full convolutional network that the invention discloses a kind of, comprising: the 1 full convolutional network of building；2 obtain the color image data collection with Pixel-level mark；3 are trained full convolutional network using color image data collection, obtain the characteristic model that the classification of Pixel-level Color Semantic can be carried out to arbitrary dimension color image；4 are classified using the Color Semantic that characteristic model carries out Pixel-level to any color image, assess the nicety of grading of characteristic model；5 optimize processing to network class result using the method for full condition of contact random field, obtain the colors label of each pixel in image, according to the mapping relations of class label and color space, by the label converting Color Semantic classification results to corresponding color space display pixel grade of colors.The present invention is able to achieve the Color Semantic classification of color image pixel grade, effectively improves the precision that color image Color Semantic is classified under environment complicated and changeable.
Description
Technical field
The invention belongs in computer/machine vision, image procossing and analysis field, it is specifically a kind of based on full volume
The color image Color Semantic classification method of product network.
Background technique
In computer vision, it is the important channel of human perception image information that color, which is a kind of important attribute of image,.
By assigning image color class label, it can further apply to image retrieval, image labeling, colour blindness auxiliary, visual pursuit, language
Say the fields such as human-computer interaction.Therefore, good Color Semantic classification results facilitate further image procossing and image analysis.
Existing image color semantic classification method includes: the method based on statistical model and the side based on deep learning
Method.
Method based on statistical model is mainly based upon chromatic stimulus, such as by the tone of perceived color, brightness and satisfies
Color Semantic classification is carried out with these three color appearance attributes are spent.Class inherited of the different color on these three attributes is less than same
Kind color difference in the class on these three attributes causes the decision boundaries of Color Semantic classification to be difficult to determine, and in different fields
There are minor changes under scape, cause classifying quality poor.
In the method based on statistical model, representative method is the hidden semantic analysis (Probabilistic of probability
Latent SemanticAnalysis, PLSA).This method is on Lab color space by the word finder and group of composition model document
It is mapped at each color pixel collection of color image, regards colors as hiding main body variable；Use natural scene
Image training PLSA model obtains the colors hidden in pictures, to realize that Color Semantic is classified.Natural scene image
The quality of training set determines the accuracy of model, therefore under the complex scenes such as bloom, shade, the classifying quality of the model is simultaneously
It is undesirable.
Method based on deep learning mainly utilizes convolutional neural networks (Convolutional Neural
Network, CNN) carry out Color Semantic classification.This method learns colors from great amount of images sample, it is intended to predict small
The colors of image block.This method training convolutional neural networks include two stages: first stage, are carried out certainly using image block
Supervised training, color histogram is as its supervision message, to obtain Color Semantic disaggregated model；Second stage, image block
Label is changed to inherit from dad image and obtain, and by iterating, filters out noise sample, further increases Color Semantic classification mould
The precision of type.However under complex scene, the variation of intensity of illumination, light source etc. will lead to pixel rgb value in a channel or
It changes on multiple channels, the color histogram obtained at this time cannot really react the COLOR COMPOSITION THROUGH DISTRIBUTION of image block, by it
It is ineffective that the Color Semantic disaggregated model come is trained as supervision message；In second stage, partial color semantic classification
Successful image block is given up to fall because of inconsistent with the colors of dad image by mistake, to reduce available
Amount of training data.
Cheng et al. is improved in the method for convolutional neural networks, constructs the network structure of PCN-CNN, the net
Full articulamentum is changed to convolutional layer, the quantity for reducing pond layer, addition one on the basis of VGG16 network structure by network structure
Warp lamination realizes the color image pixel grade Color Semantic classification based on convolutional neural networks.Method compared with PLSA is in picture
It improves a lot in plain grade Color Semantic nicety of grading, but the model uses less pond layer, to the biggish coloured silk of resolution ratio
It is low that chromatic graph picture carries out precision when Color Semantic classification.
The Color Semantic that above-mentioned Color Semantic classification method realizes color image is classified, however point under complex scene
Class precision is not high.
Summary of the invention
The present invention is to solve shortcoming in the prior art, provides a kind of color image color based on full convolutional network
Semantic classification method, it is intended to the color attribute semantic classification for solving the problems, such as color image pixel grade, by constructing full convolutional Neural
Network obtains the good color attribute semantic classification network characterization model of the nicety of grading under complex scene, to improve complexity
The precision that color image Color Semantic is classified under changeable environment.
In order to solve the above technical problems, the technical solution adopted by the present invention is that:
A kind of the characteristics of Color Semantic classification method of color image based on full convolutional network of the present invention is according to as follows
Step carries out:
Step 1, the full convolutional network of building, and Pixel-level Color Semantic is carried out to the color image I (x, y, k) of arbitrary dimension
Classification；The full convolutional network is made of convolutional layer, pond layer, warp lamination；
Step 1.1, the input data of the full convolutional network are color image I (x, y, k), and x, y, k respectively indicates described
Height, width and the port number of color image I；
Defining operation number is t, and initializes t=1；
Definition pond number is v, and initializes v=1；
Initialize δ=0；
The color image I is sent into convolutional layer L by step 1.2c(ks,nt) the t times convolution operation is carried out, it obtains described complete
The characteristic pattern F of t-th of convolutional layer of convolutional networkc(x,y,nt)；Ks indicates the size of convolution kernel, ntIndicate t-th of convolutional layer
Convolution kernel number；
Step 1.3, to the characteristic pattern F of t-th of convolutional layerc(x,y,nt) the activation operation of t sublinear is carried out, obtain feature
Scheme Fr(x,y,nt)；
Step 1.4 after enabling t+1 be assigned to t, judges whether t≤δ+2 is true, if so, then repeat step 1.2 and step
1.3, otherwise, after enabling t-1 be assigned to t, execute step 1.5；
The t sublinear is activated the characteristic pattern F after operation by step 1.5r(x,y,nt) it is sent into pond layer Lp(kp,s)
After carrying out non-overlapping sampling, the characteristic pattern F in v stage is obtainedv(x,y,nv), wherein the range that kp, s respectively indicate sampling is big
Small and sliding step, nvIndicate the quantity of the characteristic pattern in v stage；
Step 1.6 enables δ+2 be assigned to δ, judges whether δ=4 are true, if so, then follow the steps 1.7；Otherwise, by v+1
After being assigned to v, return step 1.4；
Step 1.7 after enabling t+1 be assigned to t, judges whether t≤δ+3 is true, if so, then repeat step 1.2 and step
Step 1.7 is returned again to after 1.3, otherwise, t-1 is enabled to be assigned to t, after v+1 is assigned to v, jumps to step again after executing step 1.5
1.8；
Step 1.8 enables δ+3 be assigned to δ, judges whether δ=13 are true, if so, then follow the steps 1.9；Otherwise, t+ is enabled
After 1 is assigned to t, after repeating step 1.2 and step 1.3, return step 1.7；
Step 1.9, the characteristic pattern F to the v stagev(x,y,nv) convolution operation is carried out, obtained result carries out deconvolution behaviour
Make, to obtain characteristic pattern Fv′(x,y,nv)；
To the characteristic pattern F in v-1 stagev-1(x,y,nv-1) carry out convolution operation after with characteristic pattern Fv′(x,y,nv) be added,
Deconvolution is carried out again to operate to obtain characteristic pattern F 'v-1(x,y,nv-1)；
To the characteristic pattern F in v-2 stagev-2(x,y,nv-2) carry out convolution operation after with characteristic pattern F 'v-1(x,y,nv-1) phase
Add, then carries out deconvolution and operate to obtain characteristic pattern F 'v-2(x,y,nv-2)；
Step 1.10, according to the size of the color image I (x, y, k), to characteristic pattern Fv′(x,y,nv) cut, obtain coloured silk
Chromatic graph is denoted as C (x, y, p as the corresponding colors probability of I (x, y, k)i), wherein piThe probability of i-th of colors is represented,
I ∈ (0,1 ..., N), N are the quantity of colors；
Step 2 obtains the color image data collection with Pixel-level mark, is denoted as D={ Dtr,Dval,Dte}；DtrIndicate instruction
Practice data set；DvalIndicate validation data set；DteIndicate test data set；The training dataset Dtr, validation data set DvalWith
Test data set DteComprising color image and mark image；And it is described mark image in each pixel value V ∈ (0,
1,...,N)；
The training dataset DtrWith validation data set DvalIn all color images tri- channels RGB mean value, be denoted as
M={ mr,mg,mb}；
Step 3 is trained the full convolutional network；
Step 3.1, to the training dataset DtrIn all color images add σ1Gaussian noise, for changing colour
The contrast and rgb value of image；σ is added to all color images again2Gaussian noise for changing brightness, to obtain colour
Changing image；
Step 3.2, with the training dataset DtrIn all color images and its corresponding color transformation image as institute
The input data of full convolutional network training stage is stated, with the training dataset DtrIn all mark images as the input number
According to label；And color image and its color transformation image use identical colors label；
With the validation data set DvalIn test number of all color images as the full convolutional network training stage
According to the validation data set DvalIn all labels of the mark images as the test data；
Step 3.3 is trained the full convolutional network using stochastic gradient descent algorithm, obtains the layer of Color Semantic
Secondary characteristic model；
Step 4, with the test data set DteIn all color images and the mean value M difference as the character modules
The input data of type, and by the calculating of the characteristic model, obtain the colors probability of every width color image；
Step 5 optimizes the colors probability of every width color image using the method for full condition of contact random field
Processing, obtains the test data set DteThe colors of each pixel of middle color image, then by the color of each pixel
It is color class switching into color space, to obtain the Color Semantic classification results of Pixel-level.
Compared with the prior art, the beneficial effects of the present invention are:
1, the present invention supervises full convolutional network using label data and is learnt, to obtain by constructing full convolutional network
Must have the Color Semantic characteristic of division model of robustness, solve based in statistical model use color appearance attribute carry out color
Coloured silk perception, the problem that classification boundaries are complicated and nicety of grading is not high.
2, the full convolutional network constructed by the present invention reduces network size without the network structure of full articulamentum, thus
The huge parameter training in traditional convolutional neural networks to full articulamentum is avoided, training speed is greatly improved.Reasonably
Network structure can handle the smaller and biggish image of resolution ratio；Color Semantic classification results are extended to Pixel-level, solve volume
The problem of product neural network can not carry out Pixel-level classification, and Pixel-level Color Semantic nicety of grading greatly improved.
3, the present invention slightly changes color, the brightness of training data, for simulating in the full convolutional network training stage
Complex scene increases the type and quantity of training data to change caused by color image, limited in training dataset
In the case of, as far as possible the environmental factors such as illumination simulation condition, shooting visual angle on Color Semantic classification caused by influence, solve because
For poor fitting problem caused by training data deficiency during training network.
4, the present invention is classified using the Color Semantic that full convolutional network carries out Pixel-level to color image, utilizes full connection strap
Local color semantic association between the method combination pixel of part random field, to the Color Semantic classification results of full convolutional network
It optimizes, corrects for the pixel of classification error on color area profile, keep classifying edge more smooth, improve color language
The accuracy of justice classification.
Detailed description of the invention
Fig. 1 is the flow chart that the method for the present invention carries out Color Semantic classification to color image；
Fig. 2 is the structural schematic diagram of full convolutional network used herein；
Fig. 3 a is the schematic diagram of the part subset in the color image test set that the present invention uses；
Authentic signature schematic diagram corresponding to the part subset in color image test set that Fig. 3 b uses for the present invention；
Fig. 3 c is the partial color semantic classification result schematic diagram that the present invention is obtained using full convolutional network；
Fig. 3 d is that the partial color semantic classification result obtained after the present invention is optimized using full connection random field is illustrated
Figure.
Specific embodiment
As shown in Figure 1, in the present embodiment, a kind of Color Semantic classification method of the color image based on full convolutional network,
It is to carry out in accordance with the following steps:
Step 1, the full convolutional network of building, carry out Pixel-level color language for the color image I (x, y, k) to arbitrary dimension
Justice classification；Full convolutional network is made of convolutional layer, pond layer, warp lamination,
As shown in Fig. 2, in the present embodiment, full convolutional network includes the convolution pondization operation in five stages: the first and second rank
Section respectively includes two convolutional layers, a pond layer；Third and fourth, five stages respectively include three convolutional layers, a pond layer.Full volume
Product network shares 13 convolutional layers, five pond layers, three warp laminations.The arbitrary dimension size of color image refers to: sending
The size for entering the color image of network needs not be a fixed size；The color image that full convolutional network can be handled is most
Large scale is determined by the size of the video memory of used graphics processor.
Step 1.1, the input data of full convolutional network are color image I (x, y, k), and x, y, k respectively indicates color image I
Height, width and port number；
Defining operation number is t, and initializes t=1；
Definition pond number is v, and initializes v=1；
Initialize δ=0；
In this experiment test, the height of color image, width are respectively x=128, y=64, but not limited to this it takes
Value；Port number k=3 indicates red R, the green G, tri- channels blue B of color image.
Color image I is sent into convolutional layer L by step 1.2c(ks,nt) and the t times convolution operation is carried out to it, it is rolled up entirely
The characteristic pattern F of t-th of convolutional layer of product networkc(x,y,nt)；Ks indicates the size of convolution kernel, ntIndicate the volume of t-th of convolutional layer
Product core number；
Characteristic pattern F after convolutional layerc(x,y,nt) size are as follows: x=(x+2-ks)+1, y=(y+2-ks)+1.
In this experiment test, size that the convolution kernel size ks=3 of all convolutional layers, i.e. convolution operation do not change characteristic pattern；t
When≤2, convolution kernel number nt=64；When 2 < t≤4, convolution kernel number nt=128；When 4 < t≤7, convolution kernel number nt=
256；When 7 < t≤13, convolution kernel number nt=512；
Step 1.3, to the characteristic pattern F of t-th of convolutional layerc(x,y,nt) the activation operation of t sublinear is carried out, obtain feature
Scheme Fr(x,y,nt)；
Linear activation operate used in activation primitive be ReLU, the activation primitive is by characteristic pattern Fc(x,y,nt) in negative value
Element value sets 0, keeps positive value element value constant；Characteristic pattern F will not be changedc(x,y,nt) size.
Step 1.4 after enabling t+1 be assigned to t, judges whether t≤δ+2 is true, if so, then repeat step 1.2 and step
1.3, otherwise, after enabling t-1 be assigned to t, execute step 1.5；
T sublinear is activated the characteristic pattern F after operation by step 1.5r(x,y,nt) it is sent into pond layer Lp(kp, s) is carried out
After non-overlapping sampling, the characteristic pattern F in v stage is obtainedv(x,y,nv), wherein kp, s respectively indicate sampling range size and
Sliding step, nvIndicate the quantity of the characteristic pattern in v stage；
In this experiment test, pond layer carries out the range size kp=2 of non-overlapping sampling, sliding step s=2；Chi Hua
Layer takes characteristic pattern F of the maximum value pond after the sampling of pond layerv(x,y,nv) size are as follows:The characteristic pattern quantity in each stage and the convolution kernel number in corresponding stage are equal: when v=1, nv
=64；When v=2, nv=128；When v=3, nv=256；When 4≤v≤5, nv=512；
Step 1.6 enables δ+2 be assigned to δ, judges whether δ=4 are true, if so, then follow the steps 1.7；Otherwise, by v+1
After being assigned to v, return step 1.4；
Step 1.7 after enabling t+1 be assigned to t, judges whether t≤δ+3 is true, if so, then repeat step 1.2 and step
Step 1.7 is returned again to after 1.3, otherwise, t-1 is enabled to be assigned to t, after v+1 is assigned to v, jumps to step again after executing step 1.5
1.8；
Step 1.8 enables δ+3 be assigned to δ, judges whether δ=13 are true, if so, then follow the steps 1.9；Otherwise, t+ is enabled
After 1 is assigned to t, after repeating step 1.2 and step 1.3, return step 1.7；
Step 1.9, the characteristic pattern F to the v stagev(x,y,nv) convolution operation is carried out, obtained result carries out deconvolution behaviour
Make, to obtain characteristic pattern Fv′(x,y,nv)；
To the characteristic pattern F in v-1 stagev-1(x,y,nv-1) carry out convolution operation after with characteristic pattern Fv′(x,y,nv) be added,
Deconvolution is carried out again to operate to obtain characteristic pattern F 'v-1(x,y,nv-1)；
To the characteristic pattern F in v-2 stagev-2(x,y,nv-2) carry out convolution operation after with characteristic pattern F 'v-1(x,y,nv-1) phase
Add, then carries out deconvolution and operate to obtain characteristic pattern F 'v-2(x,y,nv-2)；
Deconvolution operation is the inverse process of convolution operation, the size of gained characteristic pattern after deconvolution operates are as follows: x
=(x-1) × ks+pad, y=(y-1) × ks+pad, in this experiment test, the deconvolution core in v stage and v-1 stage
Size ks=2, pad=4；The deconvolution core size k in v-2 stages=8, pad=16.The convolution kernel number n of convolution operationt=
12, the convolution kernel number n of deconvolution operationv=12.Before two width characteristic patterns are added, need to cut out the big characteristic pattern of size
It cuts, keeps the size of two width characteristic patterns consistent.
Step 1.10, to characteristic pattern F 'v-2(x,y,nv-2) cut, keep it in the same size with color image I (x, y, k),
The corresponding colors probability of color image I (x, y, k) is obtained, C (x, y, p are denoted asi), wherein piRepresent i-th of colors
Probability, i ∈ (0,1 ..., N), N be colors quantity；
In this experiment test, the mode of cutting is random cropping, the quantity N=11 of colors, this 11 kinds of colors point
Not are as follows: black, blue, brown, grey, green, orange, pink, purple, red, white, yellow.It is specific fixed
Justice refers to " Basic colorterms:Theiruniversality and evolution ", and the book is in 1991 by University of California
Publishing house publishes.
Step 2 obtains the color image data collection with Pixel-level mark, is denoted as D={ Dtr,Dval,Dte}；DtrIndicate instruction
Practice data set；DvalIndicate validation data set；DteIndicate test data set；Training dataset Dtr, validation data set DvalAnd test
Data set DteComprising color image and mark image；And mark the value V ∈ (0,1 ..., N) of each pixel in image；
Training dataset DtrWith validation data set DvalIn all color images tri- channels RGB mean value, be denoted as M=
{mr,mg,mb}；
Training dataset picture number is 10913, and validation data set picture number is 1500, test data set picture number
It is 1800.In this experiment test, mark image is to assign colors label V, V ∈ to pixel each in color image
(0,1 ..., 11), respectively indicates the color of the pixel are as follows: dark cyan, black, blue, brown, grey, green,
orange,pink,purple,red,white,yellow；Wherein dark cyan indicates that the pixel is background.Data set
Mean value M={ 93.53614,96.15632,102.91466 }.It as shown in Figure 3a, is the portion in color image test set of the present invention
Divide color image；As shown in Figure 3b, the corresponding Pixel-level mark figure of concentrated part color image is tested for color image of the present invention
Picture.The region that the present invention carries out Color Semantic classification to color image does not include background, hair, skin；These regions are unified to assign
Give colors label 0.
Step 3 is trained full convolutional network；
Step 3.1, to training dataset DtrIn all color images add σ1Gaussian noise to change color image
Contrast and rgb value；σ is added to all color images again2Gaussian noise for changing brightness, to obtain color transformation figure
Picture；In the present embodiment, σ1∈ [0,0.04], σ2=0.2；
Step 3.2, with training dataset DtrIn all color images and its color transformation image instructed as full convolutional network
Practice the input data in stage, with training dataset DtrIn all labels of the mark images as input data；And color image and
Its color transformation image uses identical label；
With validation data set DvalIn test data of all color images as the full convolutional network training stage, with verifying
Data set DvalIn all labels of the mark images as test data；
Step 3.3, using stochastic gradient descent algorithm (Stochastic Gradient Descent, SGD) to full convolution
Network is trained, and obtains the level characteristics model of Color Semantic；
During network repetitive exercise, each random one sub-picture of selection of stochastic gradient descent algorithm
It practises, Lai Gengxin model parameter.Specific method can refer to " the Stochastic GradientDescent of LeonBottou
Tricks ", this article are published in the 421-436 pages of " Neural networks:Tricks ofthe trade " in 2012.
Step 4, with test data set DteIn all color images and mean value M input number of the difference as characteristic model
According to, and by the calculating of characteristic model, obtain the colors probability of every width color image；
Colors of the color label as the pixel corresponding to maximum probability value in assumed appearance coloured silk class probability are such as schemed
Shown in 3c, for the Pixel-level colors for the color image for using full convolutional network to obtain, the value V ∈ of each pixel (0,
1 ..., 11), indicate the color of the pixel are as follows: dark cyan, black, blue, brown, grey, green, orange,
pink,purple,red,white,yellow.Compared with Fig. 3 c authentic signature shown in Fig. 3 d, the Pixel-level color of color image
Color classification and authentic signature are substantially uniform, but there are classification errors on some zonules.
Step 5 optimizes the colors probability of every width color image using the method for full condition of contact random field
Processing, obtains test data set DteThe colors of each pixel of middle color image, then by the color class of each pixel
It is not transformed into color space, to obtain the Color Semantic classification results of Pixel-level.
The pass of each pixel He other each pixels is described in full condition of contact random field using binary potential function
System encourages similar pixel to distribute identical label, that is, is classified as same color.Specific method can refer to paper " Efficient
Inference in Fully Connected CRFs with Gaussian Edge Potentials ", this article is in 2011
Year is published in international conference " the Advances in neural informationprocessing systems " phase the 4th of volume 23
Page.It as shown in Figure 3d, is the Pixel-level color of the color image obtained after using full condition of contact Random Fields Method to optimize
Classification, the value of each pixel be V ∈ (0,1 ..., 11), indicate the color of the pixel are as follows: dark cyan, black,
blue,brown,grey,green,orange,pink,purple,red,white,yellow.Final Pixel-level color language
Adopted classification results need colors being transformed into RGB color space.Fig. 3 d is compared with Fig. 3 c, corrects for the mistake of some zonules
Accidentally Color Semantic classification, profile of the classification boundaries closer to color image；It is compared with the authentic signature of Fig. 3 b, the color of the same area
Coloured silk is unified, and Color Semantic classification boundaries are smooth, clear-cut.Colors and RGB value there are one-to-one transforming relationship,
In this experiment test, the relationship of colors label, rgb value and color is as shown in table 1.
Table 1
Table 2
| Color image Color Semantic classification method | Ours | Cheng et al. | PLSA |
| PNS | 90.1 | 74.3 | 63.1 |
It as shown in table 2, is color image Color Semantic classification method of the invention with " pixel mark average "
(Pixel Annotation Score, PNS) is module, is carried out with current existing color image Color Semantic classification method
Quantify the analytical table of comparison.PNS value is bigger, shows that Color Semantic classification results precision is higher.In table 2, Ours indicates this hair
Bright color image Color Semantic classification method；Cheng et al. indicates to use improved convolutional neural networks method；PLSA is indicated
The hidden semantic analysis of probability.3 kinds of methods are tested on the color image test set that the present invention uses, it will thus be seen that
When being tested using small in resolution image, the method for method of the invention than Cheng et al. has in precision
Significantly promoted；It is compared with PLSA method, then there is absolute predominance.
Claims (1)
1. a kind of Color Semantic classification method of the color image based on full convolutional network, it is characterized in that in accordance with the following steps into
Row:
Step 1, the full convolutional network of building, and Pixel-level Color Semantic point is carried out to the color image I (x, y, k) of arbitrary dimension
Class；The full convolutional network is made of convolutional layer, pond layer, warp lamination；
Step 1.1, the input data of the full convolutional network are color image I (x, y, k), and x, y, k respectively indicates the colour
Height, width and the port number of image I；
Defining operation number is t, and initializes t=1；
Definition pond number is v, and initializes v=1；
Initialize δ=0；
The color image I is sent into convolutional layer L by step 1.2c(ks,nt) the t times convolution operation is carried out, obtain the full convolution
The characteristic pattern F of t-th of convolutional layer of networkc(x,y,nt)；Ks indicates the size of convolution kernel, ntIndicate the convolution of t-th of convolutional layer
Core number；
Step 1.3, to the characteristic pattern F of t-th of convolutional layerc(x,y,nt) the activation operation of t sublinear is carried out, obtain characteristic pattern Fr
(x,y,nt)；
Step 1.4 after enabling t+1 be assigned to t, judges whether t≤δ+2 is true, if so, step 1.2 and step 1.3 are then repeated,
Otherwise, after enabling t-1 be assigned to t, step 1.5 is executed；
The t sublinear is activated the characteristic pattern F after operation by step 1.5r(x,y,nt) it is sent into pond layer Lp(kp, s) is carried out
After non-overlapping sampling, the characteristic pattern F in v stage is obtainedv(x′,y′,nv), wherein kp, s respectively indicate the range size of sampling
And sliding step, nvIndicate the quantity of the characteristic pattern in v stage；The height of the characteristic pattern in x ' expression v stage, y ' expression v
The width of the characteristic pattern in stage, and
Step 1.6 enables δ+2 be assigned to δ, judges whether δ=4 are true, if so, then follow the steps 1.7；Otherwise, by v+1 assignment
After v, return step 1.4；
Step 1.7 after enabling t+1 be assigned to t, judges whether t≤δ+3 is true, if so, after then repeating step 1.2 and step 1.3
Step 1.7 is returned again to, otherwise, t-1 is enabled to be assigned to t, after v+1 is assigned to v, jumps to step 1.8 again after executing step 1.5；
Step 1.8 enables δ+3 be assigned to δ, judges whether δ=13 are true, if so, then follow the steps 1.9；Otherwise, t+1 is enabled to assign
It is worth to after t, after repeating step 1.2 and step 1.3, return step 1.7；
Step 1.9, the characteristic pattern F to the v stagev(x′,y′,nv) convolution operation is carried out, obtained result carries out deconvolution behaviour
Make, to obtain characteristic pattern Fv′(x″,y″,nv)；X " indicates characteristic pattern Fv' height, y " indicate characteristic pattern Fv' width, and
X "=(x-1) × ks+ pad, y "=(y-1) × ks+pad；
To the characteristic pattern F in v-1 stagev-1(x′,y′,nv-1) carry out convolution operation after with characteristic pattern Fv′(x″,y″,nv) be added,
Deconvolution is carried out again to operate to obtain characteristic pattern F 'v-1(x″,y″,nv-1)；
To the characteristic pattern F in v-2 stagev-2(x′,y′,nv-2) carry out convolution operation after with characteristic pattern F 'v-1(x″,y″,nv-1) phase
Add, then carries out deconvolution and operate to obtain characteristic pattern F 'v-2(x″,y″,nv-2)；
Step 1.10, according to the size of the color image I (x, y, k), to characteristic pattern Fv′(x″,y″,nv) cut, obtain colour
The corresponding colors probability of image I (x, y, k), is denoted as C (x, y, pi), wherein piRepresent the probability of i-th of colors, i
∈ (0,1 ..., N), N are the quantity of colors；
Step 2 obtains the color image data collection with Pixel-level mark, is denoted as D={ Dtr,Dval,Dte}；DtrIndicate training number
According to collection；DvalIndicate validation data set；DteIndicate test data set；The training dataset Dtr, validation data set DvalAnd test
Data set DteComprising color image and mark image；And it is described mark image in each pixel value V ∈ (0,1 ...,
N)；
The training dataset DtrWith validation data set DvalIn all color images tri- channels RGB mean value, be denoted as M=
{mr,mg,mb}；
Step 3 is trained the full convolutional network；
Step 3.1, to the training dataset DtrIn all color images add σ1Gaussian noise, for changing color image
Contrast and rgb value；σ is added to all color images again2Gaussian noise for changing brightness, to obtain color transformation
Image；
Step 3.2, with the training dataset DtrIn all color images and its corresponding color transformation image as described complete
The input data of convolutional network training stage, with the training dataset DtrIn all mark images as the input data
Label；And color image and its color transformation image use identical colors label；
With the validation data set DvalIn test data of all color images as the full convolutional network training stage, with
The validation data set DvalIn all labels of the mark images as the test data；
Step 3.3 is trained the full convolutional network using stochastic gradient descent algorithm, and the level for obtaining Color Semantic is special
Levy model；
Step 4, with the test data set DteIn all color images and the mean value M difference as the characteristic model
Input data, and by the calculating of the characteristic model, obtain the colors probability of every width color image；
Step 5 optimizes processing to the colors probability of every width color image using the method for full condition of contact random field,
Obtain the test data set DteThe colors of each pixel of middle color image, then by the colors of each pixel
It is transformed into color space, to obtain the Color Semantic classification results of Pixel-level.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710236763.XA CN107016415B (en) | 2017-04-12 | 2017-04-12 | A kind of color image Color Semantic classification method based on full convolutional network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710236763.XA CN107016415B (en) | 2017-04-12 | 2017-04-12 | A kind of color image Color Semantic classification method based on full convolutional network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107016415A CN107016415A (en) | 2017-08-04 |
| CN107016415B true CN107016415B (en) | 2019-07-19 |
Family
ID=59446649
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710236763.XA Expired - Fee Related CN107016415B (en) | 2017-04-12 | 2017-04-12 | A kind of color image Color Semantic classification method based on full convolutional network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107016415B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107527351B (en) * | 2017-08-31 | 2020-12-29 | 华南农业大学 | Lactating sow image segmentation method fusing FCN and threshold segmentation |
| CN107992850B (en) * | 2017-12-20 | 2020-01-14 | 大连理工大学 | Outdoor scene three-dimensional color point cloud classification method |
| US11636665B2 (en) | 2018-01-15 | 2023-04-25 | Shenzhen Corerain Technologies Co., Ltd. | Streaming image semantic segmentation method, logical integrated circuit system and electronic device |
| CN108830870B (en) * | 2018-05-21 | 2021-12-28 | 千寻位置网络有限公司 | Satellite image high-precision farmland boundary extraction method based on multi-scale structure learning |
| CN109063723B (en) * | 2018-06-11 | 2020-04-28 | 清华大学 | Weak supervision image semantic segmentation method based on common features of iteratively mined objects |
| CN109102457B (en) * | 2018-06-12 | 2023-01-17 | 杭州米绘科技有限公司 | Intelligent color changing system and method based on convolutional neural network |
| CN109379572B (en) | 2018-12-04 | 2020-03-06 | 北京达佳互联信息技术有限公司 | Image conversion method, image conversion device, electronic equipment and storage medium |
| CN110263151B (en) * | 2019-05-06 | 2023-02-07 | 广东工业大学 | Latent semantic learning method for multi-view multi-label data |
| CN110602527B (en) | 2019-09-12 | 2022-04-08 | 北京小米移动软件有限公司 | Video processing method, device and storage medium |
| CN111179272B (en) * | 2019-12-10 | 2024-01-05 | 中国科学院深圳先进技术研究院 | Rapid semantic segmentation method for road scene |
| CN111784709B (en) * | 2020-07-07 | 2023-02-17 | 北京字节跳动网络技术有限公司 | Image processing method, image processing device, electronic equipment and computer readable medium |
| CN112070760B (en) * | 2020-09-17 | 2022-11-08 | 安徽大学 | Bone mass detection method based on convolutional neural network |
| CN112348792A (en) * | 2020-11-04 | 2021-02-09 | 广东工业大学 | X-ray chest radiography image classification method based on small sample learning and self-supervision learning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105426462A (en) * | 2015-11-13 | 2016-03-23 | 深圳码隆科技有限公司 | Image searching method and device based on image element |
| CN106407903A (en) * | 2016-08-31 | 2017-02-15 | 四川瞳知科技有限公司 | Multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method |
| CN106447658A (en) * | 2016-09-26 | 2017-02-22 | 西北工业大学 | Significant target detection method based on FCN (fully convolutional network) and CNN (convolutional neural network) |
| CN106504064A (en) * | 2016-10-25 | 2017-03-15 | 清华大学 | Clothes classification based on depth convolutional neural networks recommends method and system with collocation |
-
2017
- 2017-04-12 CN CN201710236763.XA patent/CN107016415B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105426462A (en) * | 2015-11-13 | 2016-03-23 | 深圳码隆科技有限公司 | Image searching method and device based on image element |
| CN106407903A (en) * | 2016-08-31 | 2017-02-15 | 四川瞳知科技有限公司 | Multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method |
| CN106447658A (en) * | 2016-09-26 | 2017-02-22 | 西北工业大学 | Significant target detection method based on FCN (fully convolutional network) and CNN (convolutional neural network) |
| CN106504064A (en) * | 2016-10-25 | 2017-03-15 | 清华大学 | Clothes classification based on depth convolutional neural networks recommends method and system with collocation |
Non-Patent Citations (3)
| Title |
|---|
| Efficient convolutional neural networks for pixelwise classification on heterogeneous hardware systems;Fabian Tshopp 等;《IEEE International Symposium on Biomedical Imaging》;20161231;第1228-1228页 |
| 基于并行深度卷积神经网络的图像美感分类;王伟凝 等;《自动化学报》;20160630;第42卷(第6期);第904-914页 |
| 行人检测技术研究;张旭东;《中国优秀硕士学位论文全文数据库 信息科技辑》;20111215;第2011年卷(第12期);第I138-1074页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107016415A (en) | 2017-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107016415B (en) | A kind of color image Color Semantic classification method based on full convolutional network | |
| Gijsenij et al. | Color constancy using natural image statistics and scene semantics | |
| US10535141B2 (en) | Differentiable jaccard loss approximation for training an artificial neural network | |
| US20200210773A1 (en) | Neural network for image multi-label identification, related method, medium and device | |
| Li et al. | Aesthetic visual quality assessment of paintings | |
| Narihira et al. | Learning lightness from human judgement on relative reflectance | |
| Falomir et al. | Categorizing paintings in art styles based on qualitative color descriptors, quantitative global features and machine learning (QArt-Learn) | |
| CN108921092B (en) | Melanoma classification method based on convolution neural network model secondary integration | |
| Rosenthal et al. | Color statistics of objects, and color tuning of object cortex in macaque monkey | |
| Ma et al. | Shadow detection of moving objects based on multisource information in internet of things | |
| CN108564589A (en) | A kind of plant leaf blade dividing method based on the full convolutional neural networks of improvement | |
| Yu | Online teaching quality evaluation based on emotion recognition and improved AprioriTid algorithm | |
| Zhang et al. | Inkthetics: a comprehensive computational model for aesthetic evaluation of Chinese ink paintings | |
| CN110807485B (en) | Method for fusing two-classification semantic segmentation maps into multi-classification semantic map based on high-resolution remote sensing image | |
| CN110889718B (en) | Scheme screening method, scheme screening device, medium and electronic equipment | |
| Niu et al. | CF‐based optimisation for saliency detection | |
| Murray et al. | Toward automatic and flexible concept transfer | |
| CN109978074A (en) | Image aesthetic feeling and emotion joint classification method and system based on depth multi-task learning | |
| Shete et al. | TasselGAN: An application of the generative adversarial model for creating field-based maize tassel data | |
| Pérez et al. | Interactive feature space extension for multidimensional data projection | |
| Saraswat et al. | Colour normalisation of histopathological images | |
| CN102184403B (en) | Optimization-based intrinsic image extraction method | |
| Velastegui et al. | The importance of color spaces for image classification using artificial neural networks: a review | |
| CN116778223A (en) | Multi-category target domain data set labeling method and system based on zero labeling | |
| Li et al. | Neural network’s selection of color in UI design of social software |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190719 |