CN107194437B - Image classification method based on Gist feature extraction and concept machine recurrent neural network - Google Patents

Abstract

The invention discloses an image classification method based on Gist characteristic extraction and a concept machine recurrent neural network, which comprises the following steps: 1) extracting Gist characteristics of the image; 2) building a concept machine recurrent neural network; 3) inputting the characteristics generated in the step 1) as a training sample set into a concept machine recurrent neural network; 4) exciting the internal state of the reserve pool according to the training sample set of the images; 5) calculating and storing a concept machine matrix of the images according to the internal state of the reserve pool; 6) inputting the characteristics generated in the step 1) as a test sample set into a concept machine recurrent neural network to excite the internal state of the reserve pool, and calculating and identifying the images according to the concept machine matrix and the reserve pool state calculated in the step 5). The invention can better process the image classification problem, and the concept machine matrix of the image calculated by the steps 1), 3), 4) and 5) can enable the concept machine recurrent neural network to learn and identify new images in an incremental mode.

Description

Image classification method based on Gist feature extraction and concept machine recurrent neural network

Technical Field

The invention relates to the technical field of image processing, in particular to an image classification method.

Background

Image recognition and classification are an important direction in the field of computer vision and image understanding, and at present, many models have achieved obvious results in processing the image classification problem, and no matter the support vector machines in the past or the convolutional neural networks in the present have high accuracy, but the structures of the classifiers are already established after learning, and the classifiers cannot be changed, and new images cannot be continuously learned and recognized on the original basis. If learning is required to identify new images, all structures must be reinitialized, which necessarily wastes significant losses to retrain and learn the model.

Disclosure of Invention

In view of the above, the present invention is directed to an image classification method based on Gist feature extraction and concept machine recurrent neural network, so as to implement incremental learning to identify new images without repeated learning on the learned images.

The invention relates to an image classification method based on Gist feature extraction and a concept machine recurrent neural network, which comprises the following steps:

1) extracting Gist features of an image

Filtering the scene image through a Gabor filter, dividing the filtered image into t multiplied by t grids, and extracting global feature information of the image by adopting discrete Fourier transform and window Fourier transform in each grid;

the specific process is as follows: dividing a grayscale image f (x, y) with the size of h × w into t × t grids with equal size, wherein the size of each grid block is h '× w', and h '═ h/t and w' ═ w/t; convolution filtering is carried out on each grid block image by using filters of m channels, the results of filtering of the m channels are cascaded to form the characteristics of the grid block, the characteristic values calculated by each grid block are averaged to obtain the Gist characteristics of the grid block

Wherein G' represents the average eigenvalue generated after the m channel filtering; g (x, y) represents the characteristic values generated after the mth channel is filtered, and each grid block generates m characteristic values; cascading the m average characteristic values generated in each grid block to obtain a Gist characteristic of the whole image, wherein the dimension of the Gist characteristic is t multiplied by m;

2) building concept machine recurrent neural network

The concept machine recurrent neural network consists of input neurons, a reserve pool and output neurons, wherein a connection matrix among the reserve pool neurons conforms to an ESN rule, namely the spectrum radius β of a weight matrix connected inside the reserve pool is less than 1, so that the reserve pool is ensured to have echo state attributes, a reserve pool excitation function adopts a hyperbolic tangent function, and the update equation of the network is as follows:

x^j(n+1)＝tanh(W*x^j(n)+W_in*p^j(n+1)+b)

y^j(n)＝x^j(n)

wherein, W_inThe input weight matrix between the neuron of the input layer and the neuron of the reserve pool is composed of random numbers in standard normal distribution, W is a connection weight matrix between the neurons in the reserve pool, is generated by the random numbers in standard normal distribution, and then the spectral radius of the random numbers is restrained β<1; b is an offset, and has a value of 1; p is a radical of^j(n +1) is a concept machineAn input to the neural network; w_inW and b are fixed after generation;

3) inputting the image Gist characteristics generated in the step 1) as a training sample set into a concept machine recurrent neural network, namely p^j(n +1) is the training sample set resulting from step 1);

4) exciting internal states of a reservoir from a training sample set of such images

Training sample set p of images^jInputting a recurrent neural network of a concept machine, calculating and recording a training sample set p^jExcited reserve pool internal state set { x^j}；

5) Conceptual machine matrix for computing and storing such images

When the neural network is composed of model p^jDriven, N-dimensional excitatory neuron states { x^jLocated in a state star cloud, whose geometric characteristics are determined by the driving model; { x^jThe basic description of the geometry is an ellipse C^jThe principal axis is the state set { x }^jMajor component of }; this ellipse C^jA conceptual machine representing a model of interest in the recurrent neural network; according to the state set { x^jGet C through learning rule training^j(ii) a For the pool state sequence x (1), … x (l), the following cost function is constructed:

wherein C is a concept machine matrix which describes the characteristics of the state space of the reserve pool, α is more than or equal to 0 and is an adjusting parameter, and the concept machine C is obtained by a random gradient descent method:

C(R,α)＝R(R+α^-2I)^-1

wherein R is XX^TL, is a state correlation matrix, X is a set of states { X^jFinding the appropriate α according to the gradient of Frobenius squared norm:

for measuring the sensitivity of the concept machine C on an exponential scale when

The sensitivity of C to data changes is maximum when the maximum value is reached;

6) identifying such images based on the internal state of the reservoir excited by the conceptual machine matrix and the test sample

By training samples p^jGet concept machine C^jAnd will train sample p¹,p²… into a reserve pool; obtaining a test sample set p by steps 1), 3) and 4)ⁱReserve pool internal state set { xⁱAnd obtaining the category of the test picture through the following formula:

E＝X^TCX

j^*is the category to which the test image belongs.

The invention has the beneficial effects that:

the image classification method based on Gist feature extraction and the concept machine recurrent neural network can better deal with the image classification problem, and the concept machine matrix of the image is calculated through the steps 1), 3), 4) and 5), so that the concept machine recurrent neural network can learn and identify new images in an incremental mode.

Drawings

FIG. 1 is a diagram of a recurrent neural network of a conceptual machine, in which K input units means K input neurons, Ninternal units means N pool neurons, and L output units means L output neurons;

FIG. 2 is

The response condition of (2);

fig. 3 is a geometric description of a conceptual machine. Hand-written numberWord image set "0" model p¹Excited neuron state star cloud, the shape of which can be abstracted into ellipse, namely concept machine C¹；

FIG. 4 is a flowchart of an image classification method based on Gist feature extraction and concept machine recurrent neural network of the present invention.

Detailed Description

The invention is further described below with reference to the figures and examples.

To clearly illustrate the effectiveness of the present invention in classifying images, in this embodiment, an experiment of incremental learning and recognition of handwritten digital images is performed, and a common MNIST data set is used, and the handwritten digital images have 10 categories: the number "0", the number "1", the number "2", the number "3", the number "4", the number "5", the number "6", the number "7", the number "8" and the number "9", the training set had 60000 images, the test set had 10000 images, and each image size was 28 × 28. The image classification method based on Gist feature extraction and a concept machine recurrent neural network in the embodiment specifically comprises the following steps:

1) gist features of the images are extracted for a handwritten digital image set "0". Gist feature extraction method an image global feature extraction method, which forms an overall global description of an external scene to capture context information of an image. According to the feature extraction method, a scene image is filtered through a Gabor filter, the filtered image is divided into n multiplied by n grids, and then each grid adopts discrete Fourier transform and window Fourier transform to extract global feature information of the image.

The specific process is as follows: dividing a 28 × 28 grayscale image f (x, y) into 4 × 4 grids of equal size, each grid having a size h '× w', where h 'is 7 and w' is 7; convolution filtering is carried out on each grid block image by using 32-channel filters, the results obtained after 32-channel filtering are cascaded to form the characteristics of the grid block, the characteristic values calculated by each grid block are averaged to obtain the Gist characteristics of the grid block

Wherein G' represents the average eigenvalue generated after the m channel filtering; g (x, y) represents the eigenvalues generated after the m-th channel filtering. Thus, each grid block will produce 32 eigenvalues; and (3) cascading the 32 average feature values generated in each grid block to obtain the Gist feature of the whole image, wherein the dimension of the Gist feature is 4 multiplied by 32 to 512, and the 512-dimensional Gist feature of the image is used as a training sample set.

2) The method comprises the following steps of building a concept machine recurrent neural network, wherein the concept machine recurrent neural network is shown in figure 1 and comprises input neurons, a reserve pool and output neurons, wherein the number of the input neurons is 512, the number of the reserve pool neurons is 20, the number of the output neurons is 10, a connection matrix between the reserve pool neurons conforms to an ESN rule, namely the spectral radius β of a weight matrix connected inside the reserve pool is less than 1, and the reserve pool is ensured to have echo state attributes.

x^j(n+1)＝tanh(W*x^j(n)+W_in*p^j(n+1)+b)

y^j(n)＝x^j(n)

Wherein, W_inThe initialization method is that the input weight matrix between the neuron of the input layer and the neuron of the reserve pool is composed of the random numbers in the standard normal distribution, W is the connection weight matrix between the neurons in the reserve pool, the initialization method is that the random numbers in the standard normal distribution are firstly generated, then the characteristic value d ═ abs (W) with the maximum absolute value is calculated, and finally the spectrum radius is restrained β<1, W ═ 0.8W/d; b is an offset, and has a value of 1; p is a radical of^j(n +1) is the input of the recurrent neural network of the concept machine; w_inW and b are fixed after generation.

3) Inputting the features generated in the step 1) as a training sample set into a concept machine recurrent neural network, namely p¹(n +1) is the Gist feature training sample set of the handwritten digital image set "0" generated by step 1).

4) The internal state of the reservoir is excited from a training sample set of such images. Gist feature training of handwritten digital image set "0Sample set p¹Inputting a recurrent neural network of a concept machine, calculating and recording a training sample set p¹Excited reserve pool internal state set { x¹}。

5) A conceptual machine matrix of such images is calculated and stored. When the neural network is composed of model p¹Driven, N-dimensional excitatory neuron states { x¹Located in a state star cloud whose geometry is determined by the driving model. { x¹The basic description of the geometry is an ellipse C¹The principal axis is the state set { x }¹Major component of. This ellipse C¹A conceptual machine representing a handwritten digital image set "0" in the recurrent neural network is shown in fig. 3. According to the state set { x¹Get C through simple learning rule training¹. For the pool state sequence x (1), … x (l), the following cost function is constructed:

where C is the concept machine matrix, which characterizes the reservoir state space, α ≧ 0, which is a tuning parameter called "aperture", and a balance point can be found by tuning α to minimize the objective function.

C(R,α)＝R(R+α^-2I)^-1

Wherein R is XX^TL, is a state correlation matrix, X is a set of states { X¹}. finding a suitable "iris" α according to the gradient of the Frobenius squared norm:

the sensitivity (2 norm) of the conceptual machine C on an exponential scale can be measured when

To a maximum, C is most sensitive to data changes, similar to adjusting the aperture to maximize the sensitivity of the image to changes in brightness, as shown in FIG. 2, which depicts

When α is 5.2.

6) Such images are identified based on the conceptual machine matrix and the internal state of the reservoir excited by the test sample. Assume that a training sample p has passed^jGet concept machine C^jAnd training a sample p¹,p²… have all been loaded into the pool. Then the test sample set p may be alignedⁱIdentifying to obtain a test sample set p through the steps 1), 3) and 4)ⁱReserve pool internal state set { xⁱAnd obtaining the category of the test picture according to the following formula:

E＝X^TCX

j^*is the category to which the test image belongs.

By calculating the concept machine matrix of the handwritten digit image sets of 1, 2, 3, 4) and 5), the concept machine matrix of the handwritten digit image sets of 4, 5, 6, 7, 8 and 9 can enable the concept machine recurrent neural network to learn and identify new images in an incremental mode without repeated learning on the learned images, the learned image types can be identified by the step 6, and the identification accuracy of the MNIST data set is 98.6%.

Finally, the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all of them should be covered in the claims of the present invention.

Claims (1)

1. The image classification method based on Gist feature extraction and a concept machine recurrent neural network is characterized by comprising the following steps: the method comprises the following steps:

1) extracting Gist features of an image

Filtering the scene image through a Gabor filter, dividing the filtered image into t multiplied by t grids, and extracting global feature information of the image by adopting discrete Fourier transform and window Fourier transform in each grid;

the specific process is as follows: dividing a grayscale image f (x, y) with the size of h × w into t × t grids with equal size, wherein the size of each grid block is h '× w', and h '═ h/t and w' ═ w/t; convolution filtering is carried out on each grid block image by using filters of m channels, the results of filtering of the m channels are cascaded to form the characteristics of the grid block, the characteristic values calculated by each grid block are averaged to obtain the Gist characteristics of the grid block

Wherein G' represents the average eigenvalue generated after the m channel filtering; g (x, y) represents the characteristic values generated after the mth channel is filtered, and each grid block generates m characteristic values; cascading the m average characteristic values generated in each grid block to obtain a Gist characteristic of the whole image, wherein the dimension of the Gist characteristic is t multiplied by m;

2) building concept machine recurrent neural network

The concept machine recurrent neural network consists of input neurons, a reserve pool and output neurons, wherein a connection matrix among the reserve pool neurons conforms to an ESN rule, namely the spectrum radius β of a weight matrix connected inside the reserve pool is less than 1, so that the reserve pool is ensured to have echo state attributes, a reserve pool excitation function adopts a hyperbolic tangent function, and the update equation of the network is as follows:

x^j(n+1)＝tanh(W*x^j(n)+W_in*p^j(n+1)+b)

y^j(n)＝x^j(n)

wherein, W_inIs the input weight between the input layer neuron and the reserve pool neuronW is a connection weight matrix between neurons in the reserve pool, is generated by the random numbers in the standard normal distribution, and then the spectral radius is restrained β<1; b is an offset, and has a value of 1; p is a radical of^j(n +1) is the input of the recurrent neural network of the concept machine; w_inW and b are fixed after generation;

3) inputting the image Gist characteristics generated in the step 1) as a training sample set into a concept machine recurrent neural network, namely p^j(n +1) is the training sample set resulting from step 1);

4) exciting internal states of a reservoir from a training sample set of such images

Training sample set p of images^jInputting a recurrent neural network of a concept machine, calculating and recording a training sample set p^jExcited reserve pool internal state set { x^j}；

5) Conceptual machine matrix for computing and storing such images

When the neural network is composed of model p^jDriven, N-dimensional excitatory neuron states { x^jLocated in a state star cloud, whose geometric characteristics are determined by the driving model; { x^jThe basic description of the geometry is an ellipse C^jThe principal axis is the state set { x }^jMajor component of }; this ellipse C^jA conceptual machine representing a model of interest in the recurrent neural network; according to the state set { x^jGet C through learning rule training^j(ii) a For the pool state sequence x (1), … x (l), the following cost function is constructed:

wherein C is a concept machine matrix which describes the characteristics of the state space of the reserve pool, α is more than or equal to 0 and is an adjusting parameter, and the concept machine C is obtained by a random gradient descent method:

C(R,α)＝R(R+α^-2I)^-1

wherein R is XX^TL, is a state correlation matrix, X is a set of states { X^j}; according to Frobenius squareGradient of norm finds the appropriate α:

for measuring the sensitivity of the concept machine C on an exponential scale when

The sensitivity of C to data changes is maximum when the maximum value is reached;

6) identifying such images based on the internal state of the reservoir excited by the conceptual machine matrix and the test sample

By training samples p^jGet concept machine C^jAnd will train sample p¹,p²… into a reserve pool; obtaining a test sample set p by steps 1), 3) and 4)ⁱReserve pool internal state set { xⁱAnd obtaining the category of the test picture through the following formula:

E＝X^TCX

j^*is the category to which the test image belongs.

Images