CN114462507A

CN114462507A - Rock slag classification algorithm based on convolutional neural network

Info

Publication number: CN114462507A
Application number: CN202210025857.3A
Authority: CN
Inventors: 陈昌川; 王新立; 乔飞; 刘凯; 代少升; 张天骐
Original assignee: Chongqing University of Post and Telecommunications
Current assignee: Chongqing University of Post and Telecommunications
Priority date: 2022-03-23
Filing date: 2022-03-23
Publication date: 2022-05-10

Abstract

The invention relates to a rock slag classification algorithm based on a convolutional neural network, wherein rock slag is used as a main product in a tunneling process and contains abundant information, and research and engineering practices show that the working condition of a cutter head can be inferred in real time by observing the category of the rock slag. The purpose of indirectly detecting the cutter head is achieved by identifying the rock slag type through images, so that field construction personnel are guided to check and replace the cutter in time. The invention provides a rock slag classification algorithm based on a convolutional neural network, which is motivated by the fact that a camera collects images above a conveyor belt, avoids a severe production environment, and has the advantages of simple equipment, low loss, low cost, long-time monitoring and the like. In order to extract the rock slag image characteristics, the text firstly migrates learning parameters according to a pre-training model, and simultaneously, by matching with a data amplification method, an amplified rock slag data set is used as the input of a convolutional neural network to train a rock slag classification network. In order to realize the algorithm deployment of the shield machine edge end, the invention also provides a network compression method combining quantification and pruning, which performs nearly lossless compression on the network and reduces the parameter quantity and the calculated quantity.

Description

Rock slag classification algorithm based on convolutional neural network

Technical Field

The invention belongs to the field of image recognition, and particularly relates to a rock slag classification algorithm based on a convolutional neural network.

Background

The full face rock Tunnel Boring Machine (TBM) has the advantages of high construction speed, high safety, good economy and the like, and is widely applied to heavy projects such as highways, railway transportation, urban subways and the like. Because the working environment is severe, the cutter disc hob is extremely easy to be abnormally abraded and damaged, if the damage of the cutter cannot be found and processed in time in the construction process, the tunneling efficiency can be reduced, the cutter disc is also abnormally abraded, and the engineering progress and the construction quality are influenced. Therefore, the method researches the abrasion reason of the shield cutter and monitors the abrasion condition of the cutter in real time, and is particularly necessary for reasonable selection, use, maintenance and replacement of the cutter.

At present, a plurality of different cutter head monitoring methods are available at home and abroad, mainly including a hydraulic monitoring method, an eddy current monitoring method, an ultrasonic detection method and the like, and the methods have certain success, but have certain limitations, for example, a sensor needs to be installed at the cutter head, the circuit arrangement is complex, and the service life of the sensor can be influenced by great vibration generated when the cutter head works. According to a rock breaking mechanism of a hob of a TBM cutterhead, when the cutterhead is worn or damaged, the size of rock debris generated by tunneling can be increased. Therefore, the state of the cutter head can be indirectly monitored by monitoring the rock slag fragment condition on the conveyor belt, and site constructors can be guided to check and replace the cutter in time. The camera collects images above the conveyor belt, avoids severe production environment, and has the advantages of simple equipment, low loss, low cost, long-time monitoring and the like. The rock slag is used as a main product in the tunneling process and contains abundant information, and research and engineering practices show that the working condition of the cutter head can be inferred in real time by observing the type of the rock slag. The purpose of indirectly detecting the cutter head is achieved by identifying the rock slag type through images, so that field construction personnel are guided to check and replace the cutter in time. The camera collects images above the conveyor belt, avoids severe production environment, and has the advantages of simple equipment, low loss, low cost, long-time monitoring and the like.

In the current engineering practice, the state of the rock slag is generally judged by manpower, the analysis and research on the rock slag are less, and the Yan long bin and the like are combined with actual construction scene data to research the correlation between the particle size of the rock slag and the abrasion of a cutter, and find that the particle size of the rock slag and the abrasion of the cutter are positively correlated; ganzaize and the like utilize a watershed segmentation algorithm to segment and measure the size of a rock slag image, the purpose of indirectly monitoring the abrasion of a cutter is achieved, and certain errors exist in a measurement result judged based on work experience. Based on the wide application and the superior performance of a Convolutional Neural Network (CNN) in the field of computer vision, the rock debris classification algorithm based on the convolutional neural network is provided, the convolutional neural network algorithm can extract depth features in images, has good rotation and translation invariance and can play a great role in the application of rock debris identification.

Disclosure of Invention

The invention realizes a rock slag classification algorithm based on a convolutional neural network and realizes classification and identification of three types of rock slag, and the specific technical scheme comprises the following 3 parts.

(1) And selecting 3 types of labeled rock slag images as the input of the convolutional neural network, and carrying out fine tuning and retraining on the network by using a transfer learning method.

(2) And the rock slag data set is amplified by a data amplification method, so that the classification performance of the network is improved.

(3) The rock slag classification network is compressed by adopting a quantitative pruning method, so that the scale and the calculated amount of the network are reduced, and the network is easier to deploy at the edge end of the construction environment.

Compared with other cutter head detection methods, the method has the advantages that: 1. the high classification accuracy is realized based on the convolutional neural network. 2. The method based on computer vision has the advantages of simple equipment, low loss, low cost, capability of monitoring for a long time and the like. 3. By the network compression method, the pressure is reduced for the edge deployment of the algorithm.

Drawings

FIG. 1 is a schematic diagram of a rock debris classification network structure based on a convolutional neural network according to the present invention

FIG. 2 is a sample graph of three types of rock slag data sets

FIG. 3 is a schematic diagram of a fixed-point representation of 8 bits

FIG. 4 is a diagram of a quantization process

FIG. 5 is a schematic diagram of network channel pruning

FIG. 6 is a flow chart of channel pruning

Detailed Description

The invention is used for providing a rock debris classification algorithm based on a convolutional neural network, and in order to make the technical scheme and the effect of the invention clearer and clearer, the following describes a specific implementation mode of the invention in detail with reference to the attached drawings.

As shown in figure 1, the invention designs a rock slag classification algorithm based on a convolutional neural network. And (3) taking rock slag images generated during the operation of the shield tunneling machine as network input, extracting features through the convolutional layer, and classifying the images through the full-connection layer. In order to further improve the accuracy of the network, the rock slag data set is subjected to data expansion, the expanded data set solves the problem of low recognition rate of specific rock slag, the overall performance of the network is improved, and the limitation of the algorithm is proved on a verification set. Meanwhile, in order to adapt to the edge end application scene of the rock slag classification algorithm, the calculation amount and the storage amount required by the network are greatly reduced by combining 8-bit fixed point quantization with convolution kernel channel pruning.

1. Rock slag classification network

The rock slag classification network is composed of three network layers, namely a convolution layer, a pooling layer and a full-connection layer. The convolutional layer is used for extracting features and is the most important calculation layer of CNN. High-level features in the image may be extracted. For each convolution calculation, each output characteristic value is obtained by multiplying the corresponding convolution kernel and the input characteristic value by accumulation operation and then adding offset. The operation between different characteristic values has no data dependency, thus the convolution calculation can be calculated in parallel.

The pooling layer is usually located behind the convolutional layer, plays a role in reducing the amount of calculation and controlling overfitting, and usually performs pooling downsampling respectively behind each feature map to reduce the dimensionality of the feature map, and there are generally two pooling methods: average pooling and maximum pooling, maximum pooling is used in the algorithm herein because maximum pooling allows more retention of information such as edges, textures, etc. of the image, and the extracted features allow better classification functionality.

The full connection layer plays a role of a classifier in the whole network, and because the result obtained by network convolution operation is the characteristic information of the image, and manual screening processing cannot be performed, the full connection layer is required to integrate the characteristic information, and the features extracted by the convolution layer are classified by the SoftMax classifier to establish a relation with a sample marking space.

Because the rock slag data set has too few training data, the characteristics cannot be fully learned through direct training, and overfitting is easy to generate. In the convolutional neural network, the output of the characteristic diagrams of the first layers is not in great relation with an image data set, and the last layer of the network is closely related to selected data and task targets thereof, so that according to the classification targets, full connection layers are redesigned, the obtained network structure is shown in figure 1, the model is retrained by taking the weight of the pre-trained network as an initial value, the parameters of the pre-trained model of VGG16 trained by ImageNet are transferred into the rock residue classification network, and the practice proves that the weight of the pre-trained model has strong generalization performance, and after retraining, a good classification effect can be achieved on the rock residue data set. The parameter design of the network is shown in Table 1, the network parameter number is 14.7G, and the calculated amount is 30.7GFLOPs

Table 1: rock slag classification network parameter design

2. Data expansion

The rock slag data set used in the network training is collected from a real engineering scene, and comprises three types of rock slag, as shown in fig. 2, wherein 3880 pieces of 0 type rock slag, 1840 pieces of 1 type rock slag and 640 pieces of 2 type rock slag. Wherein the class 0 and class 1 rock residues belong to normal rock residues, and the class 2 rock residues belong to abnormal rock residues. Due to the fact that the number of the rock slag images of the three types is not balanced, the training data amount of the rock slag of the type 1 and the training data amount of the rock slag of the type 2 are too small, and the characteristics learned by the network are insufficient, the classification accuracy on the rock slag of the type 1 and the rock slag of the type 2 is not high. Therefore, the data set amplification method is used for amplifying the 1-type and 2-type rock residue images in the rock residue data set, and the classification accuracy of the two types of rock residues is improved.

In the actual production process, the rock slag may have random distribution in the direction, and the camera may be influenced by the environment to acquire an image containing noise, so that for the characteristics, a data set is amplified by adopting a random rotation and Gaussian noise adding mode, and the rock slag classification network is retrained. The accuracy before and after amplification of the network data is shown in table 2.

Table 2: network accuracy before and after data amplification

	Total rate of accuracy	Class	0 rock fragment accuracy	Class	1 rock slag accuracy	Class 2 rock slag accuracy
							Prior to data amplification	96％	97.9％	90.7％	73.4％
After data amplification	96.5％	97.9％	93.1％	98.4％

3. Network compression method

The rock debris classification network is obtained based on VGG16 transfer learning, the calculated amount and the parameter amount are huge, so that the original network needs to be compressed to reduce the calculated amount and the storage amount requirement, and the rock debris classification network is deployed on shield machine edge end equipment with limited calculation capacity and storage space.

The invention uses a network compression method combining quantification and pruning to compress the rock slag classification network.

The 8-bit data storage mode is shown in figure 3, the network parameter is stored by using the storage mode of 8-bit fixed point number plus decimal position, the fixed point number is expressed by using sign bit and mantissa bit, the specific calculation mode is shown in formula (1), and the variation of decimal f can express [ -128,127 ]]*2^-fThe number of ranges.

Wherein s represents sign bit, N represents bit width, 8 represents 8bit quantization, and m represents_iRepresenting the ith mantissa bit, 2^-fRepresenting the scaling factor corresponding to the fractional position f.

If the decimal position is calculated according to layers during quantization, the network accuracy rate is reduced more due to the weight quantization, and the influence on the activation value quantization is small. Therefore, the network is quantized by the quantization method shown in fig. 4, the weight and the biased decimal position of each convolution kernel in the layer are respectively calculated, and the activation value is calculated according to the layer, so that the accuracy of the network is ensured.

Quantization reduces the storage requirement of the weight, but the parameter number and the calculation amount of the network cannot be obviously reduced, so that the purpose of reducing the network scale is achieved by pruning the redundant channel of the convolution kernel by using a channel pruning method as shown in fig. 5. The process is as follows

1. The absolute value sums of the convolutional layer weights of each layer are calculated and sorted.

2. And setting a proper pruning threshold value for each layer according to the calculation result of each layer.

3. The sum of the absolute values of each convolution kernel is compared to a threshold, pruning off convolution kernels that are less than the threshold.

4. And uniformly trimming corresponding output characteristic diagram channels according to the trimmed convolution kernels of the layer to form a new lightweight network structure.

5. And (5) retraining the trimmed network, and recovering the network accuracy.

Considering that too many convolution kernels are trimmed once, which causes great loss to the network accuracy and cannot be recovered through retraining, the iterative pruning method shown in fig. 6 is adopted to perform network pruning through multiple iterations. Considering the subsequent hardware realization, the network size is too small to be accelerated in parallel, so the number of the convolution kernels of each layer is selected to be trimmed to 50% of the original number, and the whole network size is reduced to 25% of the original network size. After the network is retrained and quantified after pruning, the accuracy of the network is 95.6 percent, wherein the accuracy of the type 0 rock residue, the accuracy of the type 1 rock residue and the accuracy of the type 2 rock residue are 97.2 percent, 93.3 percent and 96.6 percent respectively.

The compressed network parameters are shown in table 3, the network is compressed by a method of combining pruning and 8bit quantization, the compressed network parameter number is 3.7M, the calculated amount is 8.2GFLOPs, the network scale is reduced to 6.29% of the original network, and meanwhile, the overall accuracy of the network is reduced by only 0.9%.

Table 3: post-compression network parameter design

Claims

1. A rock slag classification algorithm based on a convolutional neural network is characterized by comprising the following steps:

step 1: and selecting 3 types of labeled rock slag images as the input of the convolutional neural network, and performing fine tuning design and retraining on the rock slag classification network by using a transfer learning method.

Step 2: and the rock slag data set is amplified by a data amplification method, so that the classification performance of the network is improved.

And step 3: and compressing the rock slag classification network by adopting a quantitative pruning method to obtain a lightweight network with smaller scale and calculation amount, so that the lightweight network is easier to deploy at the edge end of the construction environment.

2. The rock debris classification network fine tuning and retraining method based on transfer learning of claim 1, wherein: as the rock slag data set has too few training data, the characteristics cannot be fully learned through direct training, and the network is easy to overfit, researches show that the convolutional neural network has certain generalization performance in similar tasks, and the migration learning training network has better performance improvement speed and convergence performance. In the convolutional neural network, the output of the characteristic diagrams of the first layers is not related to an image data set, and the last layer of the network is closely related to selected data and task targets thereof, so that the fully-connected layers are redesigned according to the classification targets to obtain a new rock residue classification network structure, the model is retrained by taking the weight of the pre-training network as an initial value, the parameters of the pre-training model of VGG16 trained by ImageNet are transferred into the rock residue classification network, the practice proves that the weight of the pre-training model has strong generalization performance, the network accuracy reaches 96% after 1000 times of training, and a good classification effect can be achieved on the rock residue data set.

3. The data set amplification method of claim 2, wherein: the rock slag images collected from an actual engineering scene are not distributed in the number of categories, the categories with small data volume cannot fully extract the characteristic information, in the actual production process, the rock slag may be distributed randomly in the direction, and the camera may be influenced by the environment to collect the images containing noise, so that aiming at the characteristics, the data set is amplified by adopting the modes of random rotation and Gaussian noise addition, and the network is retrained to obtain the richer characteristic information of the three types of rock slag images.

4. The quantization and pruning-based network compression method of claim 3, wherein: and the scale of the rock slag classification network is reduced by using a network compression method combining quantification and pruning, so that the actual deployment and application of the equipment at the edge end of the shield tunneling machine are facilitated. Because the data after linear quantization and the original data are in a linear transformation relation, the process only needs to calculate a linear transformation function, the design of a hardware circuit is friendly, and the high-efficiency calculation on hardware is more conveniently realized. Therefore, the invention uses an 8bit fixed point quantization technique to perform linear quantization on the network. The convolution kernel weight and the activation value of each layer are quantized by using different quantization granularity, different quantization parameters are adopted for each convolution kernel in each layer, and the quantization parameters are calculated for the activation value according to the layer, so that the storage and bandwidth requirements of the network are reduced, and the accuracy of the network is maintained.

The network is quantized, so that the storage requirement of network parameters is reduced, but the calculated amount and the parameters of the network cannot be obviously reduced, and the lightweight network is necessary for accelerating edge-end hardware, so that the network model is further compressed by a network pruning method, and the purposes of compressing the model scale and reducing the storage calculation cost are achieved.

The pruning method is divided into two types of structured pruning and unstructured pruning. The unstructured pruning prunes the monomer weight of the network, and the weight data after pruning is stored in a sparse matrix form, so that the original network structure is greatly damaged, and meanwhile, higher requirements are provided for the hardware implementation of the algorithm; the structured pruning only causes quantity change by pruning a convolution kernel, a channel or a network layer, the original structure of the network is reserved, and the regular network structure can realize hardware acceleration of the network more easily.

Therefore, a channel pruning method is used in the method, and the aim of reducing the network size is fulfilled by pruning the redundant channels of the convolution kernels. The process is as follows

5. And (5) retraining the trimmed network, and recovering the network accuracy.

Considering that too many convolution kernels are trimmed once, the network accuracy is greatly lost, and the network cannot be recovered through retraining, the network pruning is performed through multiple iterations by adopting an iterative pruning method. Considering the subsequent hardware realization, the network size is too small to be accelerated in parallel, so the number of the convolution kernels of each layer is selected to be trimmed to 50% of the original number, and the whole network size is reduced to 25% of the original network size.

After 8bit quantization and channel pruning, the redundancy of the network is removed, the accuracy of the network is ensured, and meanwhile, the light algorithm is easier to deploy at the edge end of the shield machine scene.