CN114548884B - A package identification method and system based on pruning lightweight model - Google Patents

Abstract

The invention relates to a package recognition method and system based on a pruning lightweight model, belongs to the technical field of image recognition, and solves the problems of high computational complexity, high time and memory consumption, and difficulty in deployment on terminal equipment of the existing neural network model . Including inputting the training image into the neural network model to be pruned, extracting the feature map matrix of each convolutional layer; converting the feature map matrix into a weighted undirected graph, constructing an improved Laplacian matrix, and calculating Von Neuy Man graph entropy is taken as the original value; delete single vertices in the weighted undirected graph in turn to obtain a new weighted undirected graph, and calculate the change value of the von Neumann graph entropy of the new weighted undirected graph relative to the original value; Calculate the importance of the channel according to the change value of the entropy of the von Neumann graph, prune the channel to obtain the pruned lightweight model; deploy the pruned lightweight model to the package identification terminal device to identify the real-time collected pictures. The high pruning rate of the model is achieved and the efficiency of real-time package recognition is improved.

Description

A package identification method and system based on pruning lightweight model

technical field

The invention relates to the technical field of image recognition, in particular to a package recognition method and system based on a pruning lightweight model.

Background technique

In recent years, my country's express delivery business has shown a rapid growth trend overall, with huge development potential. In order to meet the sorting needs brought about by the rapid growth of express delivery business, the express delivery industry has put forward high requirements for the express sorting capacity of logistics transit centers. Modern logistics uses the logistics parcel automatic sorting system to sort the parcels according to different regions, in which the identification and detection of the parcels is an important part of express sorting.

In the current image recognition methods, traditional image processing methods have problems such as difficulty in artificially designing features in complex parcel sorting scenarios. In contrast, target recognition algorithms based on deep learning have obvious technical advantages.

With the continuous development of deep learning theory, deep convolutional neural networks are advancing towards deeper network layers and wider network architectures. On the one hand, this kind of complex model can effectively improve the learning ability of the neural network, so as to obtain better performance indicators; Larger memory footprint and longer training inference time. For a long time, deep convolutional neural networks have strict requirements on usage scenarios and are often deployed on the server side, but they cannot meet actual needs. In logistics sorting scenarios, real-time package recognition requires deep learning models with few parameters and fast computation. However, the inference time of complex models and the communication delay between the terminal and the server cannot meet the requirements of fast processing of package images, which greatly reduces the number of parameters. It limits the promotion and application of deep learning in this field.

SUMMARY OF THE INVENTION

In view of the above analysis, the embodiments of the present invention aim to provide a package identification method and system based on a pruned lightweight model, so as to solve the problem that the existing neural network model has high computational complexity, high time and memory consumption, and is difficult to use in terminal devices. deployment issues.

On the one hand, an embodiment of the present invention provides a package identification method based on a pruning lightweight model, including the following steps:

Input the training image into the pre-trained neural network model to be pruned, and extract the feature map matrix of each convolution layer;

Convert the feature map matrix of each convolutional layer into a weighted undirected map, construct an improved Laplacian matrix according to the magnitude matrix of the feature map matrix, and calculate the von Neumann map entropy as each convolutional layer. The original value of ; delete a single vertex in the weighted undirected graph of each convolutional layer in turn to obtain a new weighted undirected graph, and calculate the von Neumann graph entropy of each new weighted undirected graph relative to the original value. value change value;

According to the change value of the von Neumann graph entropy in each convolutional layer, the importance of the channel of each convolutional layer is calculated, and according to the pruning rate of each convolutional layer and the importance of the channel, the The channel is pruned, the pruned neural network model is trained, and the pruned lightweight model is obtained;

Deploy the pruning lightweight model to the package identification terminal equipment, and identify the package information in the real-time collected pictures.

Based on the further improvement of the above method, the feature map matrix of each convolutional layer is converted into a weighted undirected map, including:

Transform each three-dimensional feature map in the feature map matrix of each convolution layer to obtain a feature row vector matrix, where each row is a feature row vector corresponding to each channel, which is used as the weighted undirected weight of each convolution layer. vertex of the graph;

Calculate the cosine distance of any two feature row vectors in the feature row vector matrix of each convolution layer as the edge weight between the corresponding two vertices in the weighted undirected graph of each convolution layer;

According to the weighted undirected graph, get the adjacency matrix and degree matrix.

Based on the further improvement of the above method, the adjacency matrix is a real symmetric matrix, and its off-diagonal elements are the edge weights between the corresponding two vertices, and the diagonal elements are 0; the degree matrix is a diagonal matrix, and each row is diagonal. The element is the sum of all elements of the corresponding row in the adjacency matrix.

Based on the further improvement of the above method, an improved Laplacian matrix is constructed according to the magnitude matrix of the feature map matrix, including:

Obtain the magnitude matrix according to the feature map matrix. The magnitude matrix is a diagonal matrix, where each diagonal element is the sum of squares of all elements in the corresponding channel in the feature map matrix;

After multiplying the degree matrix by the magnitude matrix, and then subtracting the adjacency matrix, the improved Laplacian matrix is obtained.

Based on the further improvement of the above method, the von Neumann graph entropy is calculated by the following formula:

，n _i 表示第i个卷积层的通道数量。Among them, H _i represents the von Neumann graph entropy of the ith convolutional layer, L _d represents the improved Laplacian matrix; trace(•) represents the trace of the matrix, that is, the sum of all eigenvalues of the matrix; λ _k represents the k -th eigenvalue in the improved Laplacian matrix L _d , λ _k ≥ 0, and

, ni denotes the number of channels of the ith convolutional _layer .

Based on the further improvement of the above method, a single vertex in the weighted undirected graph of each convolutional layer is deleted in turn to obtain a new weighted undirected graph, and the von Neumann graph entropy of each new weighted undirected graph is calculated. Changes from the original value, including:

For the weighted undirected graph of each convolutional layer, delete the feature row vector corresponding to a single vertex in turn, re-convert the feature row vector matrix obtained after each deletion into a new weighted undirected graph, and construct a new improved The Laplace matrix of , calculates the new von Neumann graph entropy;

The absolute value of the difference between each new von Neumann graph entropy and the original value of the corresponding convolutional layer is calculated as the change value of the von Neumann graph entropy of the channel corresponding to the removed vertex.

Based on the further improvement of the above method, according to the change value of the von Neumann graph entropy in each convolutional layer, the importance of the channel of each convolutional layer is calculated, including:

According to the change value of the von Neumann map entropy of the channels of each convolutional layer obtained from each training picture, the average value of the change value of the von Neumann map entropy of each channel is calculated as the important value of each channel. sex.

Based on the further improvement of the above method, according to the pruning rate of each convolutional layer and the importance of the channel, the channel of each convolutional layer is pruned according to the pruning rate of each convolutional layer, according to the The importance of the channel, the channel is pruned from small to large.

Based on the further improvement of the above method, the neural network model to be pruned is a neural network model composed of convolutional layers with a CONV-BN-ReLU structure.

On the other hand, an embodiment of the present invention provides a package identification system based on a pruning lightweight model, including:

The feature map matrix extraction module is used to input the training image into the pre-trained neural network model to be pruned, and extract the feature map matrix of each convolutional layer;

The Von Neumann graph entropy calculation module is used to convert the feature map matrix of each convolution layer into a weighted undirected graph, construct an improved Laplacian matrix according to the magnitude matrix of the feature map matrix, and calculate The von Neumann graph entropy is used as the original value of each convolutional layer; the single vertex in the weighted undirected graph of each convolutional layer is deleted in turn to obtain a new weighted undirected graph, and each new weighted undirected graph is calculated. The value of the change in the von Neumann graph entropy of the graph relative to the original value;

The channel pruning module is used to calculate the importance of the channel of each convolutional layer according to the change value of the von Neumann graph entropy in each convolutional layer, and according to the pruning rate of each convolutional layer and the importance of the channel , prune the channels of each convolutional layer, train the pruned neural network model, and obtain a pruned lightweight model;

The package identification module is used to deploy the pruning lightweight model to the package identification terminal equipment, and identify the package information in the real-time collected pictures.

Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

1. Apply theories such as undirected graph and von Neumann graph entropy to neural network pruning, convert the feature map matrix of the neural network model into a weighted undirected graph, and consider the magnitude and correlation of the feature map. , calculate the entropy change value of the Von Neumann graph after deleting a single channel, and use this as the basis for pruning to ensure the performance of the lightweight model after pruning, and achieve significant model compression and acceleration;

2. The von Neumann map entropy of the feature map matrix has good stability, and has little correlation with the number of input training pictures. Only a small number of training pictures are needed to obtain accurate feature map channel importance, which makes the technical solution cut High branch efficiency, low memory usage and stable pruning results;

3. The channel pruning method of the present invention is suitable for a neural network model composed of a convolutional layer with a CONV-BN-ReLU structure. While maintaining the maximum performance, the amount of parameters and the amount of calculation are reduced as much as possible to realize the neural network. The model is lightweight, and the compressed lightweight deep neural network model is deployed on edge devices with limited resources, which expands the scope of use and maximizes the application value.

In the present invention, the above technical solutions can also be combined with each other to achieve more preferred combination solutions. Additional features and advantages of the invention will be set forth in the description which follows, and some of the advantages may become apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by means of particularly pointed out in the description and drawings.

Description of drawings

The drawings are only for the purpose of illustrating specific embodiments, and are not considered to be limitations of the present invention, and throughout the drawings, the same reference signs refer to the same components;

1 is a flowchart of a package identification method based on a pruning lightweight model in Embodiment 1 of the present invention;

2 is an example diagram of pruning of a package identification method based on a pruning lightweight model in Embodiment 1 of the present invention;

FIG. 3 is a schematic block diagram of a package identification system based on a pruning lightweight model in Embodiment 2 of the present invention.

Detailed ways

The preferred embodiments of the present invention are specifically described below with reference to the accompanying drawings, wherein the accompanying drawings constitute a part of the present application, and together with the embodiments of the present invention, are used to explain the principles of the present invention, but are not used to limit the scope of the present invention.

In logistics sorting scenarios, real-time package recognition requires deep learning models with few parameters and fast computation. However, the inference time of complex models and the communication delay between terminals and servers cannot meet the requirements for fast processing of package images. Aiming at the problems of large amount of parameters and floating-point calculation of the current deep neural network model, large time/memory consumption, and difficulty in deployment, the present invention proposes a package identification method and system based on a pruning lightweight model. Theories such as von Neumann graph entropy are applied to neural network pruning, the feature map matrix is converted into a weighted undirected graph, and the change value of von Neumann graph entropy after each channel is pruned is calculated as a measure The index of channel importance, taking into account the performance of the lightweight model after pruning, achieves significant model compression and acceleration, which is convenient for deployment in resource-constrained package recognition terminals to meet the needs of real-time package recognition.

Example 1

A specific embodiment of the present invention discloses a package identification method for pruning a lightweight model. As shown in Figure 1, the method includes the following steps:

S101: Input the training image into the pre-trained neural network model to be pruned, and extract the feature map matrix of each convolutional layer;

It should be noted that this embodiment does not limit the specific network structure of the neural network model to be pruned, as long as it is a neural network model composed of convolutional layers having a CONV-BN-ReLU structure. The CONV-BN-ReLU structure is widely used in a variety of mainstream convolutional neural network models, so that the pruning scheme in this method can be easily applied to mainstream neural network models in the fields of classification and recognition, and realizes lightweight neural network models. .

Exemplarily, the neural network model to be pruned may be a neural network model of the YOLO and ResNet series.

It should be noted that the package image dataset is constructed by pre-processing the image data, such as capturing images, labeling them, and scaling the image data, which is used to pre-train the neural network model to be pruned. In this step, the pre-trained neural network model to be pruned starts to be pruned and lightweight, and the required training images are randomly selected from the package image data set. When using more training images, theoretically more accurate feature map channel importance can be obtained, but this will increase the pruning time, reduce the pruning efficiency, and the pruning performance is not significantly improved; when using fewer training images When , the more accurate feature map channel importance cannot be obtained, and the pruning effect becomes worse. Therefore, in order to take into account the pruning efficiency and pruning performance, 600-1000 images are used for pruning. Preferably, 640 training images are used, divided into 5 batches, and each batch is input with 128 images.

Specifically, the training images are input into the pre-trained neural network model to be pruned in batches, and the extracted feature map matrix of each convolutional layer is a four-dimensional matrix, including the three-dimensional feature map generated by each image in each batch, It can be expressed by the following formula:

表示第i个卷积层输出的特征图矩阵，

表示特征图矩阵

中的第j个特征图；

表示在一个批次中输入到待剪枝的神经网络模型中的训练图片数量；n _i 表示第i个卷积层输出特征图矩阵中每个特征图的通道数量；h _i 和w _i 分别表示特征图的高度和宽度；

表示第i个卷积层输出特征图矩阵中第j个特征图

的第k个通道图。in,

represents the feature map matrix output by the ith convolutional layer,

Representation feature map matrix

The jth feature map in ;

Represents the number of training images input to the neural network model to be pruned in a batch; n _i represents the number of channels of each feature map in the output feature map matrix of the ith convolutional layer; h _i and w _i represent, respectively the height and width of the feature map;

Represents the jth feature map in the output feature map matrix of the ith convolutional layer

The kth channel map of .

Exemplarily, there are 55 convolutional layers in the ResNet56 neural network model. When 128 training images per batch are input, the feature map matrix generated by each convolutional layer is extracted, and a total of 275 features can be extracted in 5 batches. Map matrix, each feature map matrix includes 128 three-dimensional feature maps.

S102: Convert the feature map matrix of each convolutional layer into a weighted undirected map, construct an improved Laplacian matrix according to the magnitude matrix of the feature map matrix, and calculate the von Neumann map entropy as each volume The original value of the convolution layer; delete a single vertex in the weighted undirected graph of each convolutional layer in turn to obtain a new weighted undirected graph, and calculate the relative von Neumann graph entropy of each new weighted undirected graph. The change value of the original value of the corresponding convolutional layer;

It should be noted that the feature map matrix of each convolutional layer is converted into a weighted undirected map, including:

① Deform each three-dimensional feature map in the feature map matrix of each convolution layer to obtain a feature row vector matrix, where each row is a feature row vector corresponding to each channel, which is used as the weighted and non-linear vector of each convolution layer. to the vertex of the graph;

Specifically, deforming each three-dimensional feature map ( n _i , h _i , w _i ) in the extracted feature map matrix is to keep the number of channels n _i unchanged, and transform ( h _i , wi ) under each channel w _i ) All elements of the matrix are pulled into a row to obtain n _i characteristic row vectors ( n _i , h _i w _i ), corresponding to n _i vertices in the weighted undirected graph, where h _i and w _i represent respectively The height and width of the feature map, h _i w _i represent the number of elements in each feature row vector after deformation. The eigenrow vector matrix obtained after deformation is expressed by the following formula:

表示第i个卷积层第j个特征图变形后的特征行向量矩阵，

中的

表示与特征图中第k个通道对应的特征行向量，作为第i个卷积层的带权无向图G的一个顶点。in,

represents the deformed feature row vector matrix of the jth feature map of the ith convolutional layer,

middle

Represents the feature row vector corresponding to the kth channel in the feature map as a vertex of the weighted undirected graph G of the ith convolutional layer.

② Calculate the cosine distance of any two feature row vectors in the feature row vector matrix of each convolution layer as the edge weight between the corresponding two vertices in the weighted undirected graph of each convolution layer;

中任意两个特征行向量

和

的余弦距离：Specifically, according to formula (4), the calculation

Any two eigenrow vectors in

and

The cosine distance of :

和

分别表示特征行向量

和

中第m个元素。计算

中所有行向量两两之间的余弦距离，作为转换得到的带权无向图G相应两个顶点之间边的权重。in,

and

Represent the feature row vector

and

The mth element in . calculate

The cosine distance between all row vectors in , is used as the weight of the edge between the corresponding two vertices of the weighted undirected graph G obtained by conversion.

③According to the weighted undirected graph, obtain the adjacency matrix and degree matrix.

It should be noted that the adjacency matrix of a weighted undirected graph is a real symmetric matrix, and its off-diagonal elements are the edge weights between the corresponding two vertices, and the diagonal elements are 0, which are expressed as follows:

Among them, W represents the adjacency matrix of the weighted undirected graph G , and its off-diagonal elements dis _{p, q} are the weights of the edge between the vertex p and the vertex q .

The degree matrix of a weighted undirected graph is a diagonal matrix, and the diagonal elements of each row are the sum of all elements of the corresponding row in the adjacency matrix, which are expressed as follows:

Among them, S represents the degree matrix of the weighted undirected graph G , and its diagonal element sk is the sum of all elements in the kth _row of the adjacency matrix W.

Through the above three steps, the feature map matrix of each convolutional layer is converted into a weighted undirected graph, which is used to describe the correlation of the channels of each convolutional layer.

In this embodiment, in order to obtain a more accurate pruning standard, the channel correlation is described by a weighted undirected graph, the individual importance of the channel is described by the magnitude of the feature map, and finally the channel correlation and the individual channel importance are merged. the Laplace matrix.

Specifically, the magnitude of the feature map is represented by the magnitude matrix of the feature map, which is used to measure the influence of each channel on the neural network model. The magnitude matrix is a diagonal matrix, and each diagonal element is a feature map. The sum of squares of all elements in the corresponding channel in the matrix, the formula is as follows:

Among them, D represents the magnitude matrix of the feature map, and its diagonal element d _k is the sum of squares of all elements in the kth channel of the feature map matrix.

After multiplying the degree matrix by the magnitude matrix, and then subtracting the adjacency matrix, the improved Laplacian matrix is obtained, and the formula is as follows:

Among them, L _d represents the improved Laplacian matrix, which is a real symmetric positive semi-definite matrix, and * represents the multiplication of the corresponding elements of the matrix.

According to the improved Laplace matrix, the von Neumann graph entropy is calculated by the following formula:

，n _i 表示第i个卷积层的通道数量。Among them, H _i represents the von Neumann graph entropy of the ith convolutional layer, trace(•) represents the trace of the matrix, that is, the sum of all eigenvalues of the matrix; λ _k represents the improved Laplacian matrix L _d The k -th eigenvalue in , λ _k ≥ 0, and

, ni denotes the number of channels of the ith convolutional _layer .

It should be noted that the von Neumann graph entropy has good stability and is less correlated with the number of input training images, and only a small number of training images are needed to obtain accurate feature map channel importance. The von Neumann map entropy calculated according to the feature map matrix of each convolution layer is used as the original value of each convolution layer. Next, the change value of the von Neumann map entropy after deleting a single channel is calculated as Pruning basis.

Specifically, delete a single vertex in the weighted undirected graph of each convolutional layer in turn to obtain a new weighted undirected graph, and calculate the von Neumann graph entropy of each new weighted undirected graph relative to the corresponding The changing values of the original values of the convolutional layers, including:

For the weighted undirected graph of each convolutional layer, delete the feature row vector corresponding to a single vertex in turn, re-convert the feature row vector matrix obtained after each deletion into a new weighted undirected graph, and construct a new improved The Laplace matrix of , calculates the new von Neumann graph entropy;

中删除对应的特征行向量，按照公式(4)-(9)重新构造带权无向图G ^’ ，根据公式(10)得到新的改进后的拉普拉斯矩阵

，以及根据公式(11)得到新的冯·诺依曼图熵H ^’ 。It should be noted that a single vertex is removed from the original graph G , i.e. from the eigenrow vector matrix

Delete the corresponding eigenline vector in the , reconstruct the weighted undirected graph G ^' according to formulas (4)-(9), and obtain a new improved Laplacian matrix according to formula (10).

, and the new von Neumann graph entropy H ^' is obtained according to formula (11).

Calculate the absolute value of the difference between each new von Neumann graph entropy and the original value of the corresponding convolutional layer, as the change value of the von Neumann graph entropy of the channel corresponding to the deleted vertex, the formula is as follows Show:

Among them, abs(•) represents the operation to obtain the absolute value.

S103: Calculate the importance of the channels of each convolutional layer according to the change value of the von Neumann graph entropy in each convolutional layer, and prune the channels of each convolutional layer according to the pruning rate of each convolutional layer branch, train the pruned neural network model to obtain a pruned lightweight model;

It should be noted that the von Neumann graph entropy helps to measure the information difference and distance between graphs. After a channel is deleted, if the new von Neumann graph entropy changes significantly, it indicates that the channel is important It has high performance and needs to be retained during the pruning process to ensure the performance of the lightweight model after pruning. Therefore, the goal of model channel pruning is to subtract unimportant convolutional channels from the neural network model while preserving the performance and generalization of the original neural network model to the greatest extent possible. This goal can be defined as follows:

表示第k个通道的掩膜，取值为0或1，当

取0时表示该通道被剪除，当

取1时表示该通道被保留；CI(b _k )表示第k个通道b _k 的重要性；n _i 表示该卷积层中的通道数量；

表示剪枝后该卷积层保留的通道数量。Among them, Acc represents the performance of the neural network model after pruning;

Represents the mask of the kth channel, which takes a value of 0 or 1, when

When 0 is taken, it means that the channel is pruned, when

When 1 is taken, it means that the channel is reserved; CI( b _k ) means the importance of the kth channel b _k ; n _i means the number of channels in the convolutional layer;

Indicates the number of channels retained by this convolutional layer after pruning.

The importance of the channels of each convolutional layer is approximated by the following formula according to the change value of the von Neumann graph entropy in each convolutional layer:

表示输入的图片总数量。Among them, ΔH _k,j represents the von Neumann graph entropy change value obtained by the kth channel b _k of the convolutional layer when the jth image is input;

Indicates the total number of images entered.

According to formula (13) and formula (14), in order to maximize the performance of the model after pruning, it can be equivalent to retain the most important channel. Therefore, according to the pruning rate of each convolutional layer, the channel of each convolutional layer is pruned according to the pruning rate of each convolutional layer and the importance of the channel of each convolutional layer, from small to large. Perform pruning.

Taking the ResNet56 neural network model as an example, the neural network model contains 55 convolutional layers, then all channels of each layer in the 55 convolutional layers are sorted according to their importance from small to large.

FIG. 2 takes a convolutional layer as an example, showing the process of obtaining a pruned lightweight model through channel pruning in this embodiment. In Figure 2, the convolutional layer has 4 channels. After inputting a picture into the convolutional layer, a weighted undirected graph with 4 vertices is constructed, and the von Neumann graph calculated according to it is constructed. The entropy is used as the original value of the convolutional layer, and then a single vertex is deleted in turn to construct 4 new weighted undirected graphs containing 3 vertices, and the new von Neumann graph entropy is obtained and compared with the original value. , the von Neumann graph entropy change values for 4 channels are obtained. Since it is a picture, the size of the change value is equivalent to the size of the importance. When the pruning rate is 0.5, sort from small to large, and prune channels 4 and 1 with relatively small change values, and keep channels 2 and 1. 3.

It should be noted that for the neural network model, the channel importance of the output feature map of the same convolutional layer is comparable, and the feature map channels of different convolutional layers have different effects on the model, so given the neural network model channel cut Branch rate, channel importance ranking, and pruning operations are performed hierarchically in different convolutional layers of the model. The neural network model channel pruning rate can be set according to actual needs to achieve a balance between model performance and model computation. According to the experiment, the shallow convolution channel of the model is often more important than the deep convolution channel, so the pruning rate of the shallow convolution channel is generally smaller than the pruning rate of the deep convolution channel. In addition, for the residual network, it is also necessary to consider that the pruning rate of the residual output channel matches the pruning rate of the output channel of the backbone network to ensure that the two output features can be added.

In this embodiment, performing fine-tuning training on the pruned neural network model includes: setting fine-tuning training hyperparameters, including initial learning rate, cosine annealing decay strategy with preheating mechanism, batch size, number of iterations, and Stochastic gradient descent with momentum and weight decay and hierarchical pruning rate, fine-tune the pruned lightweight model on the GPU, and obtain a trained pruned lightweight model.

S104: Deploy the pruning lightweight model to the package identification terminal device, and identify the package information in the real-time collected pictures.

Specifically, based on the pictures collected in real time, the number and location information of the packages in the pictures are identified, and the index results of the package identification are counted, including: model size, recognition speed and recognition accuracy.

Exemplarily, in the CIFAR-10 dataset, pruning the ResNet56 neural network model can reduce the amount of parameters by 42.8% and the amount of floating-point calculations by 47.4%, and the accuracy of the lightweight model after pruning is improved by 1.02%. ; In the ImageNet dataset, pruning the ResNet50 neural network model can reduce 40.8% of parameters and 44.8% of floating-point calculations, and the accuracy of the lightweight model after pruning is improved by 0.25%. Compared with the unpruned neural network model, the method of this embodiment achieves significant model compression and acceleration, and can achieve better model accuracy than other similar methods at the same or even higher pruning rate.

Compared with the prior art, a package identification method based on a pruning lightweight model provided in this embodiment applies theories such as undirected graph and von Neumann graph entropy to neural network pruning, and the neural network The feature map matrix of the model is converted into a weighted undirected map, and the amplitude and correlation of the feature map are considered to calculate the change value of the von Neumann map entropy after deleting a single channel, which is used as the basis for pruning to ensure pruning. The performance of the post-branch lightweight model achieves significant model compression and acceleration; the von Neumann map entropy of the feature map matrix has good stability, and has little correlation with the number of input training pictures, only a small number of training pictures are required Accurate feature map channel importance can be obtained, so that the technical solution has high pruning efficiency, low memory usage and stable pruning results; the channel pruning method in this embodiment is suitable for convolution with CONV-BN-ReLU structure. The neural network model composed of layers, while maintaining the maximum performance, reduces the amount of parameters and computation as much as possible, realizes the lightweight of the neural network model, and deploys the compressed lightweight deep neural network model on edge devices with limited resources. , expanding the scope of use and maximizing the application value.

Embodiment 2

Another embodiment of the present invention discloses a package identification system based on a pruning lightweight model, thereby realizing the package identification method in the first embodiment. For the specific implementation of each module, refer to the corresponding description in the first embodiment. As shown in Figure 3, the system includes:

The feature map matrix extraction module S201 is used to input the training picture into the pre-trained neural network model to be pruned, and extract the feature map matrix of each convolution layer;

The von Neumann graph entropy calculation module S202 is used to convert the feature map matrix of each convolution layer into a weighted undirected graph, construct an improved Laplacian matrix according to the magnitude matrix of the feature map matrix, and calculate The von Neumann graph entropy is taken as the original value of each convolutional layer; the single vertex in the weighted undirected graph of each convolutional layer is deleted in turn to obtain a new weighted undirected graph, and each new weighted undirected graph is calculated. the change in the von Neumann graph entropy of the graph relative to the original value;

The channel pruning module S203 is used to calculate the importance of the channel of each convolutional layer according to the change value of the von Neumann graph entropy in each convolutional layer, and according to the pruning rate of each convolutional layer and the importance of the channel prune the channels of each convolutional layer, train the pruned neural network model, and obtain a pruned lightweight model;

The package identification module S204 is configured to deploy the pruning lightweight model to the package identification terminal device, and identify the package information in the real-time collected pictures.

Since the package identification system based on the pruning lightweight model in this embodiment and the aforementioned identification method can learn from each other, the description is repeated here, so it will not be repeated here. All systems used in the methods in the embodiments of the present invention should be covered within the protection scope of the present invention. Since the principles of the present system embodiments are the same as those of the foregoing method embodiments, the present system also has the corresponding technical effects of the foregoing method embodiments.

Those skilled in the art can understand that all or part of the process of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. Wherein, the computer-readable storage medium is a magnetic disk, an optical disk, a read-only storage memory, or a random-access storage memory, or the like.

The above description is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited to this. Substitutions should be covered within the protection scope of the present invention.

Claims (6)

1. A parcel identification method based on a pruning lightweight model is characterized by comprising the following steps:

inputting the training picture into a pre-trained neural network model to be pruned, and extracting a characteristic diagram matrix of each convolution layer; the training pictures are obtained by randomly extracting the training pictures from the wrapped picture data set;

converting the characteristic diagram matrix of each convolution layer into a weighted undirected graph, constructing an improved Laplace matrix according to the amplitude matrix of the characteristic diagram matrix, and calculating von Neumann diagram entropy as an original value of each convolution layer, wherein the method comprises the following steps:

deforming each three-dimensional characteristic diagram in the characteristic diagram matrix of each convolution layer to obtain a characteristic row vector matrix, wherein each row is a characteristic row vector corresponding to each channel and is respectively used as the vertex of a weighted undirected graph of each convolution layer;

calculating the cosine distance of any two characteristic row vectors in the characteristic row vector matrix of each convolution layer, and taking the cosine distance as the edge weight between two corresponding vertexes in the weighted undirected graph of each convolution layer;

acquiring an adjacency matrix and a degree matrix according to the weighted undirected graph;

obtaining a magnitude matrix according to the feature map matrix, wherein the magnitude matrix is a diagonal matrix, and each diagonal element is the sum of squares of all elements in corresponding channels in the feature map matrix;

multiplying the degree matrix by the amplitude matrix, and then subtracting the adjacent matrix to obtain an improved Laplace matrix;

the von Neumann map entropy is calculated by the following formula:

wherein,H _i is shown asiVon Neumann entropy of each convolutional layer,L _d representing the improved Laplace matrix; trace (·) represents the trace of the matrix, i.e., the sum of all eigenvalues of the matrix;λ _k representing the modified Laplace matrixL _d To middlekThe value of the characteristic is used as the characteristic value,λ _k is not less than 0, and

，n _i is shown asiThe number of channels of each convolutional layer;

deleting single vertexes in the weighted undirected graph of each convolution layer in sequence to obtain a new weighted undirected graph, and calculating the change value of von Neumann diagram entropy of each new weighted undirected graph relative to the original value, wherein the change value comprises the following steps:

sequentially deleting the characteristic row vectors corresponding to a single vertex aiming at the weighted undirected graph of each convolution layer, reconverting the characteristic row vector matrix obtained after deletion each time into a new weighted undirected graph, constructing a new improved Laplace matrix, and calculating a new von Neumann graph entropy;

calculating the absolute value of the difference value between each new von Neumann diagram entropy and the original value of the corresponding convolution layer to be used as the change value of the von Neumann diagram entropy of the channel corresponding to the deleted vertex;

calculating the importance of the channel of each convolutional layer according to the variation value of the von Neumann diagram entropy in each convolutional layer, pruning the channel of each convolutional layer according to the pruning rate of each convolutional layer and the importance of the channel, and training the pruned neural network model to obtain a pruning lightweight model;

and deploying the pruning lightweight model to a package recognition terminal device, and recognizing package information in the real-time collected pictures.

2. The package identification method based on the pruning lightweight model according to claim 1, wherein the adjacency matrix is a real symmetric matrix, the off-diagonal elements of the adjacency matrix are edge weights between two corresponding vertices, and the diagonal elements are 0; the degree matrix is a diagonal matrix with each row of diagonal elements being the sum of all elements of the corresponding row in the adjacency matrix.

3. The package recognition method based on the pruning lightweight model according to claim 1, wherein the calculating the importance of the channel of each convolution layer according to the variation value of the von Neumann map entropy in each convolution layer comprises:

and calculating the average value of the von Neumann diagram entropy change values of each channel according to the von Neumann diagram entropy change values of the channels of the convolutional layers obtained by each training picture, wherein the average value is used as the importance of each channel.

4. The package identification method based on the pruning lightweight model according to claim 3, wherein the pruning of the channels of each convolutional layer according to the pruning rate of each convolutional layer and the importance of the channels is performed from small to large according to the pruning rate of each convolutional layer and the importance of the channels of each convolutional layer.

5. The package identification method based on the pruning lightweight model according to claim 1 or 4, wherein the neural network model to be pruned is a neural network model composed of convolutional layers having a CONV-BN-ReLU structure.

6. A parcel recognition system based on a pruning lightweight model is characterized by comprising:

the characteristic diagram matrix extraction module is used for inputting the training picture into the pre-trained neural network model to be pruned and extracting the characteristic diagram matrix of each convolution layer; the training pictures are obtained by randomly extracting the training pictures from the package picture data set;

the von Neumann map entropy calculation module is used for converting the characteristic map matrix of each convolution layer into a weighted undirected graph, constructing an improved Laplace matrix according to the amplitude matrix of the characteristic map matrix, and calculating the von Neumann map entropy as an original value of each convolution layer, and comprises the following steps:

deforming each three-dimensional characteristic diagram in the characteristic diagram matrix of each convolution layer to obtain a characteristic row vector matrix, wherein each row is a characteristic row vector corresponding to each channel and is respectively used as the vertex of a weighted undirected graph of each convolution layer;

calculating the cosine distance of any two characteristic row vectors in the characteristic row vector matrix of each convolution layer, and taking the cosine distance as the edge weight between two corresponding vertexes in the weighted undirected graph of each convolution layer;

acquiring an adjacency matrix and a degree matrix according to the weighted undirected graph;

obtaining a magnitude matrix according to the feature map matrix, wherein the magnitude matrix is a diagonal matrix, and each diagonal element is the sum of squares of all elements in a corresponding channel in the feature map matrix;

multiplying the degree matrix by the amplitude matrix, and then subtracting the adjacent matrix to obtain an improved Laplace matrix;

the von Neumann map entropy is calculated by the following formula:

wherein,H _i is shown asiVon Neumann entropy of each convolutional layer,L _d representing the improved Laplace matrix; trace (·) represents the trace of the matrix, i.e., the sum of all eigenvalues of the matrix;λ _k representing an improved Laplace matrixL _d To middlekThe value of the characteristic is used as the characteristic value,λ _k not less than 0, and

，n _i is shown asiThe number of channels of each convolutional layer;

sequentially deleting single vertexes in the weighted undirected graph of each convolution layer to obtain a new weighted undirected graph, and calculating the variation value of the von Neumann graph entropy of each new weighted undirected graph relative to the original value, wherein the variation value comprises the following steps:

sequentially deleting the characteristic row vectors corresponding to a single vertex aiming at the weighted undirected graph of each convolution layer, reconverting the characteristic row vector matrix obtained after deletion each time into a new weighted undirected graph, constructing a new improved Laplace matrix, and calculating a new Von Neumann graph entropy;

calculating the absolute value of the difference value between each new von Neumann diagram entropy and the original value of the corresponding convolution layer, and taking the absolute value as the variation value of the von Neumann diagram entropy of the channel corresponding to the deleted vertex;

the channel pruning module is used for calculating the importance of the channel of each convolutional layer according to the variation value of the von Neumann diagram entropy in each convolutional layer, pruning the channel of each convolutional layer according to the pruning rate of each convolutional layer and the importance of the channel, training the neural network model after pruning and obtaining a pruning lightweight model;

and the package identification module is used for deploying the pruning lightweight model to package identification terminal equipment and identifying package information in the real-time acquired pictures.

Images