CN113963220A - Security check image classification model training method, security check image classification method and device - Google Patents
Security check image classification model training method, security check image classification method and device Download PDFInfo
- Publication number
- CN113963220A CN113963220A CN202111575378.0A CN202111575378A CN113963220A CN 113963220 A CN113963220 A CN 113963220A CN 202111575378 A CN202111575378 A CN 202111575378A CN 113963220 A CN113963220 A CN 113963220A
- Authority
- CN
- China
- Prior art keywords
- network structure
- security inspection
- feature map
- image
- security
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The application discloses a security inspection image classification model training method, a security inspection image classification method and a security inspection image classification device, wherein the security inspection image classification model comprises a first network structure and a second network structure; in the training process of the security inspection image classification model, a security inspection training image with a label is obtained; extracting a feature map by using a first network structure, and predicting a first classification result based on the extracted feature map; calculating a first loss of the first network structure based on the first classification result and the label; carrying out shape feature extraction on the feature graph by using a second network structure to obtain a shape feature graph, and predicting a second classification result based on the shape feature graph; calculating a second loss of the second network fabric based on the second classification result and the label; and calculating the total loss of the security check image classification model based on the first loss and the second loss, and adjusting the network parameters of the security check image classification model based on the total loss until a set training end condition is reached to obtain the trained security check image classification model.
Description
Technical Field
The application relates to the field of classification, in particular to a security inspection image classification model training method, a security inspection image classification method and a security inspection image classification device.
Background
More and more people select subway and high-speed railway as the mode of going out, and no matter select high-speed railway or subway, people all need carry out the safety inspection before the station entry to guarantee people's trip safety. In addition, the convenience of online shopping leads to the increasing willingness of people to online shopping, the number of packages is increased rapidly, and security check is also needed in the transportation process of the packages to ensure social security. Therefore, security inspection technology has become a focus of attention. In particular, there are typically multiple items in baggage and parcels, based on which there is a tendency for overlap between the individual item images in the security images captured by the security inspection machine, i.e., there is a tendency for the hazardous item images and the general item images to overlap in the security images, which presents a significant challenge to security inspection techniques.
In order to solve the above problem, a classification model capable of distinguishing the categories of the overlapped articles in the security inspection image may be introduced to classify the security inspection image. Therefore, a training method of the classification model is urgently needed for training on a security inspection image training data set, the accuracy of the neural network can be improved, the classification of each overlapped article category in the security inspection process can be guaranteed, and the classification of the security inspection image can be carried out based on the classification.
Disclosure of Invention
In view of the above, the present application provides a security inspection image classification model training method, a security inspection image classification method and a security inspection image classification device, which are used for performing training on a security inspection image training data set to ensure that each overlapped article category can be distinguished in a security inspection process, and performing security inspection image classification based on the above.
In order to achieve the above object, the following solutions are proposed:
a security inspection image classification model training method comprises the steps that a security inspection image classification model comprises a first network structure and a second network structure;
the training process of the security inspection image classification model comprises the following steps:
acquiring a security inspection training image labeled with a classification result label;
extracting a feature map of the security inspection training image by using the first network structure, and predicting a first classification result of the security inspection training image based on the extracted feature map;
calculating a first loss of a first network structure based on the first classification result and the classification result label labeled by the security inspection training image;
carrying out shape feature extraction on the feature map extracted by the first network structure by using the second network structure to obtain a shape feature map, and predicting a second classification result of the security inspection training image based on the shape feature map;
calculating a second loss of a second network structure based on the second classification result and the classification result label labeled by the security inspection training image;
and calculating the total loss of the security check image classification model based on the first loss and the second loss, and adjusting the network parameters of the security check image classification model based on the total loss until a set training end condition is reached to obtain the trained security check image classification model.
Preferably, the second network structure comprises a shape feature extraction layer;
and performing shape feature extraction on the feature map extracted by the first network structure by using the second network structure, wherein the shape feature extraction comprises the following steps:
and the shape feature extraction layer is used for extracting shape features of the feature graph extracted by the first network structure, wherein the shape feature extraction layer is used for extracting the shape features of the feature graph extracted by the first network structure by using an edge detection operator.
Preferably, the edge detection operator comprises a Sobel operator;
and utilizing an edge detection operator to extract shape features of the feature graph extracted by the first network structure, wherein the shape feature extraction comprises the following steps:
utilizing a Sobel operator to extract the shape characteristics of the characteristic diagram extracted from the first network structure in the horizontal direction to obtain a horizontal shape characteristic diagram;
utilizing a Sobel operator to extract the shape features of the feature graph extracted from the first network structure in the vertical direction to obtain a vertical shape feature graph;
and synthesizing the horizontal shape characteristic diagram and the vertical shape characteristic diagram to obtain a final shape characteristic diagram.
Preferably, the first network structure comprises a convolutional neural network comprising a plurality of layers of feature extraction layers;
extracting a feature map of the security inspection training image by using the first network structure, and predicting a first classification result of the security inspection training image based on the extracted feature map, wherein the method comprises the following steps:
and inputting the security check training image into a convolutional neural network to obtain a feature map output by each layer of the convolutional neural network, and predicting a first classification result of the security check training image based on the feature map output by the last layer of feature extraction layer of the convolutional neural network.
Preferably, the shape feature extraction of the feature map extracted by the first network structure by using the second network structure includes:
selecting a feature map output by any feature extraction layer in the convolutional neural network as a target feature map;
and utilizing the second network structure to extract the shape feature of the target feature graph.
Preferably, the convolutional neural network is a residual network, wherein a pooling layer of the residual network comprises a fuzzy filter, and the residual network comprises a plurality of convolutional layers;
selecting a feature map output by any feature extraction layer in the convolutional neural network as a target feature map, wherein the selecting step comprises the following steps:
and selecting a feature map output by any convolutional layer from the first N layers of the residual error network as a target feature map, wherein N is half of the number of all convolutional layers in the residual error network.
Preferably, the blurring filter comprises any one of: low pass filters and median filters.
An image classification method, comprising:
acquiring a security inspection image to be identified;
and classifying the security check images by using the security check image classification model obtained by the training of the security check image classification model training method to obtain a classification result.
Preferably, the classifying the security inspection image by using the security inspection image classification model trained by the security inspection image classification model training method to obtain a classification result includes:
classifying the security check image by using a first network structure in the security check image classification model to obtain a first identification result output by the first network structure, and taking the first identification result as a final classification result;
or the like, or, alternatively,
classifying the security check image by using a first network structure in the security check image classification model to obtain a first identification result output by the first network structure;
classifying the security check image by using a second network structure in the security check image classification model to obtain a second identification result output by the second network structure;
and obtaining a final classification result based on the first recognition result and the second recognition result.
A security inspection image classification apparatus comprising:
the image acquisition unit is used for acquiring a security check image to be identified;
the image classification unit is used for classifying the security inspection images by utilizing a pre-trained security inspection image classification model to obtain a classification result; wherein the security inspection image classification model is configured to include a first network structure and a second network structure; the method comprises the steps of extracting a feature map of an input security check image through a first network structure, predicting a first classification result of the security check image based on the extracted feature map, extracting shape features of the feature map extracted by the first network structure through a second network structure to obtain a shape feature map, and predicting a second classification result of the security check image based on the shape feature map.
According to the technical scheme, the trained security inspection image classification model comprises a first network structure and a second network structure, and in the training process, the security inspection training image labeled with the classification result label can be obtained, wherein the classification result can be the name of each dangerous goods forbidden to be carried by security inspection; then, the first network structure may be used to extract a feature map of the security inspection training image, and predict a first classification result of the security inspection training image based on the extracted feature map, at this time, the first network structure may perform feature extraction and confirm the classification result of the security inspection training image; then, calculating a first loss of the first network structure based on the first classification result and the classification result label labeled by the security inspection training image, wherein at the moment, the first loss between the classification result of the first network structure and the predicted classification result can be confirmed; then, shape feature extraction can be carried out on the feature map extracted by the first network structure by using the second network structure to obtain a shape feature map, and a second classification result of the security inspection training image is predicted based on the shape feature map; at this time, the second network structure may extract the shape feature and confirm the classification result of the security check training image according to the extracted shape feature, and then, may calculate a second loss of the second network structure based on the second classification result and the classification result label labeled by the security check training image, where the second loss is closely related to the shape feature and is determined based on the shape feature; finally, the total loss of the security inspection image classification model can be calculated based on the first loss and the second loss, the network parameters of the security inspection image classification model are adjusted based on the total loss until a set training end condition is reached, the trained security inspection image classification model is obtained, based on the network parameters, all parameter values of the security inspection image classification model are determined based on the first loss and the second loss, and since the second loss is closely related to the shape feature, the obtained security inspection image classification model with the modified parameters can pay more attention to the shape feature when image classification is carried out. According to the method, the contribution of the shape, texture and color features of the article to the distinguishing of the category of the article is different, and the article shape is easier to distinguish the category of the article, so that the image classification is performed based on the category of the article. Therefore, the object identification is performed based on the shape feature, so that the resolution and accuracy of the dangerous object can be improved, and the image classification can be performed more easily based on the improved image identification. Therefore, the security inspection image classification model trained by the security inspection image classification model training method can improve the accuracy of neural network classification, can distinguish the category of an article more easily, and can classify the image.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flowchart of a security inspection image classification model training method disclosed in the present application;
FIG. 2 is a block diagram of a security image classification model according to an example of the present application;
FIG. 3 is a flowchart of a security inspection image classification method disclosed in the present application;
fig. 4 is a block diagram of a security inspection image classification apparatus disclosed in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
During the security check, the images of the objects in the acquired baggage image are easy to overlap, which brings great challenges to the security check technology. In order to solve the problem, the application tries to introduce a classification model capable of distinguishing each overlapped article category in the security inspection image to classify the security inspection image. In order to obtain the classification model, the application provides a security inspection image classification model training method which is used for training on a security inspection image training data set so as to ensure that each overlapped article category can be distinguished in the security inspection process, and the security inspection image classification is carried out according to the overlapped article categories.
Next, a detailed description will be given of a security image classification model training method according to the present application with reference to fig. 1 and fig. 2, where the security image classification model includes a first network structure and a second network structure, for example, as shown in fig. 2, a security image classification model 3 includes a first network structure 1 and a second network structure 2, where both the first network structure 1 and the second network structure 2 can output classification results.
Based on this, the training process of the security inspection image classification model is shown in fig. 1, and includes the following steps:
and S110, acquiring a security inspection training image labeled with a classification result label.
Specifically, the classification result label may be a classification of security contraband, such as battery class, digital class, living goods class, container class, lighter class, knife class, life tool class, gun class.
In order to obtain the security inspection training image, an image only containing a single-class dangerous article can be obtained by selecting or cutting from the security inspection images on the website, corresponding classification results are marked in the obtained images, and based on the classification results, the security inspection training image with the label marked on the classification results can be obtained and used as a training sample set.
And S120, extracting the feature map of the security inspection training image by using the first network structure, and predicting a first classification result of the security inspection training image based on the extracted feature map.
Specifically, the first network structure may extract features of the security inspection training images and output a feature map. The characteristic can comprise the color, the shape and the texture of the article image in the security inspection training image, so that the characteristic map can also comprise the color, the shape and the texture of each article image in the security inspection training image, whether security inspection contraband articles are contained or not is determined based on the color, the shape and the texture of each article image, if the security inspection contraband articles are contained, the type of the contraband articles is determined, and the type of the contraband articles is taken as a first classification result.
Example fig. 2, the feature map extraction layer 4 in the first network configuration 1 may be configured to output a feature map, which may be of size [256, 256, 56, 56 ].
Step S130, calculating a first loss of a first network structure based on the first classification result and the classification result label labeled by the security inspection training image.
Specifically, after the first classification result is obtained, the first loss of the first network structure may be calculated based on the first classification result and the classification result label labeled by the security inspection training image.
Wherein the first loss of the first network structure may be calculated using a cross-entropy loss function.
For example, a batch data sample may be provided as an input sample of the security image classification model, and input into the first network structure. With xiAs a security check training image, with yiWhen m is used as the batch size as the classification result label corresponding to the sample, where the batch size may be 256, the following formula may be obtained:
at this time, E may be used as the cross entropy loss function, x is the feature map input to the first network structure, f (x) represents the function corresponding to the first network structure, and y represents the classification result of the first network structure, so that the first loss corresponding to the first network structure may be obtained as the following formula:
and S140, extracting shape features of the feature graph by using the second network structure to obtain a shape feature graph, and predicting a second classification result of the security inspection training image based on the shape feature graph.
Specifically, the feature map refers to the feature map extracted by the first network structure in step S130.
The second network structure can extract the shape feature of the feature map, so as to obtain a shape feature map, and the feature map can include the color, shape and texture of each article image in the security inspection training image. Therefore, when predicting the second classification result of the security inspection training image based on the shape feature map, the influence of color and texture is eliminated, and only the shape of each article image is referred to.
Example fig. 2, the feature map output by the feature map extraction layer 4 in the first network configuration 1 may be used as an input to the second network configuration. Specifically, the feature map output by the feature map extraction layer 4 may be acquired and input into the input layer 5 of the second network structure 2 to perform shape feature extraction.
And S150, calculating a second loss of a second network structure based on the second classification result and the classification result label labeled by the security inspection training image.
Specifically, after the second classification result is obtained, the second loss of the second network structure may be calculated based on the second classification result and the classification result label labeled by the security inspection training image. It can be seen that, when determining the loss, the classification result labels referred to by the first classification result and the second classification result are consistent, that is, if the first classification result and the second classification result are consistent, the first loss and the second loss are consistent.
Wherein the second loss of the second network structure may be calculated using a cross-entropy loss function.
For example, E may be used as a cross entropy loss function, where x 'is a feature map input to the second network structure, s (x') represents a function corresponding to the second network structure, and y represents a classification result of the second network structure, so that a second loss corresponding to the second network structure may be obtained as the following formula:
and S160, calculating the total loss of the security check image classification model based on the first loss and the second loss, and adjusting the network parameters of the security check image classification model based on the total loss until a set training end condition is reached to obtain the trained security check image classification model.
Specifically, the first loss and the second loss may be added to obtain a total loss of the security image classification model, and based on this, a network parameter of the security image classification model may be adjusted.
Based on this, the security inspection image classification model after parameter adjustment can be obtained, and step S110 can be repeatedly executed until the total loss is lower than the preset threshold value, at this time, the obtained security inspection image classification model is used as the trained security inspection image classification model.
For example, adding the first loss and the second loss may result in a total loss as follows:
substituting the equations corresponding to the first loss and the second loss in S130 and S150 in the above steps into the equation herein, the following equation can be obtained:
according to the technical scheme, the security inspection image classification model training method provided by the embodiment of the application comprises the steps that the trained security inspection image classification model comprises a first network structure and a second network structure, and in the training process, a security inspection training image labeled with a classification result label can be obtained, wherein the classification result can be the name of each dangerous goods forbidden to be carried by security inspection; then, the first network structure may be used to extract a feature map of the security inspection training image, and predict a first classification result of the security inspection training image based on the extracted feature map, at this time, the first network structure may perform feature extraction and confirm the classification result of the security inspection training image; then, calculating a first loss of the first network structure based on the first classification result and the classification result label labeled by the security inspection training image, wherein at the moment, the first loss between the classification result of the first network structure and the predicted classification result can be confirmed; then, shape feature extraction can be carried out on the feature map extracted by the first network structure by using the second network structure to obtain a shape feature map, and a second classification result of the security inspection training image is predicted based on the shape feature map;
at this time, the second network structure may extract the shape feature and confirm the classification result of the security check training image according to the extracted shape feature, and then, may calculate a second loss of the second network structure based on the second classification result and the classification result label labeled by the security check training image, where the second loss is closely related to the shape feature and is determined based on the shape feature; finally, the total loss of the security inspection image classification model can be calculated based on the first loss and the second loss, the network parameters of the security inspection image classification model are adjusted based on the total loss until a set training end condition is reached, the trained security inspection image classification model is obtained, based on the network parameters, all parameter values of the security inspection image classification model are determined based on the first loss and the second loss, and since the second loss is closely related to the shape feature, the obtained security inspection image classification model with the modified parameters can pay more attention to the shape feature when performing image classification, and can be used for assisting in correcting the propagation gradient change corresponding to the first network structure. According to the method, the contribution of the shape, texture and color features of the article to the distinguishing of the category of the article is different, and the article shape is easier to distinguish the category of the article, so that the image classification is performed based on the category of the article. Therefore, the resolution of the dangerous goods can be improved by performing the article recognition based on the shape feature, and the image classification can be performed more easily based on this. Therefore, the security inspection image classification model obtained by the training method of the security inspection image classification model can be used for distinguishing the categories of articles more easily and classifying the images.
In some embodiments of the present application, the second network structure may include a shape feature extraction layer, and then the process of extracting the shape feature of the feature map by using the second network structure in step S140 may be described, that is, the shape feature extraction layer may be used to extract the shape feature of the feature map extracted by using the first network structure.
Specifically, the shape feature extraction layer may be configured to perform shape feature extraction on the feature map extracted by the first network structure by using an edge detection operator.
The edge detection operator can be a Sobel operator, a Prewitt operator, a Roberts operator, a Canny operator, and the like.
As can be seen from the foregoing technical solutions, the present embodiment provides a way to extract a shape feature map, that is, the second network structure may include a shape feature extraction layer, and the shape feature extraction layer performs shape feature extraction by using an edge detection operator. Therefore, compared with the previous embodiment, the shape feature can be better extracted to obtain the shape feature diagram, so that the second classification result is more accurate, and the second loss is reduced.
In some embodiments of the present application, the edge detection operator may be determined as a Sobel operator, and based on this, a process of performing shape feature extraction on the feature map by using the second network structure in step S140 is described, where the steps are as follows:
and S10, utilizing a Sobel operator to extract the horizontal shape feature of the feature graph extracted by the first network structure to obtain a horizontal shape feature graph.
Specifically, the Sobel operator determines the edge information by performing convolution on an input image and performing threshold operation.
The Sobel operator has operators in the vertical direction and the horizontal direction, and convolution operation can be performed by using the operators in the horizontal direction in the Sobel operator, that is, shape feature extraction in the horizontal direction is performed, so that a horizontal shape feature map is obtained.
Operator L in horizontal direction in Sobel operatorxAs shown in table 1 below:
TABLE 1
Then, with SxA horizontal shape feature diagram, denoted by LxIndicating the operator in the horizontal direction in the Sobel operator, and indicating the feature map input to the second network structure with G, the extraction process of the horizontal shape feature map can be indicated by the following formula:
and S11, utilizing a Sobel operator to extract the shape features of the feature graph extracted by the first network structure in the vertical direction to obtain a vertical shape feature graph.
Specifically, the Sobel operator has operators in the vertical direction and the horizontal direction, and convolution operation can be performed by using the operators in the vertical direction in the Sobel operator, that is, shape feature extraction in the vertical direction is performed, so that a vertical shape feature map is obtained.
Wherein, the operator L in the vertical direction in the Sobel operatoryAs shown in table 2 below:
TABLE 2
Then, with SyShowing a vertical shape feature map, in LyIndicating the operator in the vertical direction in the Sobel operator, and indicating the feature map input to the second network structure with G, the extraction process of the vertical shape feature map can be indicated by the following formula:
and S12, synthesizing the horizontal shape characteristic diagram and the vertical shape characteristic diagram to obtain a final shape characteristic diagram.
Specifically, can be SxRepresenting a horizontal shape feature map, denoted by SyRepresenting a vertical shape feature map, S can be obtainedxAnd SyAnd root cutting is carried out on the square sum, so as to obtain a final shape feature map, wherein the shape feature map can be a black-white map. That is, the formula can be as follows:
it can be seen from the foregoing technical solutions that this embodiment provides an optional way for extracting the shape feature map from the second network structure, and specifically, a Sobel operator may be used to perform convolution operation, so as to obtain the shape feature map. Therefore, the shape feature map can be better acquired through the steps.
In some embodiments of the present application, the first network structure may include a convolutional neural network, and the convolutional neural network may include a plurality of feature extraction layers, and on this basis, the process of extracting the feature map of the security inspection training image by using the first network structure and predicting the first classification result of the security inspection training image based on the extracted feature map in step S120 is described in detail.
Specifically, the security inspection training image may be input into a convolutional neural network to obtain feature maps output by each layer of the convolutional neural network, and a first classification result of the security inspection training image may be predicted based on the feature maps output by the last feature extraction layer of the convolutional neural network.
Each feature extraction layer may output an extracted feature map, for example, as shown in fig. 2, and each feature extraction layer may obtain a feature map output by a previous layer, perform feature extraction based on the feature map output by the previous layer, and output a corresponding feature map.
It can be seen from the foregoing technical solutions that, compared with the previous embodiment, the present embodiment provides an optional first network structure, specifically, the first network structure may be a convolutional neural network, and the convolutional neural network includes multiple feature extraction layers. Therefore, the feature extraction can be well performed by the embodiment, and the first classification result is obtained.
In some embodiments of the present application, the first network structure includes a convolutional neural network, which may include a plurality of feature extraction layers, and on this basis, the process of performing shape feature extraction on the feature map by using the second network structure in step S140 is described as follows:
and S20, selecting the feature graph output by any one feature extraction layer in the convolutional neural network as a target feature graph.
Specifically, a feature map output by any feature extraction layer in the convolutional neural network can be selected as a target feature map according to the needs of an actual application scenario. In particular, a feature map output by a low level in the first network structure, for example, as shown in fig. 2, may be selected as the target feature map, and the feature map output by the first layer feature extraction layer in the convolutional neural network may be selected.
And S21, extracting shape features of the target feature graph by using the second network structure.
Specifically, the target feature map is input into a second network structure, so that the second network structure performs shape feature extraction on the target feature map.
As can be seen from the foregoing technical solutions, compared with the previous embodiment, this embodiment provides an alternative way of extracting shape features from the feature map by using the second network structure, and specifically, any feature map may be selected for extracting shape features. Therefore, the shape feature extraction can be better carried out by the embodiment.
In some embodiments of the present application, a residual network may also be used as the first network structure, wherein a pooling layer of the residual network may include a blur filter, and the residual network includes multiple convolutional layers, and on this basis, a process of selecting a feature map output by any feature extraction layer in the convolutional neural network as a target feature map in step S20 is described in detail.
Specifically, a feature map output by any convolutional layer may be selected as a target feature map in the first N layers of the residual error network.
In order to reduce the influence of the shape feature extraction of the first network structure on the shape feature extraction of the second network structure as much as possible, a feature map output by a low-level layer in the residual error network may be selected, and based on this, N may be 1/4, 1/3, 1/2, and the like of the number of all convolutional layers in the residual error network.
For example, when the number of all convolutional layers in the residual error network is 15, if N is 1/4 of the number of all convolutional layers in the residual error network, and at this time, N is 3.75, then one layer of the output feature map in the first 3 layers of convolutional layers in the residual error network can be arbitrarily selected and input into the second network structure; if N is 1/3 of the number of all convolutional layers in the residual error network, and at this time, N is 5, one layer of the output feature map can be arbitrarily selected from the first 5 convolutional layers of the residual error network and input into the second network structure; if N is 1/2 of the number of all the convolution layers in the residual error network, at this time, N is 7.5, then one layer of the output feature graph can be arbitrarily selected from the first 7 layers of the convolution layers of the residual error network and input into the second network structure; for example, a feature map output by a first layer convolutional layer in a residual network can be selected for input into a second network structure.
The residual network may be a standard structure function of the ResNet50, and then the security inspection training image may be input into the second network structure after being subjected to convolution, batch normalization, ReLU (Rectified Linear Unit), and max pooling.
Wherein, during the maximum pooling process, a blurring filter may be added.
It can be seen from the foregoing technical solutions that the present embodiment provides an optional first network structure, that is, the first network structure may be a residual error network, and provides an optional manner for selecting a feature map and inputting the feature map into a second network structure when the first network structure is the residual error network. Therefore, the shape feature extraction can be better carried out by the embodiment.
Further, the blur filter in the present application may be a low pass filter or a median filter.
Specifically, a filter according with the application of the scene can be selected according to the requirement of the actual application scene.
In some embodiments of the present application, it is contemplated that the training of the security image classification model is performed for use during the actual security screening process. Therefore, a using process using the security inspection image classification model can be provided, and the specific steps are as follows:
and S30, acquiring a security check image to be identified.
Specifically, an image of the baggage in the security check machine may be acquired, thereby obtaining a security check image to be identified.
And S31, classifying the security inspection images by using the security inspection image classification model obtained by training in the embodiment to obtain a classification result.
Specifically, the security inspection image classification model obtained by training in any one of the above embodiments is selected to classify the security inspection image, so as to obtain a classification result.
It can be seen from the above technical solutions that the present embodiment provides an alternative way of using a security inspection training model to perform image classification. Therefore, the embodiment can perform security check image classification.
In some embodiments of the present application, on the basis that there may be a first network structure and a second network structure in the security inspection training model trained by the foregoing embodiments, a detailed description of a process using the security inspection image classification model may be provided with reference to fig. 3, where the specific steps are as follows:
s310, acquiring a security inspection image to be identified, and performing step S320 or step S330.
Specifically, step S320 or step S330 may be executed according to the requirement of the actual application scenario.
S320, classifying the security inspection image by using the first network structure in the security inspection image classification model to obtain a first identification result output by the first network structure, and taking the first identification result as a final classification result.
Specifically, the security inspection image to be identified may be input into the security inspection image classification model, with the result output by the first network structure as the classification result.
S330, classifying the security check image by using a first network structure in the security check image classification model to obtain a first identification result output by the first network structure.
Specifically, the security inspection image to be identified may be input into the security inspection image classification model, and a result output by the first network structure is obtained as a first identification result, where the first identification result indicates a classification result of the security inspection image.
S340, classifying the security inspection image by using a second network structure in the security inspection image classification model to obtain a second identification result output by the second network structure.
Specifically, the security inspection image to be identified may be input into the security inspection image classification model, and a result output by the second network structure is obtained as a second identification result, where the second identification result indicates a classification result of the security inspection image.
And S350, obtaining a final classification result based on the first identification result and the second identification result.
In this case, the final classification result may be confirmed based on the first recognition result and the second recognition result, for example, if the first recognition result is the same as the second recognition result, the common classification result is used as the final classification result.
It can be seen from the above technical solutions that, compared with the above embodiments, the present embodiment provides two alternative ways of using the security inspection image classification model to perform image classification. Therefore, the embodiment can better classify the security check images.
And then, through a comparison experiment, the classification effect of the security inspection image classification model on the security inspection contraband images is verified. The experiment adopts 8 contraband classifications including battery class, digital class, living goods class, container class, lighter class, cutter class, life tool class, gun class. The security inspection image to be identified is a single-class image cut from the security inspection image, and 5000 pieces of image data are adopted in the test set. By implementing the existing network model, comparative experiments were performed on the same test set.
The anti-terminated-cnns is a model based on the ResNet50 network implementation and is compared by taking the model as a reference model. Is a model. The anti-Par is an existing network model, the anti-InfoDrap is a network model realized based on a reference model, and the anti-San is a model obtained by training through the security inspection image classification model training method.
Next, the performance of the four models will be confirmed by comparing the accuracy of the four models to the training set as shown in Table 3, the recall ratio as shown in Table 4, F1-score as shown in Table 5, and the AP value as shown in Table 6.
TABLE 3
Through table 3, it can be found that the security inspection image classification model of the application has higher accuracy than other three models in identifying the security inspection images containing digital images, living objects, lighters, cutters, tools and guns.
TABLE 4
Through table 4, it can be found that the security inspection image classification model of the present application has a higher recall rate for security inspection images containing batteries, digital products, living things, lighters, knives and guns.
TABLE 5
As can be seen from Table 5, the security inspection image classification model of the present application has a higher F1-score in identifying security inspection images containing digital codes, living things, lighters, tools and guns.
TABLE 6
Through table 6, it can be found that the security inspection image classification model of the application has a higher AP value in identifying security inspection images containing batteries, digital products, living things, lighters, cutters and guns.
From the results of the above comparison experiments, it can be seen that, for most categories of security inspection contraband, the security inspection image classification model training method of the present application plays a very positive role in improving the classification accuracy and recall rate of the models, and although the performance of a few categories is not optimal, the performance is very close to the highest value, as can be seen from the comparison results in table 5, the comprehensive performance of the security inspection image classification model of the present application is outstanding in a plurality of categories, but the categories of life tools cannot achieve the expected effect, and the analysis shows that the categories have many internal tools and different shapes, and the number of each tool is not sufficient, so the security inspection image classification model of the present application does not achieve the expected effect for the categories of life tools temporarily, and can be deeply studied for specific categories later.
Next, the security image classification apparatus provided in the present application will be described with reference to fig. 4, and the security image classification apparatus of the present embodiment and the security image classification method described above may be referred to each other.
An image acquisition unit 100 for acquiring a security inspection image to be identified;
the image classification unit 110 is configured to classify the security inspection image by using a pre-trained security inspection image classification model to obtain a classification result; wherein the security inspection image classification model is configured to include a first network structure and a second network structure; the method comprises the steps of extracting a feature map of an input security check image through a first network structure, predicting a first classification result of the security check image based on the extracted feature map, extracting shape features of the feature map extracted by the first network structure through a second network structure to obtain a shape feature map, and predicting a second classification result of the security check image based on the shape feature map.
Further, the image classification unit is configured to classify the security inspection image by using a pre-trained security inspection image classification model, and a process of obtaining a classification result may include:
classifying the security check image by using a first network structure in the security check image classification model to obtain a first identification result output by the first network structure, and taking the first identification result as a final classification result;
or the like, or, alternatively,
classifying the security check image by using a first network structure in the security check image classification model to obtain a first identification result output by the first network structure;
classifying the security check image by using a second network structure in the security check image classification model to obtain a second identification result output by the second network structure;
and obtaining a final classification result based on the first recognition result and the second recognition result.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. The various embodiments of the present application may be combined with each other. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A security inspection image classification model training method is characterized in that a security inspection image classification model comprises a first network structure and a second network structure;
the training process of the security inspection image classification model comprises the following steps:
acquiring a security inspection training image labeled with a classification result label;
extracting a feature map of the security inspection training image by using the first network structure, and predicting a first classification result of the security inspection training image based on the extracted feature map;
calculating a first loss of a first network structure based on the first classification result and the classification result label labeled by the security inspection training image;
carrying out shape feature extraction on the feature map extracted by the first network structure by using the second network structure to obtain a shape feature map, and predicting a second classification result of the security inspection training image based on the shape feature map;
calculating a second loss of a second network structure based on the second classification result and the classification result label labeled by the security inspection training image;
and calculating the total loss of the security check image classification model based on the first loss and the second loss, and adjusting the network parameters of the security check image classification model based on the total loss until a set training end condition is reached to obtain the trained security check image classification model.
2. The method of claim 1, wherein the second network structure comprises a shape feature extraction layer;
and performing shape feature extraction on the feature map extracted by the first network structure by using the second network structure, wherein the shape feature extraction comprises the following steps:
and the shape feature extraction layer is used for extracting shape features of the feature graph extracted by the first network structure, wherein the shape feature extraction layer is used for extracting the shape features of the feature graph extracted by the first network structure by using an edge detection operator.
3. The method of claim 2, wherein the edge detection operator comprises a Sobel operator;
and utilizing an edge detection operator to extract shape features of the feature graph extracted by the first network structure, wherein the shape feature extraction comprises the following steps:
utilizing a Sobel operator to extract the shape characteristics of the characteristic diagram extracted from the first network structure in the horizontal direction to obtain a horizontal shape characteristic diagram;
utilizing a Sobel operator to extract the shape features of the feature graph extracted from the first network structure in the vertical direction to obtain a vertical shape feature graph;
and synthesizing the horizontal shape characteristic diagram and the vertical shape characteristic diagram to obtain a final shape characteristic diagram.
4. The method of claim 1, wherein the first network structure comprises a convolutional neural network comprising a plurality of layers of feature extraction layers;
extracting a feature map of the security inspection training image by using the first network structure, and predicting a first classification result of the security inspection training image based on the extracted feature map, wherein the method comprises the following steps:
and inputting the security check training image into a convolutional neural network to obtain a feature map output by each layer of the convolutional neural network, and predicting a first classification result of the security check training image based on the feature map output by the last layer of feature extraction layer of the convolutional neural network.
5. The method of claim 4, wherein performing shape feature extraction on the feature map extracted by the first network structure by using the second network structure comprises:
selecting a feature map output by any feature extraction layer in the convolutional neural network as a target feature map;
and utilizing the second network structure to extract the shape feature of the target feature graph.
6. The method of claim 5, wherein the convolutional neural network is a residual network, wherein a pooling layer of the residual network includes a blurring filter, and wherein the residual network includes a plurality of convolutional layers;
selecting a feature map output by any feature extraction layer in the convolutional neural network as a target feature map, wherein the selecting step comprises the following steps:
and selecting a feature map output by any convolutional layer from the first N layers of the residual error network as a target feature map, wherein N is half of the number of all convolutional layers in the residual error network.
7. The method of claim 6, wherein the blur filter comprises any one of: low pass filters and median filters.
8. A security inspection image classification method is characterized by comprising the following steps:
acquiring a security inspection image to be identified;
classifying the security inspection images by using the security inspection image classification model trained by the security inspection image classification model training method of any one of claims 1 to 7 to obtain a classification result.
9. The method according to claim 8, wherein the step of classifying the security inspection image by using the security inspection image classification model trained by the security inspection image classification model training method according to any one of claims 1 to 7 to obtain a classification result comprises:
classifying the security check image by using a first network structure in the security check image classification model to obtain a first identification result output by the first network structure, and taking the first identification result as a final classification result;
or the like, or, alternatively,
classifying the security check image by using a first network structure in the security check image classification model to obtain a first identification result output by the first network structure;
classifying the security check image by using a second network structure in the security check image classification model to obtain a second identification result output by the second network structure;
and obtaining a final classification result based on the first recognition result and the second recognition result.
10. A security inspection image classification device is characterized by comprising:
the image acquisition unit is used for acquiring a security check image to be identified;
the image classification unit is used for classifying the security inspection images by utilizing a pre-trained security inspection image classification model to obtain a classification result; wherein the security inspection image classification model is configured to include a first network structure and a second network structure; the method comprises the steps of extracting a feature map of an input security check image through a first network structure, predicting a first classification result of the security check image based on the extracted feature map, extracting shape features of the feature map extracted by the first network structure through a second network structure to obtain a shape feature map, and predicting a second classification result of the security check image based on the shape feature map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111575378.0A CN113963220A (en) | 2021-12-22 | 2021-12-22 | Security check image classification model training method, security check image classification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111575378.0A CN113963220A (en) | 2021-12-22 | 2021-12-22 | Security check image classification model training method, security check image classification method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113963220A true CN113963220A (en) | 2022-01-21 |
Family
ID=79473618
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111575378.0A Pending CN113963220A (en) | 2021-12-22 | 2021-12-22 | Security check image classification model training method, security check image classification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113963220A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019085793A1 (en) * | 2017-11-01 | 2019-05-09 | 腾讯科技（深圳）有限公司 | Image classification method, computer device and computer readable storage medium |
| WO2019085757A1 (en) * | 2017-11-01 | 2019-05-09 | 腾讯科技（深圳）有限公司 | Image classifying method, server and computer readable storage medium |
| WO2020185198A1 (en) * | 2019-03-08 | 2020-09-17 | Google Llc | Noise tolerant ensemble rcnn for semi-supervised object detection |
| WO2020192471A1 (en) * | 2019-03-26 | 2020-10-01 | 腾讯科技（深圳）有限公司 | Image classification model training method, and image processing method and device |
| CN113435522A (en) * | 2021-06-30 | 2021-09-24 | 平安科技（深圳）有限公司 | Image classification method, device, equipment and storage medium |
| CN113780256A (en) * | 2021-11-12 | 2021-12-10 | 科大讯飞(苏州)科技有限公司 | Image target detection method combining thickness classification and related device |
-
2021
- 2021-12-22 CN CN202111575378.0A patent/CN113963220A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019085793A1 (en) * | 2017-11-01 | 2019-05-09 | 腾讯科技（深圳）有限公司 | Image classification method, computer device and computer readable storage medium |
| WO2019085757A1 (en) * | 2017-11-01 | 2019-05-09 | 腾讯科技（深圳）有限公司 | Image classifying method, server and computer readable storage medium |
| WO2020185198A1 (en) * | 2019-03-08 | 2020-09-17 | Google Llc | Noise tolerant ensemble rcnn for semi-supervised object detection |
| WO2020192471A1 (en) * | 2019-03-26 | 2020-10-01 | 腾讯科技（深圳）有限公司 | Image classification model training method, and image processing method and device |
| CN113435522A (en) * | 2021-06-30 | 2021-09-24 | 平安科技（深圳）有限公司 | Image classification method, device, equipment and storage medium |
| CN113780256A (en) * | 2021-11-12 | 2021-12-10 | 科大讯飞(苏州)科技有限公司 | Image target detection method combining thickness classification and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3008666B1 (en) | Image based object classification | |
| CN107092922B (en) | Vehicle damages recognition methods and server | |
| CN106599925A (en) | Plant leaf identification system and method based on deep learning | |
| CN110749934A (en) | Automatic matching remote centralized interpretation system and method for intelligent passage of travel inspection | |
| JP6061713B2 (en) | Inspection apparatus, inspection method and program | |
| US20030179931A1 (en) | Region-based image recognition method | |
| CN101901346B (en) | Method for identifying unsuitable content in colour digital image | |
| WO2005089966A1 (en) | Method and device for sorting mail by means of a plurality of sorting processes | |
| CN110428007B (en) | X-ray image target detection method, device and equipment | |
| WO2015072338A1 (en) | Data structure, library creation device, electronic device analysis device, and library provision system | |
| CN106203539B (en) | Method and device for identifying container number | |
| CN111428682B (en) | Express sorting method, device, equipment and storage medium | |
| CN108241821A (en) | Image processing equipment and method | |
| EP3644232B1 (en) | Method and device for classifying samples to be assessed | |
| US20090167850A1 (en) | Method for identifying guignardia citricarpa | |
| CN107679529A (en) | A kind of logistics parcel image processing method, apparatus and system | |
| CN106991363A (en) | A kind of method and apparatus of Face datection | |
| CN112805719A (en) | Classifying defects in semiconductor samples | |
| CN111539251A (en) | Security check article identification method and system based on deep learning | |
| WO2021201251A1 (en) | Method for analyzing composition of electronic/electrical apparatus component layer, method for processing electronic/electrical apparatus component layer, device for analyzing composition of electronic/electrical apparatus component layer, and device for processing electronic/electrical apparatus component layer | |
| CN110874835A (en) | Crop leaf disease resistance identification method and system, electronic equipment and storage medium | |
| CN113963220A (en) | Security check image classification model training method, security check image classification method and device | |
| CN211263806U (en) | Automatic matching remote centralized interpretation system for intelligent passage of travel inspection | |
| CN111950556A (en) | License plate printing quality detection method based on deep learning | |
| CN111797922B (en) | Text image classification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220121 |
|
| RJ01 | Rejection of invention patent application after publication |