CN111582367A - A method for small metal threat detection - Google Patents

Abstract

The invention relates to a detection method for small metal threats, which comprises: performing X-ray inspection on an inspected object to obtain an X-ray image; preprocessing the X-ray image to obtain a preprocessed X-ray image; Window for density sampling; use a classifier trained based on a preset convolutional neural network-based deep learning model to classify the detected object and calculate the confidence score of the detected image, in the case where the confidence is greater than a certain threshold The detection method can improve the detection performance of SMT, which has an order of magnitude improvement compared with the traditional neural network detection.

Description

A method for small metal threat detection

technical field

The invention relates to related applications of detecting threats in X-ray images by using a convolutional neural network, in particular to a method for detecting small metal threats.

Background technique

Nowadays, security testing is becoming more and more common in our daily life, and it has also attracted high attention from all walks of life. In important places such as airports, stations, subways, and various agencies and units around the world, security personnel use security infrastructure equipment to detect whether people are carrying guns, ammunition, flammable, explosive, toxic and radioactive and other dangerous items to ensure their personal safety. Property security, while preventing criminals or terrorist attacks, and maintaining the peace and stability of social order.

In some special scenarios, such as freight containers, the Small Metal Threat (SMT) currently relies mainly on statistical risk analysis, intelligence reports, and visual inspection of X-ray images by security personnel. These methods are very slow and unreliable in difficult tasks, usually the detected objects are small in size, and in very complex and cluttered backgrounds, it is necessary to detect possibly 50 pixel values in images containing more than 2 million pixel values The following objects, so these methods are inaccurate, and when detecting freight containers, the scene in the image is often larger and more complex, there are almost no restrictions on the arrangement of the goods, plus some dense shading, it is difficult for general security personnel to use Threats hidden in legitimate cargo are visible to the naked eye, so there is still the possibility of illegal smuggling or terrorist attacks.

Existing related technical solution one

Manual inspection of X-ray security images is a common security inspection method at present, and is often used in security inspections such as stations and airports. However, in the image of the freight container, the imaging size of small metal is small, for example, the typical value of the pixel in the image of 2600×850 pixels is only 0.1%, so during the inspection process, it is difficult for the security personnel to distinguish the image, and the inspection time is increased. . The disadvantage of the existing related technical solution 1 is: when facing a large cargo box such as a container with a length of more than 10 meters, if the cargo is placed in a complex or dense manner, and at the same time it is covered, it is difficult to distinguish small metal objects and cannot be found at all. Metal Threat in Eligible Cargo.

Existing related technical solution 2

People use feature extraction and clustering methods to detect small metals in container images, such as the BoW bag of words model, which is often used in computer vision, information retrieval and other fields. The bag-of-words model first extracts features from images through SIFT, SURF, etc., and then uses k-means clustering and other methods to construct a dictionary, and uses histogram statistics to classify images for new input data. The disadvantage of the existing related technical solution 2 is that: the method is developed for natural images, etc., and its transparency, noise level, confusion and oblique viewing angle are obviously different from X-ray images, and the detection is directly applied to X-ray images. not good.

Three existing related technical solutions:

Detection is based on X-ray cargo images of shape and texture. The scheme divides X-ray cargo images into 22 categories, such as tires, grains, etc., trained on the training set of this classification, 78% of the images can be correctly classified and identified . The disadvantage of the existing related technical solution 3 is: due to the unrestricted direction of the small metal and the inconspicuous characteristics of the features, the special feature texture cannot be extracted, and the recognition degree is not high. object.

At present, there have been some patent reports on the use of neural networks to detect objects.

Chinese patent CN108303747A discloses an inspection equipment and a method for detecting firearms. X-ray inspection of the inspected object to obtain a transmission image. A plurality of candidate regions in the transmission image are determined using a trained firearm detection neural network. The plurality of candidate regions are classified using the firearm detection neural network to determine whether the transmission image contains a firearm. Using the above solution, it is possible to more accurately determine whether a container/vehicle contains a firearm. It only proposes the disclosed embodiment to illustrate the convolutional neural network (CNN) as an example. The learning process of the convolutional neural network includes two links: output calculation (forward propagation) and parameter adjustment (back propagation). By learning the sample image, that is, through multi-layer convolution, excitation function, pool sampling, full connection, and error calculation, a network is optimized to ensure that the network has the smallest prediction error under the current sample image, that is, the model is considered to be the optimal model. . But no more detailed research has been done.

Chinese patent CN107871122A discloses a security inspection detection method, device, system and electronic equipment, wherein the method includes acquiring an X-ray image collected by an X-ray machine in the security inspection machine received by the security inspection terminal, and preprocessing the X-ray image , obtain the preprocessed X-ray image; extract the item features of the object to be detected corresponding to the preprocessed X-ray image according to the preset deep learning model, the preset deep learning model includes the depth based on the convolutional neural network. Learning model; use the classifier trained based on the preset deep learning model to identify the characteristics of the item, and generate the identification result corresponding to the object to be detected; send the identification result of the object to be detected to the security check terminal, so that the security check terminal displays the recognition result . The technical solution provided by the embodiment of the present invention realizes automatic identification and detection of contraband, and while improving the identification efficiency, the accuracy of the identification of contraband is effectively ensured, and the occurrence of potential safety hazards is prevented. The method is to use the neighborhood average method to smooth and denoise the collected X-ray image to obtain the X-ray image after the smooth denoising; use the histogram equalization method to enhance the edge information of the smooth and denoised image, A preprocessed X-ray image is obtained.

Chinese patent CN108734183A discloses an inspection equipment and an inspection method. X-ray scanning of the container to be inspected to obtain a transmission image, and then use the convolutional neural network to generate the first vector describing the local transmission image from the transmission image, and use the recurrent neural network to generate the word vector from the text description of the container cargo, as the second vector. vector. The first vector and the second vector are integrated to obtain a third vector representing the transmission image and the textual description. The category to which the cargo in the container belongs is determined based on the third vector. According to the embodiments of the present disclosure, the general category of the target goods can be preliminarily judged, which facilitates further judgment by the judge. The biggest difference of this invention is the addition of word vectors generated by text descriptions.

Chinese patent CN110488368A discloses a method and device for identifying contraband based on dual-energy X-ray security inspection machine. The method includes: acquiring a multi-channel image set with annotation information, the multi-channel image set including HLS images, equivalent atomic Ordinal image and X-ray received energy image, the labeling information includes position information and category information of contraband; input the multi-channel image set into the convolutional neural network for training; use the trained convolutional neural network to be detected Images are identified, and the location and category of contraband are output.

All in all, the above-mentioned patents optimize the training of X-ray images by using the method of convolutional neural network, reduce the influence of factors such as the external environment, so as to better obtain classification and detection results, and greatly improve the work efficiency in the field of security detection. and the accuracy of detection of prohibited items, etc. These patents are mainly for the detection and identification of prohibited and dangerous items that are common in daily life, such as guns and knives. At the same time, the training data sets are relatively sufficient and supported by a large number of cases. However, in some special cases, such as some terrorist attacks abroad, some small metal objects can detonate explosives, which are more destructive and lethal. It is often not well detected and has great security risks. The present invention is mainly aimed at detecting such small metal threats, and uses the deep learning method to achieve feature extraction and training. At the same time, the existing small metal threat data sets are relatively small, and in the face of only a small amount of data, it can be expanded through data enhancement and the like. data set. Innovation point 1: use dual-channel input in the convolutional network, and improve the natural image processing, so that the network model is more suitable for processing such translucent X-ray images; innovation point 2: the existing SMT The instance is projected by using the multiplicative feature to obtain a new data set, which is convenient for subsequent data training.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is: in view of the related solution 1, the detection of X-ray images by manual inspection is inaccurate and the detection time is long. For the related scheme 2, the method of feature extraction and clustering is used for detection and classification. The scheme does not process X-ray images to optimize the category features of different images. For the related scheme 3, the detection method based on shape and texture is adopted. When facing the small metal threat (SMT), it is not considered that its imaging size is small and the direction is not constrained, so it is difficult to extract special texture information for classification. This scheme cannot achieve accurate detection of Small Metal Threats (SMT).

The present invention provides the use of convolutional neural network to detect threats in X-ray images, effectively solves the problem of inability to identify accurately and separately, and has short time-consuming, strong computing power, can satisfy more inspection images, and expand data through learning representation Get better dataset training results.

Specifically, the present invention provides a detection method for small metal threats, which includes: performing X-ray inspection on an inspected object to obtain an X-ray image; and preprocessing the X-ray image to obtain a preprocessed X-ray image. ray image; perform density sampling in a small window; use a classifier trained on a preset convolutional neural network-based deep learning model to classify the detected object and calculate the confidence score of the detected image, where the confidence score is When it is greater than a certain threshold, determine the small metal threat; send the detection result of the detected object to the security check terminal.

Additionally, in addition to calculating the confidence scores for the detected images, a thermogram is generated during the classification process by mapping the normalized average window scores to pixel values at each location, by which the detected images are located. Small metal threat.

The preprocessing of the X-ray image includes: converting into a grayscale image, reducing noise interference, introducing logarithmic transformation, and improving the expression information of the image. The deep learning model based on the convolutional neural network is obtained by training a certain amount of sample data, and the sample data includes existing relevant commercial flow image data and enhanced data. The augmented data is based on the translucent nature of X-ray images, by projecting small metal threat instances into existing commercial flow images to synthesize a physically accurate image, while scaling the image intensity and flipping the image to obtain a diverse Augmented data.

The small metal threat instance includes the following operation steps: cropping out a patch containing a single small metal threat instance from a full-size image, manually performing pixel segmentation of the small metal threat instance, thereby obtaining a small metal threat binary mask, and by cutting out a patch of the small metal threat instance. The background correction is performed by dividing the resulting color patch by the average intensity of the pixels outside the small metal threat binary mask.

The deep learning model based on convolutional neural network, its CNN type is trained from scratch using MatConvNet library, MatConvNet is a MATLAB toolbox for CNN in computer vision, the toolbox is simple to use, supports complex models of large data sets, and provides Computational modules such as linear convolution with filtering, pooling, etc. Its network architecture is based on Simonyan and Zisserman (the neural network structure proposed by Simonyan and Zisserman can handle large-scale image recognition, and the network depth is pushed to 16 to 19 layers, which significantly improves the detection performance and promotes the recognition in the field of computer vision. Research and exploration of deep network) adopts a 19-layer CNN network structure, and the 19 layers contain 16CONV and 3FC. Or use an 11-layer CNN network structure, where the 11-layer contains 8CONV and 3FC. In the process of training the neural network, the image input adopts dual-channel image input or grayscale image input.

The present invention proposes a novel small metal threat (SMT) detection technology. Since threats are rare in commercial flow (SoC) and acquisition of staged images, data acquisition is difficult and time-consuming. This scheme firstly expands the data synthesis to solve the problem of less training data samples, then trains the convolutional neural network on the basis of obtaining a large number of data, and then locates the detected SMT (or the source of the false positive signal) to make classification decisions , which finally provides the results of automatic detection of SMT in full-scale images and performance evaluations for all types of backgrounds.

The overall structure of the present invention is shown in Figure 1 .

The experimental steps of the present invention are divided into four parts: data set enhancement, data processing, building a CNN network structure, and performance evaluation.

Data set enhancement: first collect the existing relevant commercial flow (SOC) image data, including some industrial equipment, etc. We first crop a patch containing a single SMT instance from the full-size image. Pixel segmentation of SMT instances is performed manually, resulting in SMT binary masks. Background correction is performed by dividing the cropped patches by the average intensity of the pixels outside the SMT binary mask. If irrelevant objects or structures appear in the patch (e.g. part of other SMTs or supporting structures), the corresponding pixels are also ignored during background correction. The SMT instance can then be projected into another X-ray image by intensity multiplication. Due to the translucent nature of X-ray images, projecting the same SMT instance to different images results in dramatically different appearances, synthesizing a physically accurate image by projecting the threat into the SoC image with simultaneous image intensity scaling and image flipping , to enhance the dataset to make it more diverse.

Data processing: During training, the data set is divided into training set and test set, and ensure that the training and test data cannot overlap. Before classification, preprocessing and logarithmic transformation are also performed. Logarithmic transformation can stretch low grayscale values with a narrow range, while compressing high grayscale values with a wider range. Can be used to expand dark pixel values in an image while compressing bright pixel values. Such transformations generally detect hidden editors through visual inspection for security, and logarithmic transformation of images is the key to improving performance.

CNN network structure: The CNN type of the present invention is trained from scratch using MatConvNet library (TFS), this architecture is based on Simonyan and Zisserman, the neural network layer has multiple convolutional layers (CONV) with 3 × 3 filters stacked on max pooling Between the Max-Pooling layers, and adding three fully connected layers (FC) layers, a total of 19 layers (16CONV+3FC) of CNN network structure are formed. The image input uses a two-channel image input (TFS-B), that is, the original intensity and the log-transformed intensity, and finally the classification probability is output through the softmax loss function. Batch normalization (determining the mean and variance of the input distribution) is used to regularize the network and speed up training, subtracting the average image computed in the training set from each input image. In addition, the image horizontal and vertical orientations are also randomly flipped during training.

Performance evaluation: Validate the experimental results with the test dataset. The final test results reflect the performance indicators of the network structure and the accuracy of detecting Small Metal Threats (SMT).

Beneficial effects:

Compared with the prior art, the invention proposes a method for detecting small metal threats (SMT) in X-rays based on a convolutional neural network, and at the same time, a representation learning method is used to train data to optimize class features. Fewer than 6% of false alarms were reported when 90% of SMT compositions were detected hidden in commerce flow images in the detection results. Compared with the traditional histogram statistical information retrieval method, it is improved by an order of magnitude.

For the previous related patents, the data set can be expanded by the multiplicative properties of X-rays, the feature information of objects can be accurately extracted, and the under-fitting caused by insufficient data can be reduced.

Description of drawings

In order to better understand the present invention, the present invention will be described in detail according to the following drawings:

Fig. 1 is the overall structure diagram of the present invention;

Fig. 2 is a graph of the influence of logarithmic transformation on the X-ray image of the bolt cutter;

Figure 3 is a TFS network configuration diagram;

Figure 4 is a comparison chart of the SMT detection performance of each method;

Figure 5 is a comparison diagram without SMT detection;

Figure 6 is an example of SMT detection of CNN-19-TFS-B;

Figure 7 is a comparison diagram of SMT detection.

Detailed ways

The present invention will be further described in detail below with reference to specific embodiments. It should be understood that these embodiments are intended to illustrate the basic principles, main features and advantages of the present invention, and the present invention is not to be limited in scope by the following embodiments. The implementation conditions used in the examples can be further adjusted according to specific requirements, and the unremarked implementation conditions are usually the conditions in routine experiments.

The first step is to acquire relevant image data: During the data enhancement process, we use the R60 orbital scan of the Rapiscan Eagle R scanner to obtain a benign image (without SMT) of this work, which is equipped with a 6MV linear accelerator power supply and is able to obtain An X-ray image with a resolution of 6mm/pixel-1 in the horizontal direction. While the images are 16-bit grayscale images, the image size varies between 1290×850 and 2570×850 pixels for 20-foot and 40-foot containers, respectively. In practice we randomly collected sample data from commercial flow (SoC) images, which can be empty samples (which account for about 20% of the entire dataset) or can contain commercial cargo containers, heavy machinery, industrial equipment, household Supplies and pallets for bulk materials, etc. Based on the multiplicative nature of X-ray transmission image formation, the same SMT instance is projected into different images to generate different appearance information, which is used as a new data set, and the image is rotated, zoomed and other ways to perform data at the same time. enhanced.

The second step of image data processing: In the process of data processing, due to the translucent nature of the X-ray image, we first preprocess it, convert it into a grayscale image and reduce noise interference, introduce logarithmic transformation, and improve the expression information of the image. In Figure 2, the original image of the bolt cutter, the original intensity image of the X-ray, and the logarithmically transformed image are shown, respectively, with enhanced dark pixel values. The detection of SMT in X-ray cargo images is a binary classification task, benign images (no SMT) are classified as negative class and SMT images (at least one SMT) are classified as positive class by density sampling in a small window And give the confidence, compare it with the threshold T to get the category prediction result.

The third step is to train the convolutional neural network: In the process of training the neural network, our network architecture is based on Simonyan and Zisserman, using a 19-layer (16CONV+3FC) CNN network structure, and also discusses 11 layers (8CONV+3FC) structural variants of , and three configurations for these two structures, see Figure 3: grayscale image input (TFS-A), two-channel image input (TFS-B), and then the original function and logarithmic transformation The latter inputs are respectively distributed to the functionally separated first fully connected layer FC (TFS-C) and then connected to each branch of the network (no sharing of rights). Batch normalization is used to regularize the network and speed up training, preventing overfitting during training. Weight decay and momentum were fixed at 10-4 and 0.9, respectively. Within 30 epochs of training, the learning rate is reduced from 10-3 to 10-6. In addition to the CNN model of TFS, a pre-trained (PT) network is also trained, the network features are extracted from the FC1 and FC2 layers of the VGG-VD-19 model, and its structure is very similar to the 19-layer CNN-TFS, in ImageNet ( Compared with other classifiers, random forest has fewer parameters and strong classification ability, and is often used in engineering tasks. Resize the input image to 224×224 and duplicate the grayscale channel twice in the third dimension to match the expected RGB format. In addition to computing confidence scores for detected images, thermograms can also be generated during the classification process by mapping the normalized mean window scores to pixel values at each location. These visualizations can roughly locate detected SMTs (or sources of false positive signals) to classify classification decisions, while also providing follow-up work (e.g., physical inspections) for security personnel.

Bag-of-words (BoW) functions are also evaluated in network training experiments, including Oriented Basic Image Features (oBIF) and Pyramid Histogram of Visual Words (PHOW). Among them, BIF is a fixed geometric feature, and PHOW is an extended form of dense SIFT (scale-invariant feature transform) multi-scale variation.

The fourth step performance evaluation: After the construction and training of the neural network, the performance comparison of the test classification is carried out next. Using Nvidia's Titan X GPU during training, its processing time averaged 3.5 seconds per image, which is also significantly lower than the time it takes for security workers to inspect a cargo box. It can be seen from Figure 4 that the detection effect of the bag-of-words model is not good, where the PHOW log-transformed output obtains the best AUC and H metrics. The pre-trained (PT) CNN model does not perform as well as the bag-of-words model. The detection effect of CNN trained from scratch (TFS) is better, and the CNN-19-TFS-B model used in this patent has achieved the best detection results, and it can be seen that logarithmic transformation plays an important role in improving performance. In Figure 4, AUC is the area under the ROC curve, and FPR90 is the false positive rate of 90% detection rate. Figure 5. The thermal imaging visual view shows the detection situation without SMT, and the red signal (1.0) represents FPR90, which shows that the detection effect is the best. Figure 6. The thermal imaging visual view shows the detection situation when there is SMT. Using CNN-19-TFS-B to classify and identify the SMT in various positions, the red area is the position of the SMT, which can be ready for detection. Overall, the patented scheme is compared with bag of words and pre-training (PT) network methods, and the results are shown in Figure 7, which shows that the AUC of the CNN-TFS method is 0.97, and the H-measure is 0.78, which has a good detection and classification effect. , which significantly outperforms other evaluation methods.

The present invention has been described in detail above, and the description of the specific embodiment is only used to help understand the method of the present invention and its core idea, and its purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and The protection scope of the present invention cannot be limited by this. All equivalent changes or modifications made according to the spirit of the present invention should be included within the protection scope of the present invention.

Claims (10)

1. A method for detecting a small metal threat, comprising: carrying out X-ray inspection on an object to be inspected to obtain an X-ray image; preprocessing the X-ray image to obtain a preprocessed X-ray image; density sampling is carried out by a small window; classifying the detected object by using a classifier trained on a preset deep learning model based on a convolutional neural network, calculating a confidence score of a detected image, and judging a small metal threat under the condition that the confidence is greater than a specific threshold value; and sending the detection result of the detected object to a security inspection terminal.

2. The detection method according to claim 1, characterized in that: in addition to calculating confidence scores for the detected images, a thermographic map is generated during the classification process by mapping the normalized mean window scores to pixel values at each location, through which the detected small metal threats are localized.

3. The detection method according to claim 1, characterized in that: the preprocessing of the X-ray image comprises: and the image is converted into a gray image, noise interference is reduced, logarithmic transformation is introduced, and the expression information of the image is improved.

4. The detection method according to claim 1, characterized in that: the deep learning model based on the convolutional neural network is obtained through training of a certain amount of sample data, and the sample data comprises existing related commercial flow image data and enhancement data.

5. The detection method according to claim 4, characterized in that: the enhancement data is based on the semi-transparent nature of the X-ray image, synthesizing a physically accurate image by projecting small metal threat instances into existing commercial stream images, while scaling and flipping the image intensity, resulting in diversified enhancement data.

6. The detection method according to claim 5, characterized in that: the small metal threat example comprises the following operation steps: cutting out a patch containing a single small metal threat instance from a full-size image, manually performing pixel segmentation of the small metal threat instance to obtain a small metal threat binary mask, and performing background correction by dividing the cut-out color patch by the average intensity of pixels except the small metal threat binary mask.

7. The detection method according to claim 1, characterized in that: the deep learning model based on the convolutional neural network has a CNN type which is trained from scratch by using a MatConvNet library.

8. The detection method according to claim 1, characterized in that: the network architecture of the deep learning model based on the convolutional neural network adopts a 19-layer CNN network structure, wherein the 19 layers comprise 16CONV and 3 FC.

9. The detection method according to claim 1, characterized in that: the network architecture of the deep learning model based on the convolutional neural network adopts a CNN network structure with 11 layers, wherein the 11 layers comprise 8CON and 3 FC.

10. The detection method according to claim 1, characterized in that: according to the deep learning model based on the convolutional neural network, in the process of training the neural network, image input adopts dual-channel image input or gray image input.

Images