KR101894537B1 - Method and system for illegal object classification based on computer vision - Google Patents

Abstract

According to an embodiment of the present invention, an illegal object classification method based on computer vision includes: (a) a step in which an X-ray image acquiring unit acquires X-ray images of baggage; (b) a step in which an illegal object detection unit detects an illegal object from the X-ray images of the baggage; (c) a step in which a feature extracting unit extracts the features of the illegal object from the X-ray images of the detected illegal object and combines the extracted features to generate feature vectors; and (d) a step in which an illegal object classification unit classifies the illegal object by using a random forest classifier by using the feature vectors as inputs. The step (c) of generating the feature vectors includes the steps of: (c-1) obtaining a transformation coefficient map by executing contourlet transform on the X-ray images of the detected illegal object; (c-2) extracting Tamura texture features from the transformation coefficient map; (c-3) extracting the image histogram features from the X-ray images of the detected illegal objects; and (c-4) coupling the image histogram features and the Tamura texture features to generate the feature vectors.

Description

[0001] METHOD AND SYSTEM FOR ILLEGAL OBJECT CLASSIFICATION BASED ON COMPUTER VISION [0002]

The present invention relates to a method and system for classifying illegal objects based on computer vision.

Airports, which are public facilities, are very important for commercial flights. Security checks at airports are important for any safe flight. One important task of each security check is to detect and classify illegal objects in baggage. This will put considerable pressure on the customer. The traditional method for security checks is manual inspection of baggage. However, this method requires a lot of human labor and time. An alternative is to examine the X-ray image of the baggage obtained with the X-ray generator, rather than opening the baggage by a human and manually inspecting it. Because it does not need to open baggage, much manpower and time can be saved. Therefore, instead of opening the baggage, inspecting the x-ray image of the baggage is a good solution for security checks.

X-ray radiation is in the form of electromagnetic radiation. The absorption coefficient and thickness of the material determine the X-ray absorption of the object. Due to the transmission capabilities of X-rays, X-rays are widely used in a variety of applications, including but not limited to airport security, medical use, and computed tomography. Based on the transmission capability of the X-rays, objects within the baggage can easily be acquired with an X-ray image. Compared to traditional human labor-based methods of opening baggage and manually inspecting baggage, X-ray image based methods are much faster because they do not need to open baggage. Figure 1 illustrates two exemplary x-ray images and their corresponding images.

However, existing inspection systems require that the screen be highly focused to inspect the x-ray image of the baggage. It is a very difficult task for any worker who makes a mistake when his mind is distracted. With the development of computer vision, it is desirable to automatically inspect or classify X-ray images of different illegal objects.

In the non-patent document [2], the authors proposed to use a method based on multiple X-ray views to detect some regular forbidden items with a very definite shape and size. The method consists of two steps: structure estimation and partial detection. In the non-patent document [3], the authors described the security check task at the airport as the selection of the infrastructure, technical equipment, personnel allocation, and financial means necessary to perform all the functions of the security check task. They then proposed a fuzzy system to support the configuration of the entire security inspection system. In the non-patent document [4], the authors extended previous research by exploring the effect of attribute selection on classification performance. In the non-patent document [5], the authors proposed to describe a cargo X-ray image by a joint shape and a texture feature, which can reflect both the internal detail and the cargo loading mode. Furthermore, the authors discussed the characteristics of X-ray images and explored some reasons why it is very difficult to sort cargo under X-rays.

However, much effort has been devoted to X-ray image processing, and most of these efforts are focused primarily on X-ray image enhancement and medical image recognition. There are not many studies on X - ray image recognition using computer vision for security check.

[1] G. Zentai, "X-ray Imaging for Homeland Security", IEEE International Workshop on Imaging Systems and Techniques, (2008) [2] D. Mery and V. Riffo, "Automated Object Recognition using Multiple X-ray Views", Materials Evaluation, vol. 72, no. 11, (2014), pp. 1362-1372 [3] J. Skorupski and P. Uchronskia, "A fuzzy system to support the configuration of baggage screening devices at an airport", Expert Systems with Applications, vol. 44, no. 2, (2016), pp. 114-125 [4] D. Unay, O. Soldea, A. Ekin, M. Cetin and A. Ercil, "Automatic Annotation of X-Ray Images: A Study on Attribute Selection," Lecture Notes in Computer Science, for Clinical Decision Support, vol. 5853, pp. 97-109. [5] J. Zhang, L. Zhang, Z. Zhao, Y. Liu, J. Gu, Q. Li and D. Zhang, "Joint Shape and Texture Based X-Ray Cargo Image Classification", IEEE Conference on Computer Vision and Pattern Recognition Workshops 2014, pp. 266-273. [6] M. N. Do and M. Vetterli, "The contourlet transform, an efficient directional multiresolution image representation", IEEE Transactions on Image Processing, vol. 14, no. 12, (2005), pp. 2091-2106. [7] W. Wu, Z. Liu and Y. He, "Classification of Defects with Ensemble Methods in the Automated Visual Inspection of Sewer Pipes", Pattern Analysis and Applications, vol. 18, no. 2, (2015), pp. 263-276. [8] W. Wu, Z. Liu, W. Gueaieb and X. He, "Single-image super-resolution based on Markov random field and contour transform", J. Electron, Imaging, vol. 20, no. 2, (2011), pp. 023005-023005. [9] T. Deselaers, D. Keysers and H. Ney, "Features for Image Retrieval: An Experimental Comparison", Information Retrieval, vol. 11, no. 2, (2008), pp. 77-107. [10] H. Tamura, S. Mori and T. Yamawaki, "Texture features corresponding to visual perception", IEEE Transactions on Systems Man and Cybernetics, vol. 8, no. 6, (1978), pp. 460-473, [11] Y. Liu, Z. Li and Z. Gao, "An Improved Texture Features Extraction Method for Tire Tread", Lecture Notes in Computer Science, Patterns Intelligence Science and Big Data Engineering, vol. 8261, pp. 705-713. [12] J. J. Rodriguez, L. I. Kuncheva, and C. J. Alonso, "Rotation Forest: A New Classifier Ensemble Method", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 10, (2006), pp. 1619-1630. [13] A. Verikas, A. Gelzinis and M. Bacauskiene, "Mining data with random forests: A survey and results of new tests", Pattern Recognition, vol. 44, no. 2, (2011), pp. 330-349. [14] P. O. Gislason, J. A. Benediktsson and J. R. Sveinsson, "Random Forests for land cover classification", Pattern Recognition Letters, vol. 27, no. 4, (2006), pp. 294-300.

The object of the present invention is to automatically classify illegal objects based on a feature vector that combines features of a tamura and a histogram based on a contour transform extracted from an X-ray image of an illegal object to open the baggage and manually check whether there are illegal objects , It is not necessary to focus on the mind to focus on the examination of X-ray images and to improve the classification performance of illegal objects. .

Another problem to be solved by the present invention is to automatically classify illegal objects based on a feature vector that combines the features of Tamura and histogram based on the contour transform extracted from the X-ray image of the illegal object to open the baggage, , It is not necessary to focus on the mind to concentrate on the examination of X-ray images and to improve the classification performance of illegal objects. System.

According to another aspect of the present invention, there is provided a computer vision-based illegal object classification method,

(a) in the X-ray image acquiring section, acquiring an X-ray image of the baggage;

(b) at the illegal object detecting section, detecting an illegal object in the X-ray image of the baggage;

(c) extracting features of the illegal object from an X-ray image of the detected illegal object and combining the features to form a feature vector; And

(d) classifying the illegal object using a random forest classifier with the feature vector as an input in an illegal object classifier,

Wherein the step of forming the feature vector comprises:

(c-1) performing a contourlet transformation on an X-ray image of the detected illegal object to obtain a transform coefficient map;

(c-2) extracting Tamura texture features from the transform coefficient map;

(c-3) extracting an image histogram feature from an X-ray image of the detected illegal object; And

(c-4) combining the image histogram features with the Tamura texture features to form the feature vector.

A method for classifying illegal objects based on computer vision according to an embodiment of the present invention is characterized in that, prior to step (a)

Extracting the Tamura texture features and the image histogram from the X-ray images of a plurality of known illegal objects and extracting the feature vectors by combining the extracted Tamura texture features and the image histogram feature in the feature extraction unit; And

And in the illegal object classifier, forming a classification model for the random forest classifier for classifying illegal objects based on the feature vectors.

In the illegal object classification method based on computer vision according to an embodiment of the present invention, in the illegal object classification unit, the illegal object classification unit may classify the illegal object by applying the feature vector to the classification model .

According to another aspect of the present invention, there is provided a computer vision-based illegal object classification method, wherein the Tamura texture features include a directionality of the X-ray image, a roughness of the X-ray image, Contrast of the line image may be included.

In the illegal object classification method based on computer vision according to an embodiment of the present invention, the step (c-1)

Applying a Laplacian pyramid to the X-ray image of the detected illegal object to decompose the X-ray image of the illegal object into a low frequency map and a high frequency map; And

Directional decomposition of the high frequency map using a directional filter bank to obtain a high frequency sub-image to obtain a transform coefficient map.

According to another aspect of the present invention, there is provided an illegal object classification system based on computer vision,

An X-ray image obtaining unit for obtaining an X-ray image of the baggage;

An illegal object detecting unit for detecting an illegal object in the X-ray image of the baggage;

A feature extraction unit for extracting features of the illegal object from an X-ray image of the detected illegal object and combining the features to form a feature vector; And

And an illegal object classifier for classifying the illegal object using a random forest classifier with the feature vector as an input,

The feature extraction unit may extract,

Performing a contourlet transform on an X-ray image of the detected illegal object to obtain a transform coefficient map;

Extracting Tamura texture features from the transform coefficient map;

Extracting an image histogram feature from the x-ray image of the detected illegal object; And

And combining the Tamura texture features with the image histogram feature to form the feature vector.

In the illegal object classification system based on computer vision according to an embodiment of the present invention, before the illegal object classifier performs an operation of classifying illegal objects,

The feature extraction unit further extracts the feature vectors from the X-ray image of the known plurality of illegal objects by extracting the Tamura texture features and the image histogram, combining the extracted Tamura texture features and the image histogram feature,
The illegal object classifier may further perform the operation of forming a classification model for the random forest classifier to classify illegal objects based on the feature vectors.

delete

In the illegal object classification system based on computer vision according to an embodiment of the present invention, the illegal object classifier may classify the illegal object by applying the feature vector to the classification model.

In an illegal object classification system based on computer vision according to an exemplary embodiment of the present invention, the Tamura texture features include a directionality of the X-ray image, a roughness of the X-ray image, Contrast of the line image may be included.

In an illegal object classification system based on a computer vision according to an embodiment of the present invention, an operation of performing a contour transform on an X-ray image of the detected illegal object to obtain a transform coefficient map ,

Applying an Laplacian pyramid to the X-ray image of the detected illegal object to decompose the X-ray image of the illegal object into a low-frequency map and a high-frequency map; And

Directional decomposition of the high frequency map using a directional filter bank to obtain a high frequency sub-image to obtain a transform coefficient map.

According to the computer vision-based illegal object classification method and system according to an embodiment of the present invention, the combination of the Tamura feature and the histogram feature based on the contour transform extracted from the X-ray image of the illegal object By automatically classifying illegal objects based on feature vectors, it is not necessary to open baggage and check for the presence of illegal objects manually. This saves a lot of labor and time, There is no need to improve the performance of illegal object classification.

1 shows an X-ray image of a knife, FIG. 1B shows an X-ray image of a handgun, FIG. 1C shows a visible image of a knife, 1d is a view showing a visible image of the pistol;
2 is a flowchart of a method of classifying illegal objects based on computer vision according to an embodiment of the present invention.
FIG. 3 is a detailed flowchart of the illegal object feature extraction step shown in FIG. 2. FIG.
4 is a block diagram of a computer vision based illegal object classification system in accordance with an embodiment of the present invention.
5 shows the structure of the contourlet transformation;
6A is an original X-ray image of the electric shocker, FIG. 6D is an intensity image of FIG. 6A, and FIGS. 6B, 6C and 6C show an example of a contourlet transformation for an X- 6e, and 6f show high frequency coefficient maps after performing the contour transform, respectively.
FIG. 7 shows a comparison of different histograms of different types of illegal objects, wherein FIG. 7A is an X-ray image of a knife, FIG. 7B is an X-ray image of an acetone bottle, FIG. 7C is a histogram of FIG. 7B.
Fig. 8 is a diagram comparing different histograms of the same type of illegal object, wherein Fig. 8A is an X-ray image of a knife, Fig. 8B is an X- ray image of another knife, Fig. 8C is a histogram of Fig. 8B.
9 is a view showing a false work of a random forest.
FIG. 10 is a view showing a part of X-ray images of illegal objects, FIG. 10A is a total X-ray images, FIG. 10B is X-ray images of ammunition, Fig. 10D shows X-ray images of flammable and explosive objects; Fig.
Figure 11 relates to experiments for feature extraction and classification.

BRIEF DESCRIPTION OF THE DRAWINGS The objectives, specific advantages and novel features of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

Prior to that, terms and words used in the present specification and claims should not be construed in a conventional and dictionary sense, and the inventor may properly define the concept of the term in order to best explain its invention Should be construed in accordance with the principles and the meanings and concepts consistent with the technical idea of the present invention.

It should be noted that, in the present specification, the reference numerals are added to the constituent elements of the drawings, and the same constituent elements are assigned the same number as much as possible even if they are displayed on different drawings.

Also, the terms "first", "second", "one side", "other side", etc. are used to distinguish one element from another, It is not.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description of the present invention, a detailed description of known arts which may unnecessarily obscure the gist of the present invention will be omitted.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

An illegal object classification method based on computer vision according to an embodiment of the present invention deals with X-ray image recognition using computer vision for illegal object recognition. The main study of illegal object classification method based on computer vision according to an embodiment of the present invention is to provide a prototype of automatic illegal object recognition based on X-ray image for civil aviation airport. In addition, the computer vision-based illegal object classification method according to an embodiment of the present invention includes a contour transformation [6, 7, 8] and a Tamura feature [9, 10, 11]. The method of classifying an illegal object based on computer vision according to an embodiment of the present invention not only helps an illegal object recognition system using an X-ray image, but also provides a standard for automatic identification of irregular objects.

Since illegal objects can have various shapes, it is very difficult to use a knowledge model to extract features of illegal objects. In order to solve this problem, in an illegal object classification method based on computer vision according to an embodiment of the present invention, texture features of illegal objects are extracted, which means that texture features have characteristics inherent to objects, It is widely used in image analysis and recognition. In particular, the Contourlet transformation and the Tamura feature are used in a method of classifying illegal objects based on computer vision according to an embodiment of the present invention.

Contouret transformation, which has directionality, multiscale, and anisotropy, can effectively capture edge information of an image as compared to wavelet transform. Therefore, contour transformation is widely used to extract transformation coefficient maps. Furthermore, the computer vision-based illegal object classification method according to an embodiment of the present invention extracts the Tamura features of the transform coefficient map because the Tamura texture features have a texture model similar to the human visual system. By using the contour transform and Tamura texture features, the recognition system can have better performance. In addition, an image histogram feature is extracted because the image histogram feature is not affected by image rotation and transformation. By combining the Tamura texture feature and the image histogram feature based on the contour transform, an illegal object classification method based on computer vision according to an embodiment of the present invention can obtain the final characteristic of illegal objects.

On the other hand, when compared to other classifiers, random forests have fewer parameters and strong classification capabilities, which is well suited for engineering applications. Therefore, the computer vision-based illegal object classification method according to an embodiment of the present invention uses a random forest to classify illegal objects using the extracted features.

In an illegal object classification method based on computer vision according to an embodiment of the present invention, an illegal object classification system is implemented. First, an illegal object classification method based on computer vision according to an embodiment of the present invention proposes a method of combining a histogram with a Tamura feature based on a Contour transformation. Next, a computer vision-based illegal object classification method according to an embodiment of the present invention applies a random forests classifier to classify these features from illegal objects. Experimental results show that an illegal object classification method based on computer vision according to an embodiment of the present invention can effectively distinguish different types of illegal objects.

On computer vision Based  Illegal object classification method and system

An X-ray inspection method based on computer vision in an illegal object classification method and system based on computer vision according to an embodiment of the present invention is proposed. The method automatically recognizes the category of illegal objects in the X-ray image. A block diagram of a method for classifying illegal objects based on computer vision according to an embodiment of the present invention is shown in FIG.

2, an illegal object classification method based on a computer vision according to an exemplary embodiment of the present invention includes the steps of obtaining an X-ray image of a baggage in an X-ray image obtaining unit 400 (step S200) 4) from the X-ray image of the detected illegal object (step 402 in FIG. 4) (step S202) of detecting an illegal object in the X-ray image of the baggage (Step S204) of extracting features of the illegal object and combining the features to form a feature vector, and inputting the feature vector in the illegal object classifier (406 in FIG. 4) to generate a random forest classifier And classifying the illegal object (step S206).

Referring to FIG. 3, the step of extracting the features and forming a feature vector (step S204) includes performing a contourlet transformation on an X-ray image of the detected illegal object to obtain a transform coefficient map Extracting Tamura texture features from the transform coefficient map (step S302), extracting an image histogram feature from the X-ray image of the detected illegal object (step S304) And forming the feature vector by combining the image histogram features with the Tamura texture features (step S306).

Meanwhile, in the illegal object classification method based on the computer vision according to the embodiment of the present invention, before the step S200, the characteristic extraction unit (404 in FIG. 4) extracts, from the X- (Not shown) for extracting texture features and an image histogram, extracting feature vectors by combining extracted Tamura texture features and image histogram features, and in the illegal object classifier (406 in FIG. 4) (Not shown) to form a classification model for the random forest classifier to classify illegal objects based on the classification model for the random forest classifier, and the classification model for the random forest classifier is formed in advance by these steps.

In this case, in step S206, the illegal object classifier 406 classifies the illegal object by applying the feature vector to the classification model for the random forest classifier.

In addition, the Tamura texture features include the directionality of the X-ray image, the roughness of the X-ray image, and the contrast of the X-ray image.

4, an illegal object classification system based on a computer vision according to an exemplary embodiment of the present invention includes an X-ray image obtaining unit 400 for obtaining an X-ray image of baggage, A feature extraction unit 404 for extracting the features of the illegal object from the X-ray image of the detected illegal object and combining the features to form a feature vector, an illegal object detection unit 402 for detecting an illegal object, And an illegal object classifier 406 for classifying the illegal object using a random forest classifier with the feature vector as an input.

The feature extracting unit 404 may perform a contourlet transformation on the X-ray image of the detected illegal object to obtain a transform coefficient map. The feature extracting unit 404 extracts Tamura texture features (Tamura texture extracting an image histogram feature from the X-ray image of the detected illegal object, and combining the Tamura texture features and the image histogram feature to form the feature vector .

As described above, the method of classifying illegal objects based on computer vision according to an embodiment of the present invention includes four steps; Image acquisition (step S200), position of illegal object (step S202), feature extraction (step S204), and classification (step S206).

The first step (step S200) is to capture an X-ray image using an X-ray generator. The second step (step S202), i.e., the illegal object position measurement step, is to detect illegal objects in the X-ray image. The third step (step S204) is to extract features of the illegal objects. In particular, features of the x-ray image, tamura features, and image histogram features of the illegal object detected by the contour transform are extracted.

In an illegal object classification method based on computer vision according to an embodiment of the present invention, first, a contour transform is performed to obtain transformation coefficient maps of an X-ray image. And then extracts the Tamura features from the transform coefficient maps [7]. At the same time, the image histogram feature is extracted. Finally, all features are combined to form a feature vector.

The last step (step S206) is to classify illegal objects based on the pre-formed classification model. In this step, the feature vector is provided to the random forest to obtain the final result.

It should be noted that the method of classifying illegal objects based on computer vision according to an embodiment of the present invention mainly uses the last two steps: feature extraction (step S204) and classification (step S206). Although illegal object location measurement is a very important issue in inspection systems, location of illegal objects is beyond the scope of the present invention.

Feature extraction

For image recognition or classification systems, feature extraction produces beneficial and non-redundant features. Generally, the extracted features have a smaller size than the original image. This means that feature extraction is associated with diminishing dimensionality. The main purpose of feature extraction is to help improve the performance of subsequent classification. It has a very important meaning in recognition systems. In the illegal object classification method based on computer vision according to an embodiment of the present invention, three techniques including a contour transform, a Tamura texture, and an image histogram are used to extract features of illegal objects. The details of all three techniques will be described below.

Contourlet  conversion( Contourlet  Transform)

Wavelet transform has been widely used in many image processing applications. Because the wavelet will only capture vertical and horizontal edges in the input image, the wavelet can not capture edges along the other directions. To solve this problem, contour transform rather than wavelet transform is used in a method of classifying illegal objects based on computer vision according to an embodiment of the present invention. The contour transformation is similar to the human visual system. The contour transformation results from a curvelet transform, which is capable of capturing smooth images in different directions. Thus, the contouret transformation can extract image information in different directions and on multiple scales.

The contouret transformation consists of two steps: multiscale decomposition and directional decomposition. The structure of the contour transformation is as shown in Fig.

In the multiscale decomposition step, a Laplacian pyramid is performed to decompose the input image into a low frequency and a set of high frequency maps. In the directional decomposition step, directional filter banks (DFB) are used to obtain high frequency sub-images. After performing the contouret transformation, multi-scale and multi-directional maps can be obtained from the input image. 6 shows an example of two-level decomposition of a contour transform for an X-ray image of a stun gun.

Tamura Texture  Features Tamura  Texture Features)

A texture feature is one of the inherent characteristics of an image. It is related to the properties of the object, which reflects the change in the intensity of the image. Texture features are widely used in image analysis and recognition. The computer vision-based illegal object classification method according to an embodiment of the present invention extracts the Tamura texture features from the contour transform coefficient maps because the Tamura texture features have a psychological measurement very similar to the human visual system.

In particular, the Tamura texture features include six features: coarseness, contrast, directionality, line-similarity, regularity, and roughness. Among the six features, directionality, roughness, and contrast, which are particularly important for feature extraction, are used to express the contour transformation coefficient maps of the X-ray image. The details of directionality, roughness and contrast will be described below.

Directionality

Directionality describes the overall concentration of the image in any direction, rather than locally. In order to calculate the directionality, it is necessary to calculate the modulus and edge direction of each pixel. The modulus and directionality can be calculated as follows:

와

는 각각 픽셀의 모듈러스와 방향성을 나타낸다.

와

는 각각 수평및 수직 그래디언트(gradient)를 나타낸다.

와

는 하기의 2개의 3×3 연산자들을 사용하는 이미지의 컨벌루션(convolution)에 의해 계산될 수 있다.In the above,

Wow

Represent the modulus and directionality of the pixel, respectively.

Wow

Represent horizontal and vertical gradients, respectively.

Wow

Can be calculated by the convolution of the image using the following two 3x3 operators.

)을 계산한 이후에, 방향성(

)이 양자화된다. 그 다음,

인 모든 픽셀들을 카운트하는데, t는 소정의 임계값이다. 후속하여,

를 만족하는 픽셀들의 수를 이용하여 히스토그램이 형성될 수 있다.All orientations of each pixel (

), The directionality (

) Are quantized. next,

, Where t is a predetermined threshold value. Subsequently,

The histogram may be formed using the number of pixels satisfying the following equation.

The histogram may be calculated as follows:

는 각 레벨 k에서

를 만족하는 픽셀들의 수이다.Where n represents the quantization level.

At each level k

Lt; / RTI >

The directionality may be calculated with sharpness of the histogram.

는 그것을 포함하는 모든 빈들(bins)을 나타내고,

는 최고 피크값을 갖는 빈(bin)을 나타낸다. 그리고 p는 히스토그램의 피크값이고,

는 히스토그램의 모든 피크값들이다.In the above,

Represents all bins that contain it,

Represents a bin having the highest peak value. And p is the peak value of the histogram,

Are all peak values of the histogram.

If the histogram has an apparent peak, i. E. Has a high F value, this proves that the image is an image with an apparent direction. Similarly, if the histogram is very flat, i.e. has a low F value, this proves that the image is an image with no clear direction.

Coarseness

Coarseness is a measure of the roughness of images. In particular, roughness is for evaluating the size of texture elements. In general, if the texture of the image is very rough, the image will have a large roughness value. In contrast, if the texture of the image is flat, the image will have a small roughness value.

의 로컬 영역의 평균 인텐시티 값이 계산된다. 이것은 하기와 같이 표현될 수 있다:In order to calculate the roughness, it is necessary that the size

The average intensity value of the local region of the image is calculated. This can be expressed as: < RTI ID = 0.0 >

이다. 그리고

는 위치 (i,j)에서 픽셀의 인텐시티 값을 나타낸다.In the above,

to be. And

Represents the intensity value of the pixel at position (i, j).

Then, for each pixel, the difference between the average intensity values of the two neighboring regions, which do not overlap with each other in the horizontal and vertical directions, can be obtained. This can be defined as:

가 선택된다. 최선의 크기는 하기와 같이 계산될 수 있다:For each pixel, the best size to provide the highest output value

Is selected. The best size can be calculated as:

Finally, an average over the best size for each pixel is obtained as a roughness measure for the image.

은 상기 이미지의 크기이다.In the above,

Is the size of the image.

The key idea is to measure the roughness of each pixel of various sizes. The average value of the roughness for the entire image is then calculated as the roughness of the entire image.

Contrast

Contrast is a measure of the intensity distribution over the entire image. The contrast of the image is calculated as follows.

는 네번째 모멘트(fourth moment)이고,

는 상기 이미지의 인텐시티의 분산이다.In the above,

Is the fourth moment,

Is the variance of the intensity of the image.

Image histogram

The image histogram consists of many bins. The histogram is calculated by assigning each pixel in the image to a bin according to the pixel intensities. Each pixel in the image is assigned to its corresponding bin. The image histogram is a global feature of the image. The advantage of this feature is that it is not affected by image rotation and translational motion. It is insensitive to the image scale as well as insensitive to the spatial location of objects in the image. In particular, the image histogram feature is suitable for objects that are difficult to automatically segment.

Figures 7 and 8 are two examples illustrating the distinguishing feasibility of image histogram features. Figure 7 shows a comparison of two histograms of different types of illegal objects. From Fig. 7, it can be observed that different types of illegal objects have totally different histograms. Figure 6 shows a comparison of different histograms of illegal objects of the same type. Unlike FIG. 7, it can be seen from FIG. 8 that illegal objects of the same type have similar histograms. All this proves that the histogram feature has good discrimination for classification.

Classification

With the development of statistical theory, many classifiers have been proposed recently, including but not limited to support vector machines (SVM), random forests. Compared to other classifiers, random forests have strong classification capabilities with fewer parameters. This indicates that random forests are well suited for engineering applications. Accordingly, the computer vision-based illegal object classification method according to an embodiment of the present invention uses a random forest to classify the extracted features. A random forest is an overall classification method that provides generally superior accuracy compared to the accuracy of single classification methods.

In general, there are two types of total classification methods available. One type of method is AdaBoost, and the other is bagging. The bagging methods use some random heuristic methods to form each other independently. The random forest, which is comprised of L decision trees, is a method of bagging. Each tree in the random forest is formed independently using different sets of features extracted from the train set. Random forests have the following advantages:

(1) They can provide a high recognition rate.

(2) The training process and the testing process require a small amount of computation.

(3) They can evaluate the importance of each feature variable.

The framework of the random forest is shown in FIG.

Experiment result

In order to verify the effect of the computer vision-based illegal object classification method according to an embodiment of the present invention, 276 illegal objects of X-ray images are used in the experiment. These X-ray images contain four categories of illegal objects: guns, ammunition, edge tools, and flammable and explosive objects. These images are randomly divided into two parts. One part containing 117 images is for training. The rest, including 159 images, is for testing. All images are normalized to 256 x 256, which is the same resolution. Portions of the X-ray images are shown in FIG.

Both feature extraction methods and classifiers affect the performance of the computer vision-based illegal object classification method according to an embodiment of the present invention. In order to analyze the performance of the computer vision-based illegal object classification method according to an embodiment of the present invention, different feature extraction methods and different classifiers are combined as shown in FIG. Feature extraction methods include co-occurrence matrix features, Tamura texture features, and image histogram features. The classifiers used in the experiments include random forest, Bayesian networks, decision trees and support vector machines (SVM). Table 1 summarizes the experimental results.

The combining feature used in the computer vision based illegal object classification method according to an embodiment of the present invention combines the tamura characteristic based on the histogram feature and the contour transformation. From Table 1, the random forest classifier achieves better results than other classifiers even when using any feature. This shows that the random forest classifier has better performance than the other classifiers. This is because aggregate methods generally have better performance than single classifier methods. Random forest classifiers are typical collective methods, while other classifiers are single classifier methods.

Moreover, it can be seen that the combining feature has better performance compared to other features for any classifier. This proves that the combining feature is the best feature among the compared features. This is because the combining feature combines the advantages of both the histogram feature and the contour transform based Tamura feature.

conclusion

Security checks at airports are important for any safe flight. The traditional method for security checks is manual inspection of baggage. However, this method requires a lot of human labor and time. It is desirable to automatically check the X-ray image of the baggage using computer vision. In an illegal object classification method based on computer vision according to an embodiment of the present invention, an illegal object classification method and system are implemented. First, a contouret transformation is performed to obtain the contour transformation transform coefficients maps. The next Tamura feature is extracted from the transform coefficient maps. On the other hand, an image histogram feature is obtained. Thus, by combining these features, these features are linked to the feature vector. Finally, the linked feature vectors are provided to the random forest to obtain the results. Experimental results show the effectiveness of a method and system for classifying illegal objects based on computer vision according to an embodiment of the present invention. The method and system for classifying illegal objects based on computer vision according to an embodiment of the present invention not only provides a prototype system for an automatic recognition system for X-ray images, but also provides a standard for object recognition.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It is clear that the present invention can be modified or improved.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

400: X-ray image acquisition unit 402: illegal object detection unit
404: feature extraction unit 406: illegal object classification unit

Claims (10)

(a) in the X-ray image acquiring section, acquiring an X-ray image of the baggage;
(b) at the illegal object detecting section, detecting an illegal object in the X-ray image of the baggage;
(c) extracting features of the illegal object from an X-ray image of the detected illegal object and combining the features to form a feature vector; And
(d) classifying the illegal object using a random forest classifier with the feature vector as an input in an illegal object classifier,
Wherein the step of forming the feature vector comprises:
(c-1) performing a contourlet transformation on an X-ray image of the detected illegal object to obtain a transform coefficient map;
(c-2) extracting Tamura texture features from the transform coefficient map;
(c-3) extracting an image histogram feature from an X-ray image of the detected illegal object; And
(c-4) combining the image histogram features with the Tamura texture features to form the feature vector,
Prior to step (a)
Extracting the Tamura texture features and the image histogram from the X-ray images of a plurality of known illegal objects and extracting the feature vectors by combining the extracted Tamura texture features and the image histogram feature in the feature extraction unit; And
Further comprising, in the illegal object classifier, forming a classification model for the random forest classifier for classifying illegal objects based on the feature vectors,
Wherein the Tamura texture features include a directionality of the X-ray image, a roughness of the X-ray image, and a contrast of the X-ray image,
The step (c-1)
Applying a Laplacian pyramid to the X-ray image of the detected illegal object to decompose the X-ray image of the illegal object into a low-frequency map and a high-frequency map; And
Directionally decomposing the high frequency map using a directional filter bank to obtain a high frequency sub-image to obtain a transform coefficient map,
The roughness (F _crs ) of the X-ray image is calculated by the following equation,

,,

M x N is the size of the image,

Lt;

Represents an intensity value of a pixel at a position (i, j). delete The method according to claim 1,
The step (d)
And classifying the illegal object by applying the feature vector to the classification model in the illegal object classification unit. delete delete An X-ray image obtaining unit for obtaining an X-ray image of the baggage;
An illegal object detecting unit for detecting an illegal object in the X-ray image of the baggage;
A feature extraction unit for extracting features of the illegal object from an X-ray image of the detected illegal object and combining the features to form a feature vector; And
And an illegal object classifier for classifying the illegal object using a random forest classifier with the feature vector as an input,
The feature extraction unit may extract,
Performing a contourlet transform on an X-ray image of the detected illegal object to obtain a transform coefficient map;
Extracting Tamura texture features from the transform coefficient map;
Extracting an image histogram feature from the x-ray image of the detected illegal object; And
Combining the Tamura texture features with the image histogram feature to form the feature vector,
Before the illegal object classifier performs an operation of classifying illegal objects,
The feature extraction unit further extracts the feature vectors from the X-ray image of the known plurality of illegal objects by extracting the Tamura texture features and the image histogram, combining the extracted Tamura texture features and the image histogram feature,
Wherein the illegal object classifier further performs an operation of forming a classification model for the random forest classifier for classifying illegal objects based on the feature vectors,
The Tamura texture features include a directionality of the X-ray image, a roughness of the X-ray image, and a contrast of the X-ray image,
The act of performing a contourlet transform on an X-ray image of the detected illegal object to obtain a transform coefficient map,
Applying a Laplacian pyramid to the X-ray image of the detected illegal object to decompose the X-ray image of the illegal object into a low-frequency map and a high-frequency map; And
Directionally decomposing the high frequency map using a directional filter bank to obtain a high frequency sub-image to obtain a transform coefficient map,
The roughness (F _crs ) of the X-ray image is calculated by the following equation,

,,

M x N is the size of the image,

Lt;

Represents an intensity value of a pixel at a position (i, j) based on a computer vision. delete The method of claim 6,
Wherein the illegal object classifier classifies the illegal object by applying the feature vector to the classification model. delete delete

Images