TW201820260A - All-weather thermal image-type pedestrian detecting method to express the LBP encoding in the same window by HOG as the feature representation, and use SVM and Adaboost to proceed classifier training - Google Patents

Abstract

An all-weather thermal image-type pedestrian detecting method is disclosed, which comprises the following steps: (a) capturing a pedestrian's and a non-pedestrian's daytime thermal image and nighttime thermal image in the same window to establish a sample database of thermal image, wherein the sample database comprises plural pedestrians' and plural non-pedestrians' samples; (b) for the pedestrians' and non-pedestrians' samples in the sample database, proceeding with LBP encoding in the same window, wherein the LBP encodings having complementarity in the same window are treated as the same LBP encoding; (c) expressing the LBP encoding of the same window by the Histogram of Oriented Gradient (HOG) as the feature representation, so as to obtain the feature training samples of the pedestrian sample and the non-pedestrian sample; (d) using SVM (Support Vector Machine) and Adaboost (Adaptive Boosting) to proceed with classifier training; and (e) based on the sliding window method, searching and detecting the thermal image, and determining if there is a pedestrian.

Description

All-weather thermal image type pedestrian detection method

The invention relates to a thermal image pedestrian detection technology, and more particularly to an all-weather thermal image-type pedestrian detection method based on block LBP coding technology.

Previously, the pedestrian detection technology based on thermal images was mainly based on the assumption that the human figure in the thermal image is a high-brightness area. The thermal image was cut using a threshold algorithm to obtain several possible pedestrian areas with high brightness. Human shape samples or feature comparison methods achieve the purpose of thermal image pedestrian detection. However, the effectiveness of such algorithms depends on the selection of thresholds, so they are not suitable for many environmental scenarios and weather conditions. In order to avoid the problem of threshold selection, the current thermal image pedestrian detection uses texture features to describe the appearance of human figures. Given a training database, it contains a large number of human figures and non-human figures. Using machine learning, training one can effectively A classifier that distinguishes between humanoid and non-humanoid, and uses this classifier to directly scan the thermal image to avoid the problem of erroneous cutting caused by threshold selection.

Although the technology of machine learning can avoid cutting problems and deal with issues such as clothing brightness differences (Appothance Variation) and other issues in thermal images, it cannot effectively overcome the problems caused by thermal sensors. Un-Calibrated White-Black Polarity Change. Uncorrected brightness bipolarization is caused by the brightness of the human-shaped area in the thermal image. It is the result of comparison with the ambient temperature. When the ambient temperature is low (for example, dusk and night ), The thermal image human-shaped area is a high-brightness area; conversely, when the ambient temperature is low (for example, noon and afternoon), the thermal image human-shaped area is a low-brightness area. Therefore, the prior art can be effectively applied to a single condition and cannot be effectively applied to all-weather (including day and night) conditions.

The invention aims at the shortcomings of the prior art, and proposes a multi-level machine learning algorithm based on LBP index to achieve the purpose of thermal image pedestrian detection. The proposed algorithm is mainly divided into two phases: Training and Testing. First, a set of thermal image pedestrian database is given, which contains pedestrian samples and non-pedestrian samples. For all defined rectangular blocks (Rectangular Block), using LBP to texture-encode the thermal images of all samples in this block, classify the training block thermal images according to LBP encoding, and then encode the same LBP-encoded training images of the same rectangular block. Histogram of Oriented Gradient (HOG) is used as the feature representation, and the Support Vector Machine (SVM) obtained through learning is used as the block and coded pedestrian and non-pedestrian classifier (Pedestrian / Non-Pedestrian Classifier), and then use all SVMs that belong to the same block as Weak Classifier, and select adaptive rectangular blocks (corresponding to a weak classifier) through adaptive boosting (Adaboost) ) To form a pedestrian classifier, called a strong classifier (Strong Classifier), which is used for subsequent pedestrian detection. Secondly, in the testing phase, the traditional sliding window method is used to convert the pedestrian detection problem into a binary classification problem. For each sliding window, the pedestrian classifier learned above is used to determine the pedestrian classification problem. Whether there are any pedestrians to achieve the purpose of pedestrian detection.

To achieve the above and other objectives, the present invention proposes an all-weather thermal image type pedestrian detection method, including the following steps: (a) capturing daytime thermal images and nighttime thermal images of the same pedestrian and non-pedestrian in a same defined block In order to establish a sample database of thermal images, wherein the sample database includes a plurality of pedestrians and a plurality of non-pedestrian samples; (b) for the pedestrians and the non-pedestrian samples in the sample database, the same LBP encoding in the definition block, where the complementary LBP encoding in the same definition block is the same LBP encoding; (c) the LBP encoding in the same definition block is the gradient direction histogram (HOG) as the Feature representation to obtain the feature training samples of the pedestrian samples and the non-pedestrian samples; (d) input the feature training samples to the SVM and train with Adaboost to form a strong classifier; (e) conduct pedestrian detection Detection, based on the sliding window method, searches and detects the strong classifiers in the thermal image and determines whether they are pedestrians.

In one embodiment of the present invention, the step (b) includes the following steps: (b1) LBP encoding the daytime and nighttime thermal images of the pedestrians and the non-pedestrian samples; (b2) the same A complementary LBP code in a block is defined as the same LBP code.

In an embodiment of the present invention, the step (c) includes the following steps: (c1) dividing the same defined block into a plurality of block regions; (c2) dividing each of the block regions into a plurality of units Area, each unit area has a plurality of LBP codes; (c3) calculate the gradient strength and gradient direction for all LBP codes in each block area; (c4) for all the LBP codes in each unit area, according to its Voting statistics of gradient intensity and gradient direction to obtain the feature vector of each unit area, where the feature vector of each unit area constitutes the HOG feature of each block area, and the HOG features of each block area form the same defined area Block the HOG features to obtain feature training samples for the pedestrian samples and the non-pedestrian samples.

In an embodiment of the present invention, the step (d) includes the following steps: (d1) scanning a plurality of defined areas of different sizes on the entire image; (d2) each of the defined blocks passes through steps (a) ~ (c) Obtaining feature training samples for multiple pedestrian and non-pedestrian samples; (d3) inputting these feature training samples to the SVM for training to obtain multiple weak classifiers; (d4) using Adaboost operations on the weak The classifier finds at least one strong classifier with a key position of the pedestrian.

In an embodiment of the present invention, the step (e) includes the following steps: (e1) scanning the strong classifier in the thermal image based on the sliding window method; (e2) using the block of the strong classifier as LBP Encoding; (e3) encode the LBP with HOG as the feature; (e4) input the HOG feature into the SVM classifier for pedestrian judgment.

Therefore, the all-weather thermal image pedestrian detection method of the multi-layer humanoid classifier based on block LBP coding of the present invention can avoid the uncorrected brightness polarization phenomenon through the above-mentioned LBP texture coding and classification method. The features are too chaotic to affect the recognition results; and Adaboost training classifiers can effectively select and integrate discriminative blocks to overcome misidentification caused by pedestrian appearance and posture variations and obscuration.

In order to fully understand the purpose, features and effects of the present invention, the following specific embodiments are used in conjunction with the accompanying drawings to make a detailed description of the present invention, which will be described later:

As shown in FIG. 1, the all-weather thermal image type pedestrian detection method of the present invention specifically includes steps (a) to (e), where steps (a) to (d) are training stages, and step (e) is Test phase:

Step (a): Acquire daytime thermal images and nighttime thermal images of the same pedestrian and non-pedestrian in the same defined block, and repeatedly retrieve multiple thermal images of pedestrians and non-pedestrians to create a database of thermal images. Therefore, the sample database includes a plurality of pedestrians and a plurality of non-pedestrians during the day and night at night.

Step (b): For the pedestrians and the non-pedestrian samples in the sample database, perform LBP encoding in the same defined block, where the complementary LBP codes in the same defined block are regarded as the same LBP coding to overcome the opposite problem of day and night characteristics of thermal images. Among them, (b1) encode the daytime and nighttime thermal images of these pedestrian and non-pedestrian samples by LBP encoding; (b2) use the complementary LBP encoding in the same definition block as the same LBP encoding. As shown in Figure 2, the daytime LBP code of a person's body part = 124 with night LBP coding = 131 is complementary, and it is set as a complementary feature.

Step (c): The LBP code in the same definition block is represented by HOG as a feature to obtain characteristic training samples of the pedestrian samples and the non-pedestrian samples. Among them, (c1) divides the same defined block (windows) into a plurality of block areas (block); (c2) divides each block area into a plurality of cell areas (cells), each of which has a plurality of block areas (C3) calculate the gradient strength and gradient direction for all LBP codes in each block area; (c4) vote statistics for all LBP codes in each unit area according to their gradient strength and gradient direction The feature vector of each unit area is obtained, wherein the feature vector of each unit area constitutes the HOG feature of each block area, and the HOG features of each block area form the HOG feature of the same defined block, thereby obtaining the Equally pedestrian samples and feature training samples of these non-pedestrian samples. The above HOG is mainly based on the weighted HOG to describe the feature descriptor of the human image local image texture (Feature Descriptor), which has been proven to be an effective human form representation. The HOG feature representation mainly includes the following three steps: (1) Block Definition, (2) calculation of gradient intensity and direction, and (3) histogram statistics. These three steps are explained below:

(1) Block definition: The HOG algorithm will first divide the image screen into a variety of different size and number of window areas. For example, Figure 3 illustrates the window area, block area, unit area, and pixel (pixel) divided by the HOG algorithm. ). The window area 100 is an area composed of a fixed size and number of block areas (for example, block area 110 to block area 140). The block area is a fixed size and number of unit areas (for example, unit area 111 to unit). Area 114). Each block area can overlap (for example, block area 110 and block area 120). The unit area is a fixed size and number of pixels (for example, pixels 1111 and 1112). The composition area, each unit area does not overlap. Taking the block area 110 shown in FIG. 3 as an example, the block area 110 is divided into four unit areas 111 to 114, that is, the unit dimension size is 4 × 4 pixel.

(2) Gradient intensity and direction calculation: for all pixels in the block Through horizontal mask With vertical mask Calculate their horizontal components separately With vertical component ,among them Pixels The brightness. and Gradient intensity With gradient direction The calculations are as in (Formula 1) and (Formula 2):

………………………………. (Formula 1)

…………………………………… (Formula 2)

(3) Histogram statistics: After the above calculations, for all pixels in each unit area, the gradient direction is voted according to its gradient intensity. Generally speaking, the direction is calculated by Is an interval (Bin), so Can be divided into 9 intervals (Bin), pixels The corresponding interval index value is , The voting statistical weight is the gradient intensity of the pixel , So a 9-dimensional feature vector can be obtained for each cell The 9-dimensional feature vector is used to describe the texture characteristics of each unit area. Finally, the feature vectors of the 4 unit areas are combined in a concatenation manner to form a 36-dimensional (9x4 = 36) HOG feature vector. ) . Furthermore, the present invention replaces the pixels in the HOG feature representation with the LBP code obtained in step (b). Therefore, the present invention uses the method of block LBP encoding to extract the features of the thermal image.

Step (d): input the feature training samples to the SVM and train them with Adaboost to form a strong classifier. Among them, (d1) scans a plurality of defined areas of different sizes on the entire image; (d2) each of the defined blocks obtains a plurality of pedestrian samples and non-pedestrian sample characteristic training samples through steps (a) to (c); (d3) input the feature training samples to the SVM for training to obtain a plurality of weak classifiers; (d4) find at least one strong classifier with the key position of the pedestrian in the weak classifiers through the Adaboost operation.

Step (e): Perform pedestrian detection, search and detect the strong classifier in the thermal image based on the sliding window method, and determine whether it is a pedestrian. Among them, (e1) scans the strong classifier in the thermal image based on the sliding window method; (e2) encodes the block of the strong classifier by LBP; (e3) encodes the LBP with HOG as the feature; (e4) ) Enter the HOG feature into the SVM classifier for pedestrian determination. In this way, in the test phase of the all-weather thermal image type pedestrian detection method of the present invention, it is not necessary to perform pedestrian identification for each block area in the thermal image, but only for the area obtained by the Adaboost training in step (d). The strong classifier (that is, a pedestrian classifier) can be scanned, so that the system using the all-weather thermal image type pedestrian detection method of the present invention can reduce the calculation amount.

It should be noted that the above-mentioned LBP coding, SVM and Adaboost classifier training are all quite mature technologies in the field of image recognition technology, so the operation steps will not be described in this specification.

Based on the above, the present invention uses the self-constructed thermal image humanoid database, which mainly includes images in four periods of two days (day, noon, dusk and night) to perform the above-mentioned all-weather thermal image type pedestrian detection method. Four experimental analyses. The four experimental designs are: Experiment 1 (training: all images in the first day and four periods; test: all images in the second day and four periods); Experiment 2 (training: four periods of the second day) All images; test: all images on the first day and four periods); experiment 3 (training: all images during the two days during the day and noon time; test: all images on two days during the dusk and night periods); experiment 4 (training: two days) All images at mid-dusk and night time; test: all images during two days during day and noon time), the experimental data obtained are shown in Table 1 below, where the performance evaluation criteria used in the present invention are accuracy and recall respectively (Recall) and F-Measure, from Table 1 below, it can be clearly found that the accuracy of the all-weather thermal image type pedestrian detection method proposed by the present invention is as high as 98% or more, and the recall rate is also high. At 80%, this can show that the all-weather thermal image pedestrian detection method proposed by the present invention can be effectively applied to all-weather scenes; in addition, the thermal image pedestrian detection result diagram shown in Figure 4 also shows the present invention The all-weather thermal image-type pedestrian detection method can accurately detect thermal image-type human figures.

Table 1. Effectiveness evaluation of the all-weather thermal image type pedestrian detection method of the present invention

Therefore, the all-weather thermal image type pedestrian detection method of the multi-layer humanoid classifier based on block LBP coding of the present invention can avoid the uncorrected brightness polarization phenomenon through the proposed LBP texture coding and classification method. Features are too chaotic to affect the recognition results; and Adaboost training classifiers can effectively select and integrate discriminative blocks to overcome misidentification caused by pedestrian appearance and posture variation and obscuration.

The present invention has been disclosed in the foregoing with a preferred embodiment, but those skilled in the art should understand that this embodiment is only for describing the present invention, and should not be interpreted as limiting the scope of the present invention. It should be noted that all changes and substitutions equivalent to this embodiment should be included in the scope of the present invention. Therefore, the scope of protection of the present invention shall be defined by the scope of the patent application.

(a) ~ (e) ‧‧‧step

100‧‧‧window area

110 ~ 140‧‧‧ Block area

111 ~ 114‧‧‧Unit area

1111 ~ 1112‧‧‧pixels

[Fig. 1] is a detection flowchart of an all-weather thermal image type pedestrian detection method according to an embodiment of the present invention. [Fig. 2] is a diagram for improving the complementary characteristics of LBP coding in an embodiment of the present invention. [Fig. 3] is a block definition diagram of HOG feature extraction in an embodiment of the present invention. [Fig. 4] It is a thermal image type pedestrian detection result diagram using an all-weather thermal image type pedestrian detection method in an embodiment of the present invention.

Claims (5)

An all-weather thermal image type pedestrian detection method includes the following steps: (a) capturing daytime thermal images and nighttime thermal images of the same pedestrian and non-pedestrian in a same defined block to create a sample database of thermal images, The sample database includes a plurality of pedestrians and a plurality of non-pedestrian samples; (b) for the pedestrians and the non-pedestrian samples in the sample database, LBP encoding is performed in the same defined block, where the The complementary LBP codes in the same definition block are regarded as the same LBP codes; (c) the LBP codes in the same definition block are represented by the gradient direction histogram (HOG) as a feature to obtain the pedestrian samples and the Feature training samples such as non-pedestrian samples; (d) Input these feature training samples to SVM and train them with Adaboost to form a strong classifier; (e) Perform pedestrian detection. The strong classifier performs search detection and determines whether it is a pedestrian. The all-weather thermal image type pedestrian detection method described in claim 1, wherein step (b) includes the following steps: (b1) LBP encoding the daytime and nighttime thermal images of the pedestrians and the non-pedestrian samples (B2) Treat the complementary LBP codes in the same definition block as the same LBP codes. The all-weather thermal image type pedestrian detection method as described in claim 1, wherein step (c) includes the following steps: (c1) dividing the same defined block into a plurality of block regions; (c2) dividing each of the regions The block area is divided into a plurality of unit areas, each of which has a plurality of LBP codes; (c3) gradient intensity and gradient direction calculations are performed for all LBP codes in each of the block areas; (c4) for each of the unit areas All LBP codes are voted according to their gradient intensity and gradient direction to obtain the feature vector of each unit area, where the feature vector of each unit area constitutes the HOG feature of each block area, and the The HOG features form the HOG features of the same defined block, so as to obtain the feature training samples of the pedestrian samples and the non-pedestrian samples. The all-weather thermal image type pedestrian detection method as described in claim 1, wherein step (d) includes the following steps: (d1) scanning a plurality of defined areas of different sizes on the entire image; (d2) each of the defined areas The block obtains feature training samples of a plurality of pedestrian samples and non-pedestrian samples through steps (a) ~ (c); (d3) inputs these feature training samples to the SVM for training to obtain a plurality of weak classifiers; (d4) through Adaboost operation finds at least one strong classifier with the key position of the pedestrian among the weak classifiers. The all-weather thermal image type pedestrian detection method as described in claim 4, wherein step (e) includes the following steps: (e1) scanning the strong classifier in the thermal image based on the sliding window method; (e2) The block of the classifier is LBP coded; (e3) the LBP code is represented by HOG as a feature; (e4) the HOG feature is input into the SVM classifier for pedestrian judgment.