CN104143077B - Pedestrian target search method and system based on image - Google Patents

Abstract

An image-based pedestrian target retrieval method, including: obtaining the pedestrian target sequence and the corresponding foreground sequence from the original video image; calculating the weight of the jth frame R _{i, j} of the pedestrian target sequence i, and combining The pixels of the foreground sequence corresponding to all frames are accumulated according to the weights to obtain the absolute area histogram H ₁ and the blurred area histogram H ₂ , calculate the pedestrian target histogram H _f , and use the geodesic distance to calculate the pedestrian target histogram H _f The pedestrian target sequence is sorted according to the size of the geodesic distance. Thereby avoiding the difficult problem of clothing segmentation, and extracting the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, detection based on background segmentation improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval. In addition, an image-based pedestrian target retrieval system is also provided.

Description

Image-based Pedestrian Target Retrieval Method and System

technical field

The invention relates to image processing technology, in particular to an image-based pedestrian target retrieval method and system.

Background technique

With the advent of the Internet age, image retrieval technology has been widely developed and applied. Especially with the development and application of intelligent transportation, image retrieval technology is also applied to intelligent transportation analysis. The widely distributed cameras in modern cities have made traditional traffic analysis and pedestrian and vehicle tracking easier and more convenient. However, due to the large number of surveillance videos, it is often difficult to rely on manpower to track and analyze a large number of surveillance videos in the face of sudden cases or traffic accidents.

Current pedestrian retrieval methods generally use a single detected pedestrian target image to segment the pedestrian target, and then use the segmented pedestrian clothing as features for retrieval. Pedestrian detection technology in static images has made some breakthroughs in recent years. For example, it can deal with general normal pedestrian scenes, but it cannot be very robust in actual video. The accuracy of clothing segmentation has a great impact on retrieval. In other methods, the clothing segmentation of pedestrians has been studied, but due to the diversity of pedestrian poses, the segmentation of clothing cannot be too accurate, or the time cost of too accurate segmentation is too high. There is also the use of moving target edge information in the video to detect and segment pedestrians, but it does not use the pedestrian model to distinguish pedestrians from other moving targets such as cars, but only uses stripe density and other methods to judge, which will have a large misjudgment rate.

Contents of the invention

Based on this, an image-based pedestrian target retrieval method with high detection and low false detection is provided.

An image-based pedestrian target retrieval method, comprising the following steps:

Obtain pedestrian target sequences and corresponding foreground sequences from raw video images;

Accumulate the pixels of the foreground sequence corresponding to all the frames of the pedestrian target sequence i according to the weight to obtain the absolute area histogram H ₁ and the blurred area histogram H ₂ ,

Calculate the pedestrian target histogram H _f , and use the geodesic distance to calculate the distance between the pedestrian target histograms H _f , and sort the pedestrian target sequence according to the size of the geodesic distance; where, x∈{0,1}, x= 0 means grayscale priority, x=1 means color priority; the color components are evenly divided into HBINS intervals, the grayscale value V is less than the threshold Tg is a black area, and the grayscale area [Tg, 1] is evenly divided into VBINS intervals.

In one of the embodiments, the step of obtaining pedestrian target sequence and corresponding foreground sequence from the original video image includes:

Segment the foreground and background of pedestrian targets using a mixture of Gaussian models;

Statistically, the foreground corresponding to the frame contains a window W={w _i } that may contain pedestrian targets, where the foreground area contained in the window accounts for more than half of the window area, and the height of the window is greater than Hmin=60 pixels, and the width is greater than Wmin=30 pixel;

Merge the overlapping parts of the windows;

Use the gradient orientation histogram to detect pedestrian targets in the merged window;

Use the learning-based target tracking method to track the detected pedestrian targets to obtain pedestrian target sequences;

Using the obtained pedestrian target sequence, the foreground sequence is obtained from the original video image and its corresponding background segmentation map.

In one of the embodiments, the step of merging overlapping parts of the windows includes:

Mark any two overlapping windows as the same set S _i ;

To judge the set S _i , if And there is i≠j∧w _i1 ∩w _i2 ≠φ, then merge S _i and S _j ; repeat this step until all sets containing overlapping windows are merged;

Use a rectangle T _i containing all the windows of the set S _i with the smallest area to represent all the windows in the set S _i , and re-form the window set T={T _i }.

In one of the embodiments, the calculation steps of the absolute region histogram _H1 and the blurred region histogram _H2 include:

Convert pedestrian target image to HSV color space;

If the gray value of the pedestrian target image pixel V<Tg, then H ₁ [1]=H ₁ [1]+1;

If the pixel gray value V≥Tg and the saturation value S _g < S < S _c , then calculate the fuzzy area histogram H ₂ ; wherein, the fuzzy area histogram calculation includes the color part and the gray part:

In one of the embodiments, the steps of accumulating the pixels of the foreground sequence corresponding to all the frames of the pedestrian target sequence i according to the weight, and obtaining the absolute region histogram _H1 and the blurred region histogram _H2 include:

Set the absolute area histogram H ₁ and the fuzzy area histogram H ₂ to zero;

Calculate the inscribed ellipse of the j-th frame foreground F _i,j of the pedestrian target sequence i, and set the part between the inscribed ellipse and the rectangle as the background;

Accumulate the foreground pixels corresponding to the jth frame R _i,j of the pedestrian target sequence i to the absolute area histogram H ₁ and the blurred area histogram H ₂ of the jth frame R _i,j according to the weight value, and the accumulated value is weight(j), H ₁ [index]=H ₁ [index]+weight(j) or H ₂ [index]=H ₂ [index]+weight(j), until all frames in the pedestrian target sequence are calculated ;

Save the absolute area histogram H ₁ and the fuzzy area histogram H ₂ as the color feature histogram of sequence i.

The above image-based pedestrian target retrieval method obtains robust pedestrian detection results through learning-based target tracking and foreground segmentation, and then obtains pedestrian target sequences and foreground sequences. Then obtain the color feature histogram of the pedestrian target sequence through calculation, that is, perform feature extraction and matching on the clothing color of the pedestrian target, and finally calculate the pedestrian target histogram H _f according to the formula, and use the geodesic distance to obtain the pedestrian target histogram H _f . The pedestrian target sequence is sorted according to the size of the geodesic distance. In this way, retrieval results with correlation rankings can be obtained. The above-mentioned image-based pedestrian target retrieval method avoids the difficult problem of segmenting clothing, but extracts the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, background segmentation-based pedestrian detection improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval.

In addition, an image-based pedestrian object retrieval system with high detection and low false detection is provided.

An image-based pedestrian target retrieval system, characterized in that it includes an image acquisition module, a pedestrian sequence multi-histogram feature calculation module, a pedestrian sequence feature calculation module and a geodesic distance sorting module; the pedestrian sequence multi-histogram feature calculation module Respectively connected to the image acquisition module and the pedestrian sequence feature calculation module, the pedestrian sequence feature calculation module is also connected to the geodesic distance sorting module;

The image acquisition module is used to acquire pedestrian target sequences and corresponding foreground sequences from original video images;

The pedestrian sequence multi-histogram feature calculation module is used to adopt the formula

The pedestrian sequence multi-histogram feature calculation module is also used to accumulate the pixels of the foreground sequence corresponding to all the frames of the pedestrian target sequence i according to the weights to obtain the absolute area histogram _H1 and the blurred area histogram _H2 ,

The pedestrian sequence feature calculation module is used to calculate according to the formula

Calculate pedestrian target histogram H _f ; among them, x∈{0,1}, x=0 means gray priority, x=1 means color priority; the color components are evenly divided into HBINS intervals, and the gray value V is less than the threshold Tg is The black area, the gray area [Tg,1] is evenly divided into VBINS intervals;

The geodesic distance sorting module is used to calculate the distance between the pedestrian target histograms H _f using the geodesic distance, and sort the pedestrian target sequence according to the size of the geodesic distance.

In one of the embodiments, the image acquisition module includes a foreground segmentation module, a pedestrian target detection module and a pedestrian target sequence extraction module;

The foreground segmentation module is respectively connected with the pedestrian target detection module and the pedestrian target sequence extraction module;

The foreground segmentation module is used to segment the foreground and background of each frame pedestrian target by using a Gaussian mixture model;

The pedestrian target detection module includes a pedestrian position preliminary estimation module and a pedestrian detection module based on HOG;

The pedestrian position preliminary estimation module is used to count the window W={w _i } that may contain pedestrian targets in the foreground template window corresponding to the frame, wherein the foreground area contained in the window accounts for more than half of the window area, and the height of the window is Greater than Hmin=60 pixels, width greater than Wmin=30 pixels, and overlapped windows are merged according to the following steps:

Mark any two overlapping windows as the same set S _i ;

To judge the set S _i , if And there is i≠j∧w _i1 ∩w _i2 ≠φ, then merge S _i and S _j ; repeat the judgment until all sets containing overlapping windows are merged;

A rectangle T _i containing all the windows of the set S _i with the smallest area is used to represent all the windows in the set S _i , and the window set T={T _i } is re-formed.

The HOG-based pedestrian detection module uses HOG features to detect pedestrians at the initially estimated pedestrian positions.

In one of the embodiments, the pedestrian target sequence extraction module includes a pedestrian tracking module and a foreground sequence acquisition module;

The pedestrian tracking module is used to track the detected pedestrian target by using the pedestrian tracking technology (TLD) based on the learning method, and obtain the pedestrian target sequence;

The foreground sequence acquisition module is used to extract the corresponding foreground sequence in the pedestrian target sequence;

In one of the embodiments, the pedestrian sequence multi-histogram feature calculation module includes a color space conversion module and a single-frame image color histogram calculation module;

The color space conversion module is used to convert the pedestrian target image to the HSV color space;

The single frame image color histogram calculation module includes the following calculation steps:

If the gray value of the pedestrian target image pixel V<Tg, then H ₁ [1]=H ₁ [1]+1;

If the pixel gray value V≥Tg and the saturation value S _g < S < S _c , then calculate the fuzzy area histogram H ₂ ; wherein, the fuzzy area histogram calculation includes the color part and the gray part:

In one of the embodiments, the pedestrian sequence multi-histogram feature calculation module also includes a pedestrian sequence color histogram accumulation calculation module, and the pedestrian sequence color histogram accumulation calculation module includes the following calculation steps:

Set the absolute area histogram H ₁ and the fuzzy area histogram H ₂ to zero;

Calculate the inscribed ellipse of the j-th frame foreground F _i,j of the pedestrian target sequence i, and set the part between the inscribed ellipse and the rectangle as the background;

Accumulate the foreground pixels corresponding to the jth frame R _i,j of the pedestrian target sequence i to the absolute area histogram H ₁ and the blurred area histogram H ₂ of the jth frame R _i,j according to the weight value, and the accumulated value is weight(j), H ₁ [index]=H ₁ [index]+weight(j) or H ₂ [index]=H ₂ [index]+weight(j), until all frames in the pedestrian target sequence are calculated ;

Save the absolute area histogram H ₁ and the fuzzy area histogram H ₂ as the color feature histogram of sequence i.

The above-mentioned image-based pedestrian target retrieval system obtains robust pedestrian detection results through learning-based target tracking and foreground segmentation, and then obtains pedestrian target sequences and foreground sequences. Then obtain the color feature histogram of the pedestrian target sequence through calculation, that is, perform feature extraction and matching on the clothing color of the pedestrian target, and finally calculate the pedestrian target histogram H _f according to the formula, and use the geodesic distance to obtain the pedestrian target histogram H _f . The pedestrian target sequence is sorted according to the size of the geodesic distance. In this way, retrieval results with correlation rankings can be obtained. The above-mentioned image-based pedestrian target retrieval system avoids the difficult problem of segmenting clothing, but extracts the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, background segmentation-based pedestrian detection improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval.

Description of drawings

Fig. 1 is the flowchart of image-based pedestrian target retrieval method;

Figure 2(a) is a schematic diagram of the acquired pedestrian target sequence;

Figure 2(b) is a schematic diagram of the foreground sequence corresponding to the pedestrian target sequence;

Figure 2(c) is a schematic diagram of inscribed ellipse de-interference;

The distribution schematic diagram of Fig. 3 color color histogram;

Fig. 4 is a schematic diagram of weight function;

Fig. 5 is a schematic diagram of an image-based pedestrian target retrieval system.

Detailed ways

As shown in Figure 1, it is a flowchart of an image-based pedestrian target retrieval method.

An image-based pedestrian target retrieval method, comprising the following steps:

Step S110, acquiring pedestrian target sequences and corresponding foreground sequences from the original video images. Specifically, pedestrian targets in the video are detected, the detected pedestrian targets are tracked to obtain pedestrian sequences, and the foreground sequences are obtained from the original video image and its corresponding background segmentation map according to the position of the pedestrian target sequence. As shown in Figure 2(a), it is the acquired pedestrian target sequence, and Figure 2(b) is the foreground sequence corresponding to the pedestrian target sequence.

Step S110 specifically includes:

①Using a mixed Gaussian model to segment the foreground and background of pedestrian targets.

The mixed Gaussian model uses K (basically 3 to 5) Gaussian models to characterize the characteristics of each pixel in the image, and updates the mixed Gaussian model after a new frame of image is obtained, using each pixel in the current image and the mixed Gaussian Model matching, if successful, it is determined that the point is a background point, otherwise it is a foreground point.

② Preliminary estimation of possible pedestrian positions, specifically, includes the following steps:

⑴Statistically, the frame corresponds to the window W={w _i } that may contain pedestrian targets in the foreground, where the foreground area contained in the window accounts for more than half of the window area, and the height of the window is greater than Hmin=60 pixels, and the width is greater than Wmin=30 pixels. The single-frame image for pedestrian target detection is generally extracted from multiple frames of images, preferably, extracted from 15 frames of images.

(2) Mark any two overlapping windows as the same set S _i .

(3) if And there is i≠j∧w _i1 ∩w _i2 ≠φ, then merge S _i and S _j ; repeat this step until all sets containing overlapping partial windows are merged.

(4) Use a rectangle T _i containing all the windows of the set S _i with the smallest area to represent all the windows in the set S _i , and re-form the window set T={T _i }.

③ Use the gradient orientation histogram to detect pedestrian targets in the merged window.

The histogram of image gradient orientation is an image descriptor for human target detection. This method uses the Histogram of Oriented Gradients (HOG) feature to express the human body, extracts the shape information and motion information of the human body, and forms a rich feature set. In this embodiment, the gradient direction histogram is used to detect pedestrian targets included in the combined window. The classifier threshold used was -0.5.

④A learning-based target tracking method is used to track the detected pedestrian targets to obtain pedestrian target sequences.

Using the Tracking-Learning-Detection (TLD) algorithm, the target can be tracked continuously for a long time, and the target in the dynamic image sequence can be tracked. TLD can continuously track the target, even if the tracking fails under visible light, it can be compensated by the infrared image, so that the tracking effect is more accurate.

Using the acquired pedestrian target sequence, the pedestrian target sequence and the foreground sequence are respectively obtained from the original video image and its corresponding background segmentation image.

Step S130: Accumulate the pixels of the foreground sequence corresponding to all frames of the pedestrian target sequence i according to the weights to obtain the absolute area histogram H ₁ and the blurred area histogram H ₂ , and the accumulated absolute area histogram H ₁ and the fuzzy area histogram H ₂ is recorded as the color feature histogram of the pedestrian target sequence i.

Convert the color image from the RGB space to the HSV space. According to the visual characteristics of the human eye, the color color is divided into 3 areas according to the color saturation value, the color area with saturation greater than S _c , and the gray area with saturation less than S _g and saturation at The fuzzy area between, the color area and the grayscale area are collectively referred to as the accurate color area. For the color area, only the color component H value is considered, for the grayscale area, only the grayscale component V value is considered, and for the blurred area, both the color component and the grayscale component value are considered; in addition, for the grayscale whose grayscale value is too small The component values are all treated as black in grayscale.

The RGB color mode is a color standard in the industry. It obtains a variety of colors by changing the three color channels of red (R), green (G), and blue (B) and superimposing them with each other. of.

HSV (also called HSB) is two related representations of points in the RGB color space. It describes more accurate perceptual color connections than RGB, and is simple to calculate. H refers to hue (hue), S refers to saturation (saturation), L refers to lightness (brightness), V refers to value (hue), and B refers to brightness (brightness).

Hue (H) is the basic attribute of color, which is commonly referred to as the color name, such as red, yellow, etc.

Saturation (S) refers to the purity of the color, the higher the color, the purer the color, and the lower it will gradually become gray, taking the value of 0-100%. Hue (V), brightness (L) range from 0-100%.

HSV describes colors as points in a cylindrical coordinate system. The central axis of this cylinder takes values from black at the bottom to white at the top and gray in between. The angle around this axis corresponds to the "hue", to this axis The distance corresponds to "saturation", while the height along this axis corresponds to "brightness", "hue" or "lightness".

HSV (Hue, Saturation, Hue) can be conceptually thought of as an inverted cone of color (black point at the lower apex, white at the center of the upper base). Because HSV is a simple device-dependent transformation of RGB, the colors defined by (h,s,l) or (h,s,v) triplets depend on the particular red, green, and blue "additive primaries" being used. Each unique RGB device is accompanied by a unique HSV space. But (h, s, l) or (h, s, v) triplets are constrained to a specific RGB space. For example, when sRGB becomes clear.

The HSV model is a nonlinear transformation of the three primary color light modes.

The color components in the HSV space are evenly divided into HBINS intervals, preferably, the color space 0-360 degrees is evenly divided into HBINS=100 intervals. A gray value V less than Tg=0.05 is a black area, and the gray area [Tg, 1] is evenly divided into VBINS intervals, preferably, VBINS=100. The temporary color histogram of the image includes two parts: the absolute region histogram H ₁ corresponding to the accurate color region, whose length is 1+VBINS+HBINS; and the fuzzy region histogram H ₂ of the region corresponding to the blurred color, whose length is VBINS +HBINS, as shown in Figure 3. The grayscale area in the absolute area histogram _H1 of the accurate color area is 1+VBINS, the color area HBINS is colored, and the colors are arranged in the order of red, yellow, green, blue and red. In the color intersection part, according to each color The proportion forms the new color. The gray area of the fuzzy area histogram _H2 of the fuzzy color area is VBINS, and the color arrangement of the color area HBINS is consistent. The absolute area histogram H ₁ of the accurate color area and the final color histogram H _f are the sum of the absolute area histogram H ₁ and the fuzzy area histogram H ₂ , and its color distribution is consistent with the histogram distribution of the accurate color area, and the length is 1+VBINS+HBINS. The final color histogram is defined as:

Among them, x∈{0,1}, x=0 means grayscale priority, and x=1 means color priority.

After the color histogram is defined, the absolute area histogram H ₁ and the fuzzy area histogram H ₂ are calculated. The specific calculation steps are:

① Convert the pedestrian target image to HSV color space.

② If the gray value V<Tg of the pedestrian target image pixel, then H ₁ [1]=H ₁ [1]+1.

⑤ If the pixel gray value V≥Tg and the saturation value S _g < S < S _c , calculate the fuzzy area histogram H ₂ ; wherein, the calculation of the fuzzy area histogram includes the color part and the gray scale part.

Preferably, the thresholds of color regions S _c =0.15, S _g =0.05

Based on the absolute area histogram H ₁ and the fuzzy area histogram H ₂ , the calculation steps of the color feature histogram H _f are:

① Set the absolute area histogram H ₁ and the fuzzy area histogram H ₂ to zero.

② Take the i-th pedestrian target sequence and the corresponding foreground sequence Where N _i is the sequence length of the i-th pedestrian target sequence; use the following formula to calculate the weight f corresponding to each frame image of the pedestrian target sequence, and weight the pedestrian target sequence i, taking T=N _i *2/3. As shown in Figure 4, it is a schematic diagram of the function of T and the cumulative value weight(j).

Calculate the inscribed ellipse of the foreground F _i,j of the jth frame of the pedestrian target sequence, and set the part between the inscribed ellipse and the rectangle as the background. The interference in the image can be removed by using an inscribed ellipse, as shown in Figure 2(c).

③Accumulate the foreground pixels corresponding to the jth frame R _i,j of the pedestrian target sequence i to the absolute area histogram H ₁ and the blurred area histogram H ₂ of the jth frame R _i,j according to the weight, and accumulate The value is weight(j), H ₁ [index]=H ₁ [index]+weight(j) or H ₂ [index]=H ₂ [index]+weight(j), until all frames in the pedestrian target sequence are calculated Finish.

④ Save the absolute area histogram H ₁ and the fuzzy area histogram H ₂ as the color feature histogram of sequence i.

Step S140, according to the formula

Calculate the pedestrian target histogram H _f , and use the geodesic distance to calculate the distance between the pedestrian target histograms H _f , and sort the pedestrian target sequence according to the size of the geodesic distance; where, x∈{0,1}, x= 0 means grayscale priority, x=1 means color priority; the color components are evenly divided into HBINS=100 intervals, the grayscale value V is less than the threshold Tg=0.05 is a black area, and the grayscale area [Tg,1] is evenly divided into VBINS= 10 intervals.

The geodesic distance EMD (Earth Mover's Distance) is used to calculate the distance between histograms. When the distance between features (bin and bin) can be obtained using ground distance, using Earth Mover's Distance to do similar calculations can get more accurate results .

The above image-based pedestrian target retrieval method obtains robust pedestrian detection results through learning-based target tracking and foreground segmentation, and then obtains pedestrian target sequences and foreground sequences. Then obtain the color feature histogram of the pedestrian target sequence through calculation, that is, perform feature extraction and matching on the clothing color of the pedestrian target, and finally calculate the pedestrian target histogram H _f according to the formula, and use the geodesic distance to obtain the pedestrian target histogram H _f . The pedestrian target sequence is sorted according to the size of the geodesic distance. In this way, retrieval results with correlation rankings can be obtained. The above-mentioned image-based pedestrian target retrieval method avoids the difficult problem of segmenting clothing, but extracts the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, background segmentation-based pedestrian detection improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval.

As shown in Figure 5, an image-based pedestrian target retrieval system includes an image acquisition module 500, a pedestrian sequence multi-histogram feature calculation module 540, a pedestrian sequence feature calculation module 550 and a geodesic distance sorting module 560; the pedestrian sequence The multi-histogram feature calculation module 540 is respectively connected to the image acquisition module 500 and the pedestrian sequence feature calculation module 550 , and the pedestrian sequence feature calculation module 550 is also connected to the geodetic distance sorting module 560 .

The image acquisition module 500 is used to acquire pedestrian target sequences and corresponding foreground sequences from original video images.

The pedestrian sequence multi-histogram feature calculation module 540 is used to adopt the formula

The pedestrian sequence multi-histogram feature calculation module 540 is also used to accumulate the pixels of the foreground sequence corresponding to all the frames of the pedestrian target sequence i according to the weights to obtain the absolute area histogram _H1 and the blurred area histogram H ₂ ,

The pedestrian sequence feature calculation module 550 is used to calculate according to the formula

Calculate pedestrian target histogram H _f ; among them, x∈{0,1}, x=0 means gray priority, x=1 means color priority; the color components are evenly divided into HBINS intervals, and the gray value V is less than the threshold Tg is The black area, the gray area [Tg,1] is evenly divided into VBINS intervals;

The geodesic distance sorting module 560 is used to calculate the distance between the pedestrian target histograms _Hf using the geodesic distance, and sort the sequence of pedestrian targets according to the size of the geodesic distance.

The image acquisition module 500 includes a foreground segmentation module 510 , a pedestrian target detection module 520 and a pedestrian target sequence extraction module 530 .

The foreground segmentation module 510 is connected to the pedestrian target detection module 520 and the pedestrian target sequence extraction module 530 respectively.

The foreground segmentation module 510 is used to segment the foreground and background of pedestrian targets in each frame by using a Gaussian mixture model.

The pedestrian target detection module 520 includes a pedestrian position preliminary estimation module 522 and a pedestrian detection module 524 based on HOG.

The pedestrian position preliminary estimation module 522 is used to count the window W={w _i } that may contain pedestrian targets in the foreground template window corresponding to the frame, wherein the foreground area contained in the window accounts for more than half of the window area, and the window's The height is greater than Hmin=60 pixels, the width is greater than Wmin=30 pixels, and the overlapping windows are merged according to the following steps:

Mark any two overlapping windows as the same set S _i ;

To judge the set S _i , if And there is i≠j∧w _i1 ∩w _i2 ≠φ, then merge S _i and S _j ; repeat the judgment until all sets containing overlapping windows are merged;

A rectangle T _i containing all the windows of the set S _i with the smallest area is used to represent all the windows in the set S _i , and the window set T={T _i } is re-formed.

The HOG-based pedestrian detection module 524 uses HOG features to perform pedestrian detection at the initially estimated pedestrian position.

The pedestrian target sequence extraction module 530 includes a pedestrian tracking module 532 and a foreground sequence acquisition module 534 .

The pedestrian tracking module 532 is used to track the detected pedestrian targets by using the pedestrian tracking technology (TLD) based on the learning method, and acquire pedestrian target sequences;

The foreground sequence acquisition module 534 is used to extract the corresponding foreground sequence in the pedestrian target sequence.

The pedestrian sequence multi-histogram feature calculation module 540 includes a color space conversion module 542 and a single frame image color histogram calculation module 544 .

The color space conversion module 542 is used to convert the pedestrian object image into HSV color space.

The single-frame image color histogram calculation module 544 includes the following calculation steps:

If the gray value of the pedestrian target image pixel V<Tg, then H ₁ [1]=H ₁ [1]+1;

If the pixel gray value V≥Tg and the saturation value S _g < S < S _c , then calculate the fuzzy area histogram H ₂ ; wherein, the fuzzy area histogram calculation includes the color part and the gray part:

The pedestrian sequence multi-histogram feature calculation module 540 also includes a pedestrian sequence color histogram accumulation calculation module 546, and the pedestrian sequence color histogram accumulation calculation module 546 includes the following calculation steps:

Set the absolute area histogram H ₁ and the fuzzy area histogram H ₂ to zero;

Calculate the inscribed ellipse of the j-th frame foreground F _i,j of the pedestrian target sequence i, and set the part between the inscribed ellipse and the rectangle as the background;

Accumulate the foreground pixels corresponding to the jth frame R _i,j of the pedestrian target sequence i to the absolute area histogram H ₁ and the blurred area histogram H ₂ of the jth frame R _i,j according to the weight value, and the accumulated value is weight(j), H ₁ [index]=H ₁ [index]+weight(j) or H ₂ [index]=H ₂ [index]+weight(j), until all frames in the pedestrian target sequence are calculated ;

Save the absolute area histogram H ₁ and the fuzzy area histogram H ₂ as the color feature histogram of sequence i.

The image-based pedestrian target retrieval system extracts the pedestrian targets in the video and extracts the features of the pedestrian targets to quickly retrieve pedestrian targets with similar features in this video or other videos. The Histogram of Oriented Gradient (HOG) features of pedestrians are extracted, and then these features are used to distinguish pedestrians from non-pedestrians. Pedestrians are moving in the video, and the general pedestrian detection algorithm (such as HOG) can be implemented more robustly in the video by using this effective information.

The above-mentioned image-based pedestrian target retrieval system obtains robust pedestrian detection results through learning-based target tracking and foreground segmentation, and then obtains pedestrian target sequences and foreground sequences. Then obtain the color feature histogram of the pedestrian target sequence through calculation, that is, perform feature extraction and matching on the clothing color of the pedestrian target, and finally calculate the pedestrian target histogram H _f according to the formula, and use the geodesic distance to obtain the pedestrian target histogram H _f . The pedestrian target sequence is sorted according to the size of the geodesic distance. In this way, retrieval results with correlation rankings can be obtained. The above-mentioned image-based pedestrian target retrieval system avoids the difficult problem of segmenting clothing, but extracts the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, background segmentation-based pedestrian detection improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval.

The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.

Claims (10)

1. a kind of pedestrian target search method based on image, which is characterized in that include the following steps：

Pedestrian target sequence and corresponding prospect sequence are obtained from raw video image；

Using formulaCalculate every frame image R of the pedestrian target sequence i_i,jPower Value, T=N_i*2/3；N_iFor the sequence length of i-th of pedestrian target sequence, T is sequence length；

The pixel of the corresponding prospect sequence of all frames of pedestrian target sequence i according to the weights is added up, is obtained exhausted To region histogram H₁With fuzzy region histogram H₂,

According to formula

Calculate pedestrian target histogram H_f, and calculate pedestrian target histogram H using geodesic distance_fThe distance between, according to geodetic The size of distance is ranked up pedestrian target sequence；Wherein, x ∈ { 0,1 }, x=0 are gradation preference, and x=1 represents colored excellent First；Color component is uniformly divided into HBINS interval, and it is black region that gray value V, which is less than threshold value Tg, and gray areas [Tg, 1] is uniform It is divided into VBINS interval.

2. the pedestrian target search method according to claim 1 based on image, which is characterized in that from raw video image The step of middle acquisition pedestrian target sequence and corresponding prospect sequence, includes：

Using the foreground and background of mixed Gauss model segmentation pedestrian target；

Count the window W={ w for corresponding to and including in prospect there are pedestrian target in window in the frame_i, wherein, it is included in window Foreground area account for window area more than half, and window is tall and big in Hmin=60 pixels, is wider than Wmin=30 pixels；

The intersection of window is merged；

Using the windows detecting pedestrian target of the method based on gradient orientation histogram (HOG) after merging；

Pedestrian target sequence is obtained into line trace to the pedestrian target detected using based on the method for tracking target of study；

Using the pedestrian target sequence got, prospect sequence is obtained in the background segment figure corresponding from original video picture Row.

3. the pedestrian target search method according to claim 2 based on image, which is characterized in that the weight by window The step of part merges is closed to include：

It is same set S by the window indicia that any two has coincidence_i；

Judge set S_iIfw_i2∈S_j, and have i ≠ j ∧ w_i1∩w_i2≠ φ, then merge S_iWith S_j；Repeat this Step is until all set for including intersection window are all merged；

With area it is minimum include set S_iThe rectangle T of all windows_iTo represent set S_iIn all window, shape again Into window set T={ T_i}。

4. the pedestrian target search method according to claim 1 based on image, which is characterized in that the absolute region is straight Side figure H₁With fuzzy region histogram H₂Calculating step include：

Pedestrian target image is transformed into hsv color space；

If gray value V the ＜ Tg, H of pedestrian target image pixel₁[1]=H₁[1]+1；

If the gray value V >=Tg and intensity value S≤S of pedestrian target image pixel_g, then the gray value of pixel is calculated, is indexedH₁[index]=H₁[index]+1；

If pedestrian target image pixel gray level value V >=Tg and intensity value S >=S_c, then the color value of pixel is calculated, is indexedH₁[index]=H₁[index]+1；

If grey scale pixel value V >=Tg and intensity value S_g＜ S ＜ S_c, then fuzzy region histogram H is calculated₂；Wherein, it obscures Region histogram calculating includes color part and gray portion：

Gray portion：H₂[index]=H₂[index]+1；

Chrominance section：H₂[index]=H₂[index]+1。

5. the pedestrian target search method according to claim 1 based on image, which is characterized in that described by pedestrian target The pixel of the corresponding prospect sequence of all frames of sequence i adds up according to the weights, obtains absolute region histogram H₁ With fuzzy region histogram H₂The step of include：

By absolute region histogram H₁With fuzzy region histogram H₂Zero setting；

Calculate the jth frame prospect F of pedestrian target sequence i_i,jInner ellipse, the back of the body will be partly set between inner ellipse and rectangle Scape；

By the jth frame R of pedestrian target sequence i_i,jCorresponding foreground part pixel carries out being added to jth frame R according to the weights_i,j Absolute region histogram H₁With fuzzy region histogram H₂, accumulated value is weight (j), H₁[index]=H₁[index]+ Weight (j) or H₂[index]=H₂[index]+weight (j), until all frames all calculate completion in pedestrian target sequence；

Preserve absolute region histogram H₁With fuzzy region histogram H₂Color characteristic histogram for sequence i.

6. a kind of pedestrian target searching system based on image, which is characterized in that how straight including image collection module, pedestrian's sequence Square figure feature calculation module, pedestrian's sequence signature computing module and geodesic distance sorting module；The more histograms of pedestrian's sequence Feature calculation module is connect respectively with described image acquisition module and pedestrian's sequence signature computing module, pedestrian's sequence Feature calculation module is also connect with the geodesic distance sorting module；

Described image acquisition module is used to obtain pedestrian target sequence and corresponding prospect sequence from raw video image；

The more histogram feature computing modules of pedestrian's sequence are used for using formula

Calculate every frame image R of the pedestrian target sequence i_i,jWeights, T=N_i* 2/3；N_iFor the sequence length of i-th of pedestrian target sequence, T is sequence length；

The more histogram feature computing modules of pedestrian's sequence are additionally operable to the corresponding prospect sequence of all frames of pedestrian target sequence i The pixel of row adds up according to the weights, obtains absolute region histogram H₁With fuzzy region histogram H₂,

Pedestrian's sequence signature computing module is used for according to formula

Calculate pedestrian target histogram H_f；Wherein, x ∈ { 0,1 }, x=0 are gradation preference, and x=1 represents colored preferential；Color point Amount is uniformly divided into HBINS interval, and it is black region that gray value V, which is less than threshold value Tg, and gray areas [Tg, 1] is uniformly divided into VBINS A interval；

The geodesic distance sorting module is used to calculate pedestrian target histogram H using geodesic distance_fThe distance between, according to survey The size of ground distance is ranked up pedestrian target sequence.

7. the pedestrian target searching system according to claim 6 based on image, which is characterized in that described image obtains mould Block includes foreground segmentation module, pedestrian target detection module and pedestrian target sequential extraction procedures module；

The foreground segmentation module is connect respectively with the pedestrian target detection module and the pedestrian target sequential extraction procedures module；

The foreground segmentation module is used for using foreground and background of the mixed Gauss model segmentation per frame pedestrian target；

The pedestrian target detection module includes pedestrian position module and the pedestrian detection module based on HOG according to a preliminary estimate；

Module is interior comprising there are pedestrian targets for counting the corresponding foreground template window of the frame according to a preliminary estimate for the pedestrian position Window W={ w_i, wherein, the foreground area included in window account for window area more than half, and window is tall and big in Hmin =60 pixels are wider than Wmin=30 pixels, and there will be the windows of coincidence to merge in accordance with the following steps：

It is same set S by the window indicia that any two has coincidence_i；

Judge set S_iIfw_i2∈S_j, and have i ≠ j ∧ w_i1∩w_i2≠ φ, then merge S_iWith S_j；Repetition is sentenced Break until all set for including intersection window are all merged；

By area it is minimum include set S_iThe rectangle T of all windows_iTo represent set S_iIn all window, shape again Into window set T={ T_i}；

The pedestrian detection module based on HOG is that pedestrian is carried out at the pedestrian position obtained according to a preliminary estimate using HOG features Detection.

8. the pedestrian target searching system according to claim 6 based on image, which is characterized in that the pedestrian target sequence Row extraction module includes pedestrian tracking module and prospect retrieval module；

The pedestrian tracking module is used for using the pedestrian tracking technology (TLD) based on learning method to the pedestrian target that detects Into line trace, pedestrian target sequence is obtained；

The prospect retrieval module is used to extract corresponding prospect sequence in pedestrian's target sequence.

9. the pedestrian target searching system according to claim 6 based on image, which is characterized in that pedestrian's sequence is more Histogram feature computing module includes color space conversion module and single-frame images color histogram computing module；

The color space conversion module is used to pedestrian target image being transformed into hsv color space；

The single-frame images color histogram computing module includes following calculating step：

If gray value V the ＜ Tg, H of pedestrian target image pixel₁[1]=H₁[1]+1；

If the gray value V >=Tg and intensity value S≤S of pedestrian target image pixel_g, then the gray value of pixel is calculated, is indexedH₁[index]=H₁[index]+1；

If pedestrian target image pixel gray level value V >=Tg and intensity value S >=S_c, then the color value of pixel is calculated, is indexedH₁[index]=H₁[index]+1；

If grey scale pixel value V >=Tg and intensity value S_g＜ S ＜ S_c, then fuzzy region histogram H is calculated₂；Wherein, it obscures Region histogram calculating includes color part and gray portion：

Gray portion：H₂[index]=H₂[index]+1；

Chrominance section：H₂[index]=H₂[index]+1。

10. the pedestrian target searching system according to claim 6 based on image, which is characterized in that pedestrian's sequence More histogram feature computing modules further include pedestrian's sequential color histogram accumulation computing module, pedestrian's sequential color Nogata Figure accumulation computing module includes step is calculated as below：

By absolute region histogram H₁With fuzzy region histogram H₂Zero setting；

Calculate the jth frame prospect F of pedestrian target sequence i_i,jInner ellipse, the back of the body will be partly set between inner ellipse and rectangle Scape；

By the jth frame R of pedestrian target sequence i_i,jCorresponding foreground part pixel carries out being added to jth frame R according to the weights_i,j Absolute region histogram H₁With fuzzy region histogram H₂, accumulated value is weight (j), H₁[index]=H₁[index]+ Weight (j) or H₂[index]=H₂[index]+weight (j), until all frames all calculate completion in pedestrian target sequence；

Preserve absolute region histogram H₁With fuzzy region histogram H₂Color characteristic histogram for sequence i.