CN103049788A - Computer-vision-based system and method for detecting number of pedestrians waiting to cross crosswalk - Google Patents

Abstract

The invention relates to a system and method for detecting the number of pedestrians waiting to pass in a crosswalk based on computer vision. The system includes two CCD cameras and a computer image processing system. The computer image processing system is connected to the camera through an interface. At both ends, the field of view angles of the two CCD cameras include the opposite waiting pedestrian area and the diagonally opposite vehicle area; the detection method is to obtain the image in the video image, preprocess the pedestrian waiting area, and then obtain the foreground through the Gaussian mixture background model method , and obtain the vertical integral projection map, process it and information statistics, and finally obtain the number of pedestrians waiting to pass the crosswalk. Compared with the existing method for detecting the number of pedestrians, the present invention has the advantages of simplicity and high efficiency, so that the method can also be used in complex situations.

Description

System and method for detecting the number of pedestrians waiting to pass based on computer vision

technical field

The invention relates to the technical field of computer vision, in particular to a method for detecting the number of pedestrians waiting to pass in a crosswalk based on computer vision.

Background technique

At present, in the field of intelligent transportation and computer vision, the detection and analysis of pedestrians is an important part. Pedestrian detection and analysis technology has been studied for more than ten years, but there is still no standard, accurate, high-performance, and real-time pedestrian detection and analysis algorithm. Due to some inherent characteristics of pedestrians, the complexity of application scenarios, and the interaction between people or people and the environment, the detection and analysis of pedestrians is the most difficult challenge in the field of computer vision research.

In the past ten years, many existing detection methods have been produced when pedestrian detection technology has received extensive attention and research from academia and engineering. For example, Haritaoglu Gavrila et al. use human body contour features to detect pedestrians , since the human body presents a certain symmetry on the coordinates of the central axis of the body, the projection histogram of the target contour in a certain area in the horizontal and vertical directions can be calculated, and the symmetry can be analyzed to determine whether the target is a pedestrian. Rivlin, Senior et al. used an ellipse to match the target after motion segmentation. The major and minor axes of the ellipse and their length ratios, as well as the angle formed between the major and minor axes in the image plane coordinate system, can be used as shape features to classify pedestrians. Lipton et al. defined the ratio of the square of the perimeter of the moving target edge to the area as the degree of dispersion, and used this feature to distinguish objects such as pedestrians and cars. Collins et al. combined the above parameters and used the area, aspect ratio, and dispersion of the target as features to train a three-layer neural network to classify pedestrians, vehicles, and crowds. However, there are some defects in the above four methods. First, they are more susceptible to noise interference. The changes in pedestrian movements and the complexity of the background will destroy the feature extraction. Secondly, for shape features, since the analysis is performed on the segmented foreground area, they are very dependent on the performance of the segmenter, and there are still many problems to be solved in the background segmentation technology.

Contents of the invention

Aiming at the problem that the existing head detection method is not suitable for pedestrian detection in a complex environment, the present invention provides a detection method for the number of pedestrians waiting to cross a crosswalk, which can not only detect the number of pedestrians more accurately, but also is simple and effective , can be applied to more complex environments.

The present invention is achieved through the following technical solutions:

The detection system for the number of pedestrians waiting to pass based on computer vision includes two CCD cameras and a computer image processing system; the computer image processing system is connected to the camera through an interface; the two CCD cameras are installed at both ends of the crosswalk; The field of view angles of the two CCD cameras include the opposite waiting area for pedestrians and the diagonally opposite vehicle area.

The detection method of the computer vision-based detection system for the number of pedestrians waiting to pass comprises the following steps:

1) The video camera captures video images;

2) Segment the pedestrian waiting area and preprocess the image in the video image with adaptive change of light intensity;

3) Use the method based on the Gaussian mixture background model to update the background, and obtain the foreground image of the pedestrian waiting area and preprocess it;

4) Based on the vertical integral projection method, the preprocessed foreground image is processed to obtain the vertical integral projection image of the pedestrian waiting area;

5) Through the processing and information statistics of the vertical integral projection map of the pedestrian waiting area, the number of pedestrians waiting to pass in the crosswalk is finally obtained.

The light intensity adaptive change preprocessing in the aforementioned step 2) includes,

2-1) Convert the color image of each frame of image captured by the camera into a grayscale image;

2-2) Perform statistics according to the gray histogram to judge the intensity of light in the image and the contrast of the image;

Take the gray level statistics as

$\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 127</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}$

Among them, _Hi is the probability of occurrence of each gray level, and i is the number of gray levels;

2-3) If L≥O. 8 or L≤0.2, adjust the contrast of the current frame image.

In the preceding step 3), select the background to be updated when the pedestrian starts to pass the crosswalk, and select the foreground image to be extracted 1s~2s before the end of the vehicle passage.

The preprocessing of the foreground image in the aforementioned step 3) is to sequentially perform histogram equalization, median filtering, connected domain denoising, and expansion operations on the foreground image.

In the preceding step 4), the vertical integral projection V is,

$V_{\left(i\right)}=\underset{i=1}{\overset{h}{\mathrm{\&Sigma;}}}P(i,j),$ j=1,2,...,W,

Where P(i, j) represents the number of pixels in the foreground image corresponding to the i position with a pixel value of 0, W is the width of the preprocessed image, and H is the height of the preprocessed image.

The aforementioned step 5) includes,

5-1) Delineate the threshold line, the line points greater than the threshold are the vertical areas of the human head potential area, and remove the points that are not the human head potential area;

5-2) Delineate the threshold of the potential area of the head to remove the interference point;

5-3) Make statistics on the width and height of the effective vertical area to determine the width and height occupied by a person;

5-4) Compare the width and height of each effective vertical area with the acquired width and height occupied by a person in this detection to determine the number of people contained in the area;

5-5) Add up the counted number of pedestrians in each valid vertical area, and use it as the final number of pedestrians waiting to cross the crosswalk acquired in this detection.

The delineation of the threshold line in the aforementioned step 5-1) is based on the distribution of the integral projection in the vertical direction to find out adaptively.

to a reasonable threshold Threshold,

Among them, P _i is V _(i) , the probability of i∈[1,W], α is the scaling factor, α∈[0.6,1].

The threshold in the aforementioned step 5-2) is 20 pixels.

The aforementioned step 5-3) refers to: make statistics on the width and height of the effective vertical area, first find the minimum width and height, and then, on this basis, use the average value of the values that differ from the minimum value by less than 5 pixels The value determines the width and height of a person in this detection.

The aforementioned steps 5-4) refer to:

If a vertical area exceeds the width and height occupied by one person, the area only contains one pedestrian;

When a vertical area exceeds the width occupied by one person and the value of height is outside the allowable range of the threshold, if the width exceeds the width occupied by one person, the area contains two pedestrians, and if the height exceeds the height occupied by one person, then The area contains two pedestrians, and if the width and height both exceed the width and height occupied by one person, the area contains four pedestrians.

The threshold of the allowable range of the aforementioned threshold is set according to the specific situation of the intersection and the location where the camera is erected.

The aforementioned thresholds have a height threshold of 12 pixels and a width threshold of 17 pixels.

The advantage of the present invention is that the method is simple and effective. Compared with the method of detecting the number of people based on feature matching, which is easily disturbed by surrounding objects and the error is relatively large in complex situations, the present invention is more suitable for operation in complex situations and is applied to pedestrian crossings. The intelligent traffic light supervision system is especially suitable, because the principle is simple, and the cost can also be reduced appropriately.

Description of drawings

Fig. 1 is the flow chart of the detection method for the number of pedestrians waiting to pass at a crosswalk based on computer vision in the present invention;

Fig. 2 is a flowchart of the processing and information statistical analysis of the vertical integral projection diagram according to the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

The detection system for the number of pedestrians waiting to pass based on computer vision includes two CCD cameras and a computer image processing system; the computer image processing system is connected to the camera through an interface; the two CCD cameras are installed at both ends of the crosswalk; The field of view angles of the two CCD cameras include the opposite waiting area for pedestrians and the diagonally opposite vehicle area.

As shown in Figure 1, the detection method of the computer vision-based detection system for the number of pedestrians waiting to pass the crosswalk includes the following steps:

1) The video camera captures video images;

2) Segment the pedestrian waiting area and preprocess the image in the video image with adaptive change of light intensity;

3) Using the Gaussian mixture background model method to obtain the foreground image of the pedestrian waiting area and preprocess it;

4) Based on the vertical integral projection method, the preprocessed foreground image is processed to obtain the vertical integral projection image of the pedestrian waiting area;

5) Through the processing and information statistics of the vertical integral projection map of the pedestrian waiting area, the number of pedestrians waiting to pass in the crosswalk is finally obtained.

Specifically divided into the following four parts:

In the first part, the image obtained from the video image is used for preprocessing such as segmentation of the pedestrian waiting area and adaptive change of light intensity, which specifically includes the following steps:

(1) Segmentation of pedestrian standing area. The biggest feature of images is that they can provide a lot of information, but at the same time, they will also bring a lot of interference information. The purpose of segmenting the pedestrian standing area from the image is to extract the effective area of pedestrian standing while reducing some interference information to a certain extent.

若L≥O．8或L≤0．2，对当前帧图像进行对比度调整。(2) Adaptive change of light intensity. For specific intersections, the intensity of light caused by weather and other factors is different, which will bring great inconvenience to the detection of pedestrians. When the light in the video image sequence is very weak or very strong, the target and the background have a great similarity. In the grayscale image obtained by inter-frame difference calculation, the target is not easy to be detected, and the influence of noise is relatively large. . Therefore, after segmenting the standing area of pedestrians, the color image of each frame of image (image width is W and height is H) collected by the CCD camera is first converted into a grayscale image, and then statistics are made according to the grayscale histogram to determine the The intensity of the light and the contrast of the image. Pixel-based processing is performed on images with weaker or stronger light to achieve the purpose of image enhancement. The specific judgment process is as follows: If the current frame grayscale image, I(x,y) , to count the probability H _i of each gray level, and the gray level statistics are taken as:

If L≥O. 8 or L≤0.2, adjust the contrast of the current frame image.

In the second part, the foreground image of the pedestrian waiting area is obtained and preprocessed by using the method based on the Gaussian mixture background model, which specifically includes the following steps:

(1) Real-time update of the background. The method of mixed Gaussian background model is used to realize the real-time update of the background. By counting the frequency of the corresponding gray value of each pixel in the continuous multi-frame image, when the detected target is in the process of moving, the same pixel Among all the gray values that appear at the point, the gray value of the pixel in the background image appears the most, that is, the gray value of the pixel in the background image appears more frequently than other gray values. The frequency should be high. According to this principle, after the statistics are finished, save the gray value with the highest frequency as the gray value at the corresponding pixel, then restore the entire background image, and finally save the background image for future use. use. For the timing of the background update of the pedestrian standing area, after a lot of experiments, the final choice is to start updating the background when the pedestrian starts to pass the crosswalk (it can be delayed for a short period of time), and the cut-off time of the background update should be slightly earlier than the set pedestrian passing time ( Because in the last 1 to 2 seconds, pedestrians may stop and wait for the next pass. If you continue to update the background, you may also use the pedestrians who stopped at this time as the background).

(2) Acquisition of the foreground including the detected object. In the present invention, select the last few seconds when the vehicle traffic is about to end, such as 1s~2s, to obtain a frame image containing pedestrians, the number information of pedestrians contained in this frame image is closer to the real situation (the closer the time is to the vehicle traffic end time, the closer the information contained in this image is to the real situation). Then make a frame difference between the obtained frame image and the previously obtained background image, so as to obtain the foreground image.

(3) Preprocessing of the foreground image. In order to further reduce the interference, the acquired foreground image is preprocessed. The processing process is sequentially performing histogram equalization, median filtering, connected domain denoising, and expansion operations on the foreground image.

In the third part, the method based on vertical integral projection processes the denoised foreground image to obtain the vertical integral projection image of the pedestrian waiting area. Assume preprocessing to obtain a binary image G of size W*H, W is the width of the preprocessed image, and H is the height of the preprocessed image. Note that the background and part of the target area at this time become zero, and the two In the value map, the part with more pixels in the vertical direction basically comes from the vertical area where the head of the human body is located. Based on this analysis, the vertical area where the head is located is determined according to the integral projection in the vertical direction of graph G. Note that V represents the integral projection of the vertical direction of the graph G, and V is a W-dimensional vector, where

$V_{\left(i\right)}=\underset{i=1}{\overset{h}{\mathrm{\&Sigma;}}}P(i,j)$ j=1,2,...,W,

P(i,j) represents the number of pixels in the foreground image corresponding to the i position with a pixel value of 0, W is the width of the image, and H is the height of the image.

The fourth part is to finally obtain the number of pedestrians waiting to pass through the crosswalk through the processing and information statistics of the vertical integral projection map of the pedestrian waiting area, as shown in Figure 2, which specifically includes the following steps:

即阈值取向量V的统计均值一部分，其中α为一比例因子，α∈[0.6,1]，其值与采集图像的亮度相关，由统计得到的当前图像的灰度直方图计算得到。通过大量的测试，L≤0.6，α取0.6，其他情况下，α取L的值，效果比较好。(1) Delineate the threshold line, determine the vertical area of the head potential area, and remove the points that are not the head potential area. In order to determine the vertical area where the potential human head is located, the present invention adaptively finds a reasonable threshold to divide these areas according to the distribution of the integral projection in the vertical direction of the graph G. The method is: statistics V _(i) , i∈ The probability distribution of [1,W], denote V _(i) , the probability of i∈[1,W] is P _i , and the threshold is taken as:

That is, the threshold takes part of the statistical mean value of the vector V, where α is a proportional factor, α∈[0.6,1], its value is related to the brightness of the collected image, and is calculated from the gray histogram of the current image obtained through statistics. Through a large number of tests, L≤0.6, α takes 0.6, and in other cases, α takes the value of L, and the effect is better.

(2) Define the threshold to remove the vertical area with too small width as the interference point. After obtaining the vertical area where the potential human head is located, through a large number of experiments, select a suitable threshold to remove the vertical area with too small width. In this embodiment, the threshold is selected as 20 pixels. This threshold needs to be based on the actual intersection situation. And the position of camera erection is measured.

(3) Make statistics on the width and height of the effective vertical area to determine the width and height occupied by a person. Make statistics on the width and height of the effective vertical area, first find the minimum width and height, and then on this basis, find the average of several values that are not much different from the minimum value (for example, the difference is less than 5 pixels) The value determines the width and height of a person in this detection.

(4) Compare the width and height of each effective vertical area with the acquired width and height occupied by a person in this detection to determine the number of people contained in the area. When comparing, if the width or height of a certain area exceeds a certain threshold value of the width or height occupied by a person in this detection obtained, it is considered that there is not only one pedestrian in the vertical area, and the area is determined according to the difference in the threshold The final number of people, the setting of the threshold needs to be determined according to the actual intersection situation and the location of the camera erection. In this embodiment, the height threshold is 12 pixels, and the width threshold is 17 pixels. The specific statistics of the number of people in each vertical area The method of how much is: a vertical area exceeds the width and height occupied by one person and is within the allowable range of the threshold, and the area is still considered as containing only one pedestrian; and if a vertical area exceeds the width and height occupied by one person When the value of the height is outside the allowable range of the threshold, the area contains two pedestrians only if the width exceeds the width occupied by one person, and the area contains two pedestrians if only the height exceeds the height occupied by one person. If the width and height If the width and height both exceed the width and height occupied by one person, the area is considered to contain four pedestrians.

(5) Add up the counted number of pedestrians in each valid vertical area, and use it as the final number of pedestrians waiting to cross the crosswalk acquired in this detection.

Claims (13)

1. based on the space number of the treating purpose detection system of computer vision, it is characterized in that: comprise two ccd video cameras and Computerized image processing system; Described Computerized image processing system links to each other with video camera by interface; Described two ccd video cameras are installed on the two ends of crossing; The field angle of described two ccd video cameras comprises the wait pedestrian zone on opposite and the vehicle region at opposite slightly to the right or left.

2. the detection method of the space number of the treating purpose detection system based on computer vision according to claim 1 is characterized in that: may further comprise the steps:

1) video camera capture video images;

2) image in the video image is carried out cutting apart and light intensity adaptive change pre-service of pedestrian's waiting area;

3) adopt based on Gaussian mixture model-universal background model method realization context update, and obtain the foreground picture of pedestrian's waiting area and it is carried out pre-service;

4) based on the method for vertical integral projection pretreated foreground picture is processed to obtain the vertical integral projection figure of pedestrian's waiting area;

5) treat number by the pedestrian by the processing of the vertical integral projection figure of pedestrian's waiting area and Information Statistics finally being obtained crossing.

3. detection method according to claim 2 is characterized in that: light intensity adaptive change pre-service comprises described step 2),

2-1) coloured image with every two field picture of camera acquisition is converted into gray level image;

2-2) add up according to grey level histogram, judge the intensity of light in the image and the contrast of image;

Getting the gray level statistics is

$L=\underset{i=0}{\overset{i=127}{\mathrm{\&Sigma;}}}{H}_i$

Wherein, H _iBe the probability that each gray level occurs, i is gray level progression;

If 2-3) L 〉=O.8 or L≤0.2 are to the adjustment of current frame image degree of comparing.

4. detection method according to claim 2 is characterized in that: in the described step 3), choose when the pedestrian begins by lateral road and begin to upgrade background, extract foreground picture during 1s before being chosen in vehicle pass-through and finishing ~ 2s.

5. detection method according to claim 2 is characterized in that: in the described step 3) pre-service of foreground picture for foreground image is carried out histogram equalization successively, medium filtering, connected domain denoising, expansive working.

6. detection method according to claim 2 is characterized in that: in the described step 4), vertically integral projection V is,

$V_{\left(i\right)}\underset{i=1}{\overset{H}{\mathrm{\&Sigma;}}}P(i,j),$ j=1,2,…，W，

Wherein P (i, j) the expression foreground picture pixel value corresponding with the i position is that 0 pixel is counted, and W is the width of pretreated image, and H is the height of pretreated image.

7. detection method according to claim 2 is characterized in that: described step 5) comprises,

5-1) delimit threshold line, be the vertical zone in the potential zone of the number of people greater than the line point of threshold value, and remove the point in the potential zone of the non-number of people;

The threshold value of 5-2) delimiting the potential zone of the number of people is removed noise spot;

5-3) to the width in effective vertically zone with highly add up, determine width that people is shared and highly;

5-4) with every effectively vertically the width in zone and height with obtain this time detect in the shared width of people and highly comparing, determine the contained number in this zone;

5-5) will count every effectively vertically the quantity of the pedestrian in the zone all add up, as this detect the final wait of obtaining pedestrian's the quantity of lateral road.

8. detection method according to claim 7 is characterized in that: delimit threshold line described step 5-1)

To find a rational threshold value according to the distribution situation of the integral projection of vertical direction is adaptive

Threshold， $\mathrm{Threshold}=\mathrm{\&alpha;}\&times;\underset{i=1}{\overset{W}{\mathrm{\&Sigma;}}}{V}_{\left(i\right)}\&CenterDot;P_i,$

P wherein _iBe V _(i), the probability of i ∈ [1, W], α are scale factor, α ∈ [0.6,1].

9. detection method according to claim 7, it is characterized in that: the threshold value described step 5-2) is 20 pixels.

10. detection method according to claim 7, it is characterized in that: described step 5-3) refer to: to the width in effective vertically zone with highly add up, obtain first minimum width and height, again on this basis, by with minimum value differ mean value less than the value of 5 pixels determine this time to detect in width and the height that the people is shared.

11. detection method according to claim 7 is characterized in that: described step 5-4) refer to:

If one vertically the zone surpass the shared width of people and highly all in the threshold value allowed band then should only comprise a pedestrian in the zone;

When the vertical zone value outside the threshold value allowed band time that surpasses the shared width of people and height, only width surpasses then two pedestrians of this district inclusion of the shared width of people, only highly surpass then two pedestrians of this district inclusion of the shared height of people, if width and highly all above the shared width of people and four pedestrians of this district inclusion highly then.

12. detection method according to claim 11 is characterized in that: described threshold value arranges according to the position that concrete condition and the video camera at crossing sets up.

13. detection method according to claim 12 is characterized in that: the height threshold of described threshold value is 12 pixels, and width threshold value is 17 pixels.

Images