WO2016059643A1

WO2016059643A1 - System and method for pedestrian detection

Info

Publication number: WO2016059643A1
Application number: PCT/IN2015/000300
Authority: WO
Inventors: Vinay Govind Vaidya; Krishnan KUTTY; Smita Nair; Reena Kumari BEHERA; Jiji GANGADHARAN
Original assignee: Kpit Technologies Ltd.
Priority date: 2014-10-16
Filing date: 2015-07-29
Publication date: 2016-04-21
Also published as: IN2014MU03302A

Abstract

The present invention describes method and system for pedestrian detection during daytime. The method comprises detecting edges between object boundaries in a captured image based on color data, linking the edges by detecting missing links between object boundaries based on identifying and filling broken links, segmenting the image based on color edge labeling, removing clutter in the segmented image, determining at least one of a leg pattern and head region, and classifying the object in the image as one of a pedestrian object and non-pedestrian object in the image based on the determination of the at least one of a leg pattern and head region. The system comprises a pre-processing module, a segmenting module, a post-processing module, a detection module, a classification module, and a tracking module. The pre-processing module comprises an edge detecting module, and an edge linking module.

Description

SYSTEM AND METHOD FOR PEDESTRIAN DETECTION

Field of Invention The present invention relates to image processing and more specifically relates to system and method for vision based pedestrian detection during day time.

Background of Invention With increase in the number of vehicles on the road, the pedestrian safety has become a new challenge area for the Automotive OEMs. The statistics released by the World Health Organization (WHO) is alarming with more than 270000 pedestrians losing their lives on the road. The pedestrians constitutes to about 22% of the road accidents globally. The insight shows a crucial need for pedestrian safety system on vehicle that could warn the drivers well ahead of time. Sensors play an important role when it comes to detecting obstacles in the surrounding. Commonly used sensors on the vehicle are LIDAR (Light Detection and Ranging), RADAR (Radio Detection and Ranging), ultrasound and camera. As compared to various sensors, vision-based systems are gaining significant importance due to their lower cost and advantages as compared to other sensors.

Camera based pedestrian detection is a challenging problem because of various poses and clothing of pedestrians, which needs to be handled under varying illumination and environmental conditions. Generally, there are two main approaches for vision based pedestrian detection, the whole and the part based approach. In the whole body detection, the pedestrian is detected as a whole object; where as in part based the detection process is concentrated on parts like head, torso arms, legs, etc. The general process for detection constitutes of pre- processing, foreground segmentation, object classification and tracking. Pre- i processing includes exposure correction, dynamic ranging, noise removal etc. to provide a better input for further processing. Foreground segmentation extracts possible candidate ROI by eliminating background and sky region. This restricts the search ROI, thereby, reducing the processing time and false positives.

In some of the existing methods of pedestrian detection, the entire image is scanned at various scales and the process is extremely slow. Saliency based method uses 2D features such as gradient, color, intensity, edge etc., to extract object segments. Since the method is highly dependent on the selected features, human detection is not much efficient. Stereo-based foreground segmentation is another way to eliminate background. For most of the existing techniques, one of the major assumption is that pedestrians possess a vertical structure at a specific depth. Some of the existing techniques are, v-disparity representation to find vertical and horizontal planes to extract candidate ROIs, stereo-based piane fitting to find different planes, disparity map analysis with Pedestrian Size Constraint (PSC) to extract better ROIs, multimodal stereo methods that make use of different spectrums like visual and thermal infrared, etc.

Most of the algorithms are invariant to illumination changes. Time complexity is more for the stereo-based algorithms and the detection rate is very less in non- textured regions. Motion based segmentation is another method to extract ROIs. Both motion and stereo are utilized in some methods to get the best foreground. Object classification aims at classifying pedestrian or non-pedestrian segments in a given candidate ROI. Numerous methods based on number of features with different classifiers are available in literature for whole body and part based detection. Tracking is used to predict future pedestrian positions in order to avoid false detection and to reduce the computational time. Kalman filters, Particle filters, stereo-odometry based trackers are some of the commonly used methods for tracking. Additionally, pedestrian detection during day time is difficult, requires complex systems and has less accuracy. Thus, there is need for a system and method for detecting pedestrian by segmenting the regions in real time by using the fact that pedestrians have strong edge features, especially in the leg and the torso region. Additionally, there is a need to consider the fact that the pedestrians are always vertically aligned. There is a need for robust, accurate and a simple system and method for pedestrian detection during day time.

Summary

The present invention discloses method and system for providing pedestrian detection during daytime. The present method accurately segments the pedestrian regions in real time. The fact that the pedestrians are always vertically aligned is taken into consideration. As a result, the edge image is scanned from bottom to top and left to right. Both the color and edge data are combined in order to form the segments. The segmentation is highly dependent on the edge map. Even a single pixel dis-connectivity would lead to incorrect segments. To improve this, a unique edge linking method is performed prior to segmentation. The segmentation would consist of foreground and background segments as well. The background clutter is removed based on certain predefined conditions governed by the camera features.

The present invention discloses an edge based head detection method for increasing the probability of the pedestrian detection. The combination of head and leg pattern determines the presence of pedestrians. The extracted segments are merged to form the complete pedestrian based on the evident leg and head pattern. The method provides good detection capability. The accuracy of the disclosed method is further improved by using a classifier on the segmented region. An embodiment of the present invention describes a method of providing pedestrian detection during daytime. The method comprises detecting edges between object boundaries in a captured image based on color data, linking the edges by detecting missing links between object boundaries based on identifying and filling broken links, segmenting the image based on color edge labeling, removing clutter in the segmented image, determining at least one of a leg pattern and head region, and classifying the object in the image as one of a pedestrian object and non-pedestrian object in the image based on the determination of at least one of a leg pattern and head region.

In one embodiment, detecting edges between object boundaries in the captured image comprises using a canny edge detection process.

In one embodiment, linking the edges by detecting missing links between object boundaries comprises scanning the captured image from left to right and bottom to top, performing a check to determine disconnect between the pixels at the edge of the object in the captured image, comparing magnitude and orientation criteria of the pixels with neighboring pixels when the disconnect between the pixels at the edge is found, checking if more than one pixel in neighborhood has equal magnitude and orientation criteria, setting coordinate (xi,yi) pixel with shortest distance to link with center pixel, fixing a link with one of the neighboring pixel based on orientation of center pixel when magnitude and orientation criteria is not equal to neighboring pixels, and storing added link address to memory. In another embodiment, linking the edges by detecting missing links between object boundaries comprises scanning the captured image from left to right and bottom to top, performing a check to determine disconnect between single pixels at the edge of the object in the captured image, scanning nXn area for edge pixels when the disconnect between the single pixels is found, checking the edge pixel present in the nXn area, discarding the edge pixel when the edge pixel is absent in the nXn area, finding shortest distance between center pixel and existing neighbor pixel when the edge pixel is present in the nXn area, and highlighting pixel in a neighboring area. Here, the neighboring area could be n-1 , n-2 or so on.

In one embodiment, segmenting the image based on color edge labeling comprises scanning left edge pixel, checking for the availability of corresponding right edge pixel, checking whether distance between the left and right edge pixels is less than a predefined threshold value, scanning again the left edge pixel when the corresponding right edge pixel is either unavailable or the distance between the left and right edge pixel is more than the predefined threshold value, calculating mean and standard deviation when the distance between the left and right edge pixel is less than threshold, checking mean difference and standard deviation difference are less than a predefined threshold value between the current scanning line and a line below the scanning line, assigning a first color to the current scan line when mean difference and standard deviation difference are more than the predefined threshold value between the current scanning line and a line below the scanning line, and assigning second color, that of the below scan line, when mean difference and standard deviation difference between the current scanning line and a line below the scanning line is less than the predefined threshold.

In one embodiment, removing clutter in the segmented image comprises removing dangling segments.

In another embodiment, removing clutter in the segmented image comprises removing segments that are not bounded by vertical edges.

In one embodiment, determining a leg pattern comprises checking the confidence value greater than a predefined threshold value when leg pattern is detected, detecting the object in the image as the pedestrian when the confidence value is greater than the predefined threshold value, and detecting the object in the image as the non-pedestrian when the confidence value is less than the predefined threshold value.

In one embodiment, determining a head region comprising replacing consecutive horizontal or vertical pixels with a single pixel, calculating angle between the pixels, performing a check whether a predefined pattern of angles is for the detected head, detecting the head in the image when the predefined pattern of angles is similar to the detected head else head is not detected, checking the confidence value greater than a predefined threshold value when the head is detected, detecting the object in the image as the pedestrian when the confidence value is greater than the predefined threshold value, and detecting the object in the image as the non-pedestrian when the confidence value is less than the predefined threshold value.

In another embodiment, the method further comprises predicting future pedestrian positions in order to avoid false detection and to reduce the computational time.

In yet another embodiment, the method further comprises processing a captured image to exposure correction, dynamic ranging, noise removal before detecting edges between object boundaries in the captured image. Another embodiment of the present invention describes a system for providing pedestrian detection during daytime. The system comprises a pre-processing module configured for detecting and linking edges of a captured image, a segmenting module connected with the pre-processing module for determining an object in the captured image based on color-edge labelling, a post-processing module connected with the segmenting module for removing clutter in the segmented image, a detection module connected with the post-processing module for determining at least one of a leg pattern and head region based on region filling operation on the post-processed segmented regions and based on the orientation of pixels over the edges in the image respectively, a classification module connected with the detection module for classifying the object in the image as one of a pedestrian object and non-pedestrian object, and a tracking module connected with the classification module for predicting future pedestrian positions in order to avoid false detection and to reduce the computational time. BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS

The aforementioned aspects and other features of the present invention will be explained in the following description, taken in conjunction with the accompanying drawings, wherein:

Figure 1a illustrates a block diagram of a system for pedestrian detection during daytime according to an embodiment of the present invention.

Figure 1b illustrates a block diagram of a pre-processing module according to an embodiment of the present invention.

Figure 2 illustrates a method of pedestrian detection during daytime, according to an embodiment of the present invention. Figure 3 illustrates the four neighboring pixels used for dis-connectivity check according to an embodiment of the present invention.

Figure 4 illustrates different disconnected patterns considered for edge linking according to an embodiment of the present invention. Figure 5 illustrates the relation between the gradient orientation and the corresponding pixel locations, according to an embodiment of the present invention. Figure 6 illustrates a flow chart of a method of linking the edges by detecting missing links between object boundaries according to an embodiment of the present invention.

Figure 7 illustrates a flow chart of a method of linking the edges by detecting missing links between object boundaries for single pixel disconnect according to an embodiment of the present invention.

Figure 8(a) illustrates the circled regions (red) that undergo edge linking process according to an embodiment of the present invention.

Figure 8 (b) illustrates the output of edge linking stage according to an embodiment of the present invention.

Figure 9 illustrates the center pixel and its pixel neighborhood according to an embodiment of the present invention.

Figure 10 illustrates a flow chart of a method of segmenting the image based on color edge labeling, according to an embodiment of the present invention. Figure 11(a) illustrates an input image according to an embodiment of the present invention.

Figure 11(b) illustrates the initial segmentation output according to an embodiment of the present invention. Figure 12(a) illustrates the removal of dangling segments output according to an embodiment of the present invention.

Figure 12(b) illustrates removal of segments based on width and height criteria output according to an embodiment of the present invention.

Figure 12(c) illustrates the second round of dangling segment removal output according to an embodiment of the present invention. Figure 12(d) illustrates the removal of blobs based on vertical edge bounding criteria output according to an embodiment of the present invention.

Figure 13 illustrates the region filling performed on segmented regions that would avoid erroneous leg detections according to an embodiment of the present invention.

Figure 14(a) illustrates pedestrian silhouettes leg pattern with wide separation according to an embodiment of the present invention. Figure 14(b) illustrates pedestrian silhouettes leg pattern with narrow separation according to an embodiment of the present invention.

Figure 15 illustrates the results of leg detection on pedestrian images according to an embodiment of the present invention.

Figure 16(a1) and Figure 16(b1) are smooth circular curves according to an embodiment of the present invention. Figure 16 (a2) and Figure 16 (b2) represent corresponding angle pattern for the curves when traversed in the direction mentioned in the figure according to an embodiment of the present invention. Figure 17 (a) illustrates an edge image (a1 ) zoomed region according to an embodiment of the present invention.

Figure 17 (b) illustrates an edge after performing moving average of (a) (b1 ) zoomed region according to an embodiment of the present invention.

Figure 17 (c) illustrates result of pixel elimination step on the input (b) according to an embodiment of the present invention.

Figure 17 (d) illustrates angle pattern of the curve according to an embodiment of the present invention.

Figure 18 (a1, b1) illustrates an input Region according to an embodiment of the present invention. Figure 18 (a2, b2) illustrates a foreground segmented Image according to an embodiment of the present invention.

Figure 18 (a3, b3) illustrates corresponding edge images of a2, b2 according to an embodiment of the present invention.

Figure 18 (a4, b4) illustrates output pixels after the pixel elimination step according to an embodiment of the present invention.

Figure 18 (a5, b5) illustrates detected head region according to an embodiment of the present invention. Figure 18 (a6, b6) illustrates angle pattern of the edges and the detected head pattern according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments. The present invention can be modified in various forms. Thus, the embodiments of the present invention are only provided to explain more clearly the present invention to the ordinarily skilled in the art of the present invention. In the accompanying drawings, like reference numerals are used to indicate like components. The specification may refer to "an", "one" or "some" embodiment(s) in several locations. This does not necessarily imply that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment. Single features of different embodiments may also be combined to provide other embodiments.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms "includes", "comprises", "including" and/or "comprising" when used in this specification, specify the presence of stated features, integers, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term "and/or" includes any and all combinations and arrangements of one or more of the associated listed items. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The present invention describes a driver assistance system which helps the driver by alerting about the situation well ahead. The system helps in improving the response time of the driver of the vehicle. The system enables the driver to avoid possible collision with the pedestrian. The night time pedestrian detection is usually performed on NIR images. The pedestrians are highlighted with a bright pixel in such images. The present invention describes day time pedestrian detection which is performed on optical images. These images hold all the information about the pedestrian as well as the background. Distinguishing the pedestrian from such a complicated background makes it a difficult task to perform. The existing methods use Histogram of Oriented Gradient (HOG) as a prominent feature to define the pedestrian. These features are used to train a classifier such as Support Vector Machine (SVM). Once the classifier is trained with sufficient pedestrian and non-pedestrian examples, the classifier is ready to classify a given segment as a pedestrian or a non-pedestrian.

The visibility in the day time scenario is sufficiently good where the pedestrians are clearly visible to the driver. The sudden movement of pedestrians in front of the vehicle poses a challenge to the driver. The driver would not have enough time to respond to the situation. In such a situation, pedestrian detection system would help the driver in knowing the movement of the pedestrian well in advance. This would greatly decrease the accidents involving the pedestrian. In one embodiment, an on board forward facing camera captures the scene ahead of the vehicle. The extracted image is first segmented and the background clutter is removed from the scene. The pedestrians are detected from the foreground segmented regions based on leg and head detection criteria. Segmentation is a crucial step in any detection and tracking based system.

The pedestrians have strong edge based features, especially in the leg and the torso region. Additionally, the pedestrian attire is also important. The pedestrian cloth's color has some spatial relationship. According to the method of the invention, both the color and edge data are combined to form the segments for detecting pedestrians. Since edge is crucial information, any breaks in the edge data caused by thresholding can lead to unwanted segments. This is handled by an edge linking technique to fill in the broken gaps. With the edge and the color information, the segments are labeled and provided with unique code. The later stage is followed by grouping like segmented regions and removal of background data.

Figure 1 illustrates a block diagram of a system 100 for providing pedestrian detection during daytime according to an embodiment of the present invention. According to one embodiment as depicted in Figure 1a, the system 100 comprises an image capturing module 101 , a pre-processing module 102, a segmenting module 103, a post-processing module 104, a detection module 105, and a display unit 107. The system 00 further comprises a classification module 106 and a tracking module 108. The image capturing module 101 captures the image and performs initial processing such as exposure correction, dynamic ranging, and noise removal before sending the captured image to the preprocessing module 102. The pre-processing module 102 is connected to the image capturing module 101 for receiving the initially processed captured image. The pre-processing module 102 comprises an edge detecting module 109 and an edge linking module 110, for detecting and linking edges of the captured image as depicted in Figure 1 b. In one embodiment, the edge detecting module 109 uses a canny edge detection process for detecting edges between object boundaries in the captured image. The segmenting module 103 is connected with the pre-processing module 102 for determining an object in the captured image based on color-edge labelling. The post-processing module 104 is connected with the segmenting module 103 for removing clutter in the segmented image. The detection module 105 is connected with the post-processing module 104 for determining leg pattern and/or head region based on region filling operation on the post-processed segmented regions and based on the orientation of pixels over the edges in the image respectively.

In one embodiment, the classification module 106 is connected with the detection module 105 for classifying the object in the image as one of a pedestrian object and non-pedestrian object. The tracking module 108 is connected with the classification module 106 for predicting future pedestrian positions in order to avoid false detection and to reduce the computational time. In one embodiment, the display unit 106 is connected to the classification module for the object classified as the pedestrian object or non-pedestrian object.

Figure 2 illustrates a method of providing pedestrian detection during daytime, according to an embodiment of the present invention. At step 201 , input image is captured and processed for exposure correction, dynamic ranging, and noise removal before detecting edges between object boundaries in the captured image. At step 202, the edges between object boundaries in the captured image are detected using a canny edge detection method.. At step 203, the edges are linked by detecting missing links between object boundaries based on identifying and filling broken links. At step 204, the image is segmented based on color edge labeling. At step 205, the segments having color similar to the road color (i.e. clutter) are removed. At step 206, dangling segments (if any) are removed. At step 207, segments (if any) that are not bounded by vertical edges are removed. In order to determine the object in the image as either a pedestrian or non- pedestrian, a leg pattern and/or head region is determined. At step 208, a check is performed for the leg pattern. At step 209, the object is determined as non- pedestrian when the leg pattern is not detected. At step 210, the object is a probable pedestrian when the leg pattern is detected. At step 21 1 , a check is performed whether the confidence value is greater than a predefined threshold value when leg pattern is detected. The object in the image is detected as the pedestrian when the confidence value is greater than the predefined threshold value at step 212. The object in the image is detected as the non-pedestrian when the confidence value is less than the predefined threshold value at step 209.

In another embodiment, the object is determined as a pedestrian or non- pedestrian by determining the head region. At step 213, the canny edge detection is performed. At step 214, consecutive horizontal or vertical pixels are replaced with a single pixel. At step 215, the angles between the pixels are calculated. At step 216, a check is performed whether a predefined pattern of angles is for the detected head. If yes, the head is detected at step 217. If no, the head is not detected at step 218. The steps 21 1 and 212 are repeated to determine whether the object is either pedestrian or non-pedestrian. Edge Detection:

To form probable segments, the color data is considered between two edges in a row. For this reason, the detected edges should be strong, smooth and of one pixel width. There are various methods to perform edge detection which includes but not limited to Sobel, Prewitt and Canny. In a preferred embodiment, the present invention uses a Canny edge detection process which provides comparatively good detection, localization and single response to a particular edge. Two main highlights in the Canny detector implemented for the present pedestrian detection is as follows: The image gradient is computed using the following centered mask in both the x and y directions

Centered Mask:

Magnitude (\VF\) = y'Fx² + Fy

Where,

Fx is gradient along x direction,

Fy is gradient along y direction,

6 is the gradient orientation

The above 1-D centered mask provides best results for pedestrian images. The gradient computation is followed by non-maximal suppression and hysteresis thresholding stage. Figure 3 illustrates the four neighboring pixels used for dis- connectivity check according to an embodiment of the present invention.

To obtain optimal threshold values for hysteresis thresholding stage, the threshold values are taken as a factor of the distribution of the edge pixels. For example, the upper threshold TH is considered to be 0.2, such that 20% of the total pixels above TH are retained. The lower threshold TL is a factor of the high threshold value. TL is set at 0.9, which is 90% of the high threshold value. The gradient values greater than TH are retained while lower than TL are removed. The gradient values between TH and TL are retained based on the connectivity with the high threshold pixel. Even after getting optimal threshold values, thresholding introduces gaps in the edge map. Since the segmentation is based on the edge map, even a single pixel gap could lead to unwanted segments. This is handled by using edge linking methods.

Figure 4 illustrates different disconnected patterns considered for edge linking according to an embodiment of the present invention. Figure 5 illustrates the relation between the gradient orientation and the corresponding pixel locations, according to an embodiment of the present invention. Since the edge pixels are scanned from left to right and bottom to top, to ensure connectivity in the forward direction, following steps are adopted in the local edge linking method:

For a center pixel (x, y), for example, a 3x3 neighborhood is monitored.

• A pixel is identified as disconnected, if all the pixels in the four forward direction i.e. pixels at positions (x-1 ,y), (x-1 ,y-1 ), (x-1 ,y+1 ), (x,y+1 ) are zero ( Figure .3)

• Once a pixel is identified as disconnected, check it's neighborhood for one of the disconnected patterns, as shown in Figure 4.

• If the pixel does not satisfy the disconnected pattern, the pixel is considered to be connected and hence ignored.

• Pixels that are disconnected are considered for edge linking. The detailed process of edge linking is explained in figures 6 & 7. A check is performed between the center pixel and the four forward positions in 3x3 window (shown in Figure 4) and subsequently the shortest distance is set with the corresponding neighboring pixel.

• If the above condition is not satisfied, instead of ignoring the pixel, choose a neighboring pixel, based on the orientation value of the center pixel (x, y)- • As shown in Figure 5, the gradient orientation is grouped into four angles (0, 45°, 90° and 135°). If the center pixel has orientation value of say, 90°, then the pixel (x, y+1 ) is set and linked to (x, y). The above step of force fitting a neighboring pixel is performed only if the magnitude of the corresponding neighboring pixel is greater than a threshold. In an exemplary case, the threshold is maintained at 10. As required, the above force fitting is performed, for example, for 3 times continuously. Even after three attempts, if the similarity criteria is not satisfied, then pixel linking is discontinued and the force fitted links are removed.

For every disconnected pixel, the edges are iteratively extended. A certain maximum threshold, for example, of 5, is maintained for every iteration. Also, the disconnected link is visited during the scanning process. This ensures that long disconnected edges can be filled as shown in Figure 8.

Edge Linking

Figure 6 illustrates a flow chart of a method of linking the edges by detecting missing links between object boundaries according to an embodiment of the present invention. At step 601 , the captured image is scanned from left to right and bottom to top. At step 602, a check is performed to determine disconnect between the pixels at the edge of the object in the captured image. At step 603, magnitude and orientation criteria of the pixels with neighboring pixels are compared when the disconnect between the pixels at the edge is found. At step 604, a check is performed if more than one pixel in neighborhood has equal magnitude and orientation criteria. At step 605, coordinate (xi,yi) pixel with shortest distance to link with center pixel is set. At step 606, a link is fixed with one of the neighboring pixel based on orientation of center pixel when magnitude and orientation criteria are not equal to neighboring pixels. At step 607, added link address is stored in memory.

Single Pixel Disconnectivity The method of linking edges as discussed in figure 6 handles only specific disconnectivity, as shown in Figure 4. The single pixel dis-connectivity that might occur in all possible directions is not handled in the above edge linking stage. To remove one pixel gap:

• Each pixel is scanned in its 3x3 neighborhood. The pixel is considered as single pixel connected if it has only one pixel connected in the 8 neighborhood.

• For every disconnected pixel, check for the 5x5 neighborhood. If only one pixel exists in this neighborhood, set the corresponding pixel in 3x3 neighborhood based on the criteria shown in figure 8.

· If more than one pixel in the 5x5 neighborhood is connected, select the pixel with the least Euclidian distance. The algorithm accurately fills the single pixel gaps in the image.

Figure 7 illustrates a flow chart of a method of linking the edges by detecting missing links between object boundaries for single pixel disconnect according to an embodiment of the present invention. At step 701 , the captured image is scanned from left to right and bottom to top. At step 702, a check is performed to determine disconnect between single pixels at the edge of the object in the captured image. At step 703, nXn area is scanned for edge pixels when disconnect between the single pixels is found. At step 704, a check is performed whether the edge pixel is present in the nXn area. At step 705, the edge pixel is discarded when the edge pixel is absent in the nXn area. At step 706, shortest distance is found between center pixel and existing neighbor pixel when the edge pixel is present in the nXn area. At step 707, pixel in a n-2 X n-2 area is highlighted. In one exemplary embodiment, the nXn area is 5X5 area as shown in figure 9, which depicts the center pixel and its pixel neighborhood.

Practical issues with any edge detection method are that it causes missing links in the edge image. This leads to issues while identifying object boundaries. The local edge-linking method extends edges by finding the most compatible edge in its neighborhood.

Let - -^χ>-^ be the center pixel value. The main criteria for establishing similarity are:

a. Magnitude of the gradient vector I W(x,y)\ b. Direction of the gradient vector <¾ ,>')

Edge pixel (xO, vO), in the 3x3 neighborhood, is similar to pixel (x, y) if,

\ W(x,y^ - W(xQ,yO)^" \ < M. (2)

I <5f , y) - e(xQ, yO) | < _. a (3) Where,

M is the threshold for similarity in gradient magnitude. a is the threshold for similarity in gradient direction.

The value for ^M is set to 25. The threshold value for ^a is maintained at 20. If both the above conditions are satisfied, the pixel (xO, yO) is set and is linked to pixel (x, y).

Color-edge Based Labelling

Figure 10 illustrates a flow chart of a method of segmenting the image based on color edge labeling, according to an embodiment of the present invention. At step 1001 , left edge pixel is scanned. At step 1002, a check is performed for the availability of corresponding right edge pixel. At step 1003, a check is performed whether distance between the left and right edge pixels is less than a predefined threshold value. When the corresponding right edge pixel is either unavailable or the distance between the left and right edge pixel is more than the predefined threshold value, step 1001 is repeated and the left edge pixel is again scanned. At step 1004, mean and standard deviation are calculated when the distance between the left and right edge pixel is less than threshold. At step 1005, a check is performed whether -mean difference and standard deviation difference are less than a predefined threshold value between the current scanning line and a line below the scanning line. At step 1006, a first color is assigned to the current scan line when mean difference and standard deviation difference are more than the predefined threshold value between the current scanning line and a line below the scanning line. At step 1007, a second color, picked from the below scan line, is assigned to the current scan line when mean difference and standard deviation difference between the current scanning line and a line below the scanning line is less than the predefined threshold.

According to the method of the present invention, the labeling is performed based on color as well as edge information. There are certain characteristics specific to pedestrians in any given image. The most important characteristic with the pedestrians is that the pedestrians would always appear vertical in a given image. The edge information is obtained using the method as mentioned above. The edge image is scanned from bottom to top and left to right. Whenever, an edge pixel is encountered, the pre-processing module expects for next edge pixel on the same row. Mean and standard deviation is calculated for that scan line. If this is the first scan line to be encountered, a new label is assigned. Figure 11(a) illustrates an input image according to an embodiment of the present invention. Figure 11(b) illustrates the initial segmentation output according to an embodiment of the present invention. If this is not the first scan line, then the calculated mean and standard deviation is compared against the mean and standard deviation of the below scan line. If the value difference is within certain range, then the label of the below scan line is assigned to the current scan line. Otherwise, a new label is assigned to the current scan line. This method results in an initial segmentation of all the objects in an image that are vertically oriented as shown in Figure 11(b). The unwanted segments are removed based on certain pre-defined conditions. Figure 12(a) shows the removal of dangling segments output according to an embodiment of the present invention. Since the camera parameters are known, the width and height of the pedestrian at a given distance is also known. The segments with width and height greater than the predefined limit are removed as shown in figure 12(b). The second round for removing dangling segment having no connectivity with other segments is shown in figure 12(c). Perfect shapes such as rectangular, square, triangle etc. are also removed. Since the pedestrians have evident vertical edges, the segments that are not bounded by vertical edges are also removed. Figure 12 (d) shows the removal of blobs based on vertical edge bounding criteria output according to an embodiment of the present invention. Detection: Leg Detection

Pedestrian legs have strong edge features. Extracting leg patterns is comparatively much easier than extracting patterns of other parts of the body. The extracted segments are initially removed based on their height-width criteria and the area of each segmented blob. To avoid erroneous detections, a basic region filling operation is performed on the segmented regions. The result of region filling is as shown in Figure 13.

For walking or standing pedestrians, cases are considered for leg separation, as shown in Figure 14. For a given input segment, a typical leg like pattern (high- low-high-low pattern) is monitored. The segments that satisfy this condition are marked as detected leg regions. The aspect ratio of the leg region with respect to the row number is monitored to avoid false detection. The results of various pedestrian images and detected pedestrians are shown in Figure 15. Detection: Head Detection

The present head detection technique is for verification of a pedestrian in the refined segments. Given an ROI (region of interest) based on the segmentation explained, the head detection technique detects the head region based on the orientation of pixels over the edges in the ROI. The pedestrian head in an image appears circular or elliptical with trivial variation. For a smooth circular curve, the tangent (or normal) angle distribution of edge pixels has a set of possible patterns (varies with respect to the direction of traversing), as shown in Figure 16. These pre-defined patterns are utilized in the method of the invention to verify the presence of a head. Moving average calculation is performed on the edge pixels in order to smoothen the edge. In order to avoid the digitization error, which might add noise to the expected pattern, a pixel elimination step is adapted. Pixel elimination is performed by replacing the continuous vertical and horizontal pixels with a pre-defined minimum length with the centroid of that line segment, as illustrated in Figure 17.

Tangent angle at each point is calculated in a specific direction for all the selected pixels of the edges. If there are repeating angles, only one angle value is kept for further processing to avoid complexity. The plot of the angles after eliminating the continuously repeating values is exploited to find the pre-defined pattern. Pattern matching is performed over the selected angles based on the defined pattern and the matching region will give the probable head region in the edge image, as illustrated in Figure 18. Thus, the system and method of the present invention uses novel and inventive segmentation and detection process that detects pedestrians in the segmented region. The method of the invention relies on the edge map for efficient segmentation of the objects in the given image. The novel way of linking the edges help in segmenting the objects in a reliable way. The clutter removal step after the segmentation helps in reducing the segments to be analyzed considerably. This in turn helps in reducing the computational time efficiently. The head and leg pattern hel s in detecting the pedestrians with greater confidence.

All equivalent relationships to those illustrated in the drawings and described in the application are intended to be encompassed by the present invention. The examples used to illustrate the embodiments of the present invention, in no way limit the applicability of the present invention to them. It is to be noted that those with ordinary skill in the art will appreciate that various modifications and alternatives to the details could be developed in the light of the overall teachings of the disclosure, without departing from the scope of the invention.

Claims

A method of pedestrian detection during daytime, the method comprising: detecting edges between object boundaries in a captured image based on color data;

linking the edges by detecting missing links between object boundaries based on identifying and filing broken links;

segmenting the image based on color edge labeling;

removing clutter in the segmented image;

determining at least one of a leg pattern and head region; and

classifying the object in the image as one of a pedestrian object and non-pedestrian object in the image based on the determination of the at least one of a leg pattern and head region.

The method as claimed in claim , wherein detecting edges between object boundaries in the captured image comprises using a detection process.

The method as claimed in claim 1, wherein linking the edges by detecting missing links between object boundaries comprises:

scanning the captured image from left to right and bottom to top;

performing a check to determine disconnect between the pixels at the edge of the object in the captured image;

comparing magnitude and orientation criteria of the pixels with neighboring pixels when the disconnect between the pixels at the edge is found;

checking if more than one pixel in neighborhood has equal magnitude and orientation criteria;

setting coordinate (xi.yi) pixel with shortest distance to link with center pixel; fixing a link with one of the neighboring pixel based on orientation of center pixel when magnitude and orientation criteria is not equal to neighboring pixels; and

storing added link address to memory.

The method as claimed in claim 1 , wherein linking the edges by detecting missing links between object boundaries comprises:

scanning the captured image from left to right and bottom to top;

performing a check to determine disconnect between single pixels at the edge of the object in the captured image;

scanning nXn area for edge pixels when the disconnect between the single pixels is found;

checking the edge pixel present in the nXn area;

discarding the edge pixel when the edge pixel is absent in the nXn area; finding shortest distance between center pixel and existing neighbor pixel when the edge pixel is present in the nXn area; and

highlighting pixel in a neighboring area.

The method as claimed in claim 1 , wherein segmenting the image based on color edge labeling comprises:

scanning left edge pixel;

checking for the availability of corresponding right edge pixel;

checking whether distance between the left and right edge pixels is less than a predefined threshold value;

scanning again the left edge pixel when the corresponding right edge pixel is either unavailable or the distance between the left and right edge pixel is more than the predefined threshold value;

calculating mean and standard deviation when the distance between the left and right edge pixel is less than threshold; checking mean difference and standard deviation difference are less than a predefined threshold value between the current scanning line and a line below the scanning line;

assigning a first color to the current scan line when mean difference and standard deviation difference are more than the predefined threshold value between the current scanning line and a line below the scanning line; and

assigning a second color of the line below the current scan line when mean difference and standard deviation difference between the current scanning line and a line below the scanning line is less than the predefined threshold.

6. The method as claimed in claim 1 , wherein removing clutter in the segmented image comprises removing dangling segments.

7. The method as claimed in claim 1 , wherein removing clutter of the segmented image comprises removing segments that are not bounded by vertical edges.

8. The method as claimed in claim 1 , wherein determining a leg pattern comprising:

checking the confidence value greater than a predefined threshold value when leg pattern is detected;

detecting the object in the image as the pedestrian when the confidence value is greater than the predefined threshold value; and

detecting the object in the image as the non-pedestrian when the confidence value is less than the predefined threshold value.

9. The method as claimed in claim 1 , wherein determining a head region comprising: replacing consecutive horizontal or vertical pixels with a single pixel; calculating angle between the pixels;

performing a check whether a predefined pattern of angles is for the detected head;

detecting the head in the image when the predefined pattern of angles is similar to the detected head else head is not detected;

checking the confidence value greater than a predefined threshold value when the head is detected;

10. The method as claimed in claim 1 further comprising predicting future pedestrian positions in order to avoid false detection and to reduce the computational time.

11. The method as claimed in claim 1 further comprising processing a captured image to exposure correction, dynamic ranging, noise removal before detecting edges between object boundaries in the captured image.

12. A system for pedestrian detection during daytime, the system comprising:

a pre-processing module configured for detecting and linking edges of a captured image;

a segmenting module connected with the pre-processing module for determining an object in the captured image based on color-edge labelling;

a post-processing module connected with the segmenting module for removing clutter in the segmented image; a detection module connected with the post-processing module for determining at least one of a leg pattern and head region based on region filling operation on the post-processed segmented regions and based on the orientation of pixels over the edges in the image respectively;

a classification module connected with the detection module for classifying the object in the image as one of a pedestrian object and non- pedestrian object; and

a tracking module connected with the classification module for predicting future pedestrian positions in order to avoid false detection and to reduce the computational time.