KR101765568B1 - Pedestrian detecting system and control method thereof - Google Patents

Abstract

Disclosed are a pedestrian recognition system and a control method thereof. A method for a vehicle to recognize a pedestrian according to an embodiment of the present invention comprises the steps of: photographing an external image of a vehicle; detecting a pedestrian candidate in an interest area of the photographed image; calculating a score for the detected pedestrian candidate based on a pre-stored basic pedestrian model; determining as a pedestrian when the score is greater than or equal to a preset threshold value, and alerting the pedestrian according to a distance between the pedestrian and the vehicle.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a pedestrian recognition system,

The present invention relates to a pedestrian recognition system for a vehicle and a control method thereof, and more particularly, to a pedestrian recognition system and a control method thereof for improving the accuracy of pedestrian recognition.

A system has been developed to notify the driver that a pedestrian is located in front of the vehicle by recognizing the pedestrian from the image photographed by the camera installed in the front of the vehicle in case the driver can not detect the external situation.

For example, in the nighttime, the driver's view may suddenly decrease so that the driver may not see the pedestrian. In order to prevent such a case, researches on a system that enables a driver to secure a view by using a sensor such as a near infrared ray camera or a far-infrared ray camera, or a warning system by recognizing a pedestrian at night,

In addition, with respect to the pedestrian protection, a system has been developed to prevent a pedestrian accident in advance by recognizing a pedestrian using a camera mounted on the vehicle, and giving an alarm to the driver or controlling the vehicle to brak have.

In particular, Haag Wavelet Feature based method, Histogram of Oriented Gradient (HOG) method and LRF (Local Receptive Field) are mainly studied as pedestrian detection methods. It is possible to show the characteristics of the object more stably with respect to distortion such as tilt and size change compared to the existing HOG feature.

However, there is a problem in recognizing an obstacle such as a tree, a tire, and a door (hereinafter referred to as "false positives") as a pedestrian, rather than a pedestrian when recognizing a pedestrian.

In addition, there is a problem that the performance of the pedestrian is recognized as a pedestrian (hereinafter referred to as "True Positives") when the probability of recognizing false positives is reduced.

U.S. Published Patent Application No. US 2013/0136308 A1

Embodiments of the present invention are intended to improve the accuracy of pedestrian recognition.

In other words, the problem of erroneously recognizing an obstacle as a pedestrian while maintaining the accuracy of recognizing a pedestrian as a pedestrian is improved.

According to an aspect of the present invention, there is provided a method of detecting a pedestrian, comprising: photographing an external image of a vehicle; detecting a candidate for a pedestrian in the photographed image; Calculating an accuracy based on a local binary pattern (LBP) method for an obstacle that is not a pedestrian among the pedestrian candidates, calculating a score based on the score and the accuracy, And determining that the pedestrian is a pedestrian if the calculated result value exceeds a preset first threshold value.

If the result of the Histogram of Oriented Gradient (HOG) method for the pedestrian candidate is less than the preset second threshold value, it can be determined that the pedestrian candidate is not a pedestrian (False Positives).

Also, the score may be an average value corresponding to one or more pre-stored pedestrian models.

In addition, the previously stored one or more pedestrian models may be trained based on a plurality of pedestrian candidates.

In addition, the pre-stored basic pedestrian model can be trained in accordance with the pose or direction of the pedestrian candidate determined as the pedestrian.

If it is determined that the pedestrian is a pedestrian, a step of displaying a bounding box on the outer side of the determined pedestrian and displaying the bounding box to the user may be further included.

According to another aspect of the present invention, there is provided an image processing apparatus including an image capturing unit for capturing an external image of a vehicle, an image processor for detecting a candidate for a pedestrian in the captured image, Calculates a score based on the stored basic pedestrian model, calculates an accuracy according to a LBP (Local Binary Pattern) method for an obstacle that is not a pedestrian among the pedestrian candidates, and calculates a linear model based on the score and the accuracy And determining that the pedestrian is a pedestrian when the calculated result value exceeds a first threshold value that is set in advance.

The control unit may further include a pedestrian candidate recognizing unit for determining that the result of the Histogram of Oriented Gradient (HOG) method is less than a second threshold, .

Also, the score may be an average value corresponding to one or more pre-stored pedestrian models.

Also, the control unit may calculate an accuracy according to a LBP (Local Binary Pattern) method for an obstacle that is not a pedestrian among the pedestrian candidates, and output a result calculated according to the linear model based on the score and the accuracy, And a pedestrian candidate determining unit determining that the pedestrian is a pedestrian when the first threshold value is exceeded.

Also, the controller may be configured to train the one or more pre-stored pedestrian models based on a plurality of pedestrian candidates, or to divide and train the pre-stored basic pedestrian models according to pose or direction of the pedestrian candidates judged as pedestrians, And a model training unit.

In addition, if the pedestrian is determined to be a pedestrian, the display unit may display a bounding box on the outer side of the determined pedestrian and display the bounding box to the user.

Embodiments of the present invention can improve the accuracy of pedestrian recognition.

That is, the probability of erroneously recognizing an obstacle as a pedestrian can be reduced while maintaining the accuracy of recognizing the pedestrian as a pedestrian.

1 is a block diagram of a pedestrian recognition system according to an embodiment of the present invention.
2 is a block diagram of a control unit according to an embodiment of the present invention.
3 is a detailed block diagram of a controller configuration according to an embodiment of the present invention.
4 is a schematic diagram illustrating pedestrian recognition according to an embodiment of the present invention.
5 is a schematic diagram illustrating a pedestrian model cluster stored in a controller according to an embodiment of the present invention.
FIG. 6 is a schematic view for explaining recognition size according to distance when recognizing a pedestrian according to an embodiment of the present invention.
7 is a schematic diagram illustrating a method for removing noise in pedestrian recognition according to an embodiment of the present invention.
8 is a flowchart illustrating a pedestrian recognition control method according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments are provided to fully convey the spirit of the present invention to a person having ordinary skill in the art to which the present invention belongs. The present invention is not limited to the embodiments shown herein but may be embodied in other forms. For the sake of clarity, the drawings are not drawn to scale, and the size of the elements may be slightly exaggerated to facilitate understanding.

FIG. 1 is a block diagram of a pedestrian recognition system 1 of a vehicle according to the present invention, and FIG. 2 is a block diagram of a controller of a pedestrian recognition system 1 according to an embodiment of the present invention.

Referring to FIG. 1, a pedestrian recognition system 1 of a vehicle includes an image capturing unit 10, an image processing unit 11, a control unit 20, and a display unit 30.

The image capturing unit 10 may include a camera sensor (not shown) for capturing an image of the front of the vehicle 2, thereby capturing an external image of the vehicle and generating shooting information thereon.

Specifically, a camera sensor usually uses a camera having more than one channel, and a CMOS (Complementary Metal Oxide Semiconductor) can be used as the image sensor. A CMOS image sensor is a semiconductor device that converts exposed images into electrical form and transmits them.

However, the present invention is not limited thereto, and may be realized by a CCD (Change Coupled Device) image sensor (not shown). Thus, the image obtained through the camera sensor is transmitted to the image processing unit 11.

That is, an image captured through a camera sensor in the image capturing unit 10 is captured and transmitted to the image processing unit 11.

The image processing unit 11 processes the image received from the image capturing unit 10. That is, the image processing unit 11 can detect motion of an object photographed in the image, extract an area of the object, and distinguish a plurality of objects. At this time, the corresponding object is an object to be recognized of the image processing. For example, it may be an object such as a landscape, a foreground, a specific object, a pedestrian, and the like.

The image processor 11 extracts an outline of an object from the acquired image using a deviation of neighboring RGB values, and analyzes the change of the outline by comparing the outline of the object with the previous image to determine the motion of the object.

In addition, an area where motion of an object occurs can be obtained by using the detected area period as a set of setting intervals.

In addition, two or more objects may be merged or divided in one image. In order to distinguish an area of an accurate object, an estimated position and size of the object through object tracking may be calculated to extract the position and area of the object .

At this time, the image processing unit 11 transmits information of the extracted object including the image information acquired by the image capturing unit 10 to the control unit 20 at predetermined time intervals.

Accordingly, the image processing unit 11 can transmit the detected motion of the object photographed in the image, the region of the object, and information in which a plurality of objects are superimposed to the control unit 20 using the image signal received by the image capturing unit 10 have.

The control unit 20 controls the pedestrian recognition system 1 as a whole.

Specifically, the control unit 20 mediates data input / output between various constituent devices included in the pedestrian recognition system 1 of the vehicle 2 and the control unit 20, and includes a memory 22 for storing programs and data, A main processor 21 for recognizing the pedestrian using the image information received by the image processor 11 and performing contrast and model training between the obtained pedestrian and the basic model, And a communication unit (not shown) for communicating with a warning unit (not shown) for warning the driver in accordance with the display unit 30 and the pedestrian recognition result.

2, the control unit 20 includes a pedestrian candidate recognizing unit 210 for recognizing a pedestrian candidate sensed by software, a pedestrian model training unit for training a pedestrian model based on the recognized pedestrian, A pedestrian candidate determining unit 230 for determining a recognized pedestrian as a pedestrian candidate, and a pedestrian tracking unit 240 for tracking the determined pedestrian candidate.

As shown in FIG. 4, the pedestrian candidate recognizing unit 210 recognizes that the pedestrian is recognized as a pedestrian (True Positives) T1 to T4, F1 and F2 displayed on the outer periphery of the recognized object, (False Positives).

These steps can be distinguished by HOG (Histogram of Oriented Gradient) method.

The HOG method detects the inclination of a pedestrian's outer edge in a pedestrian extracted from an image, cuts a certain area including the pedestrian at a finely spaced interval, and generates a feature point generated through a histogram of the inclination direction of the pedestrian outline for each of the subdivided areas This is a method of detecting a pedestrian based on the basis.

More specifically, T1 to T4 in FIG. 3 schematically show a case where a pedestrian is recognized as a pedestrian (True Positives), and F1 to F2 indicate a case where an object other than a pedestrian is recognized as a pedestrian (False Positives).

That is, the degree of coincidence of the detected HOG feature points of the pedestrian obtained by the image processing unit 11 with the pedestrian model in the plurality of communities of FIG. 4 is scored, and when the average value of the scores is equal to or greater than a preset threshold value It is recognized as a pedestrian.

In this case, the score refers to an average value corresponding to one or more pre-stored pedestrian models.

In this case, the accuracy of pedestrian recognition can be improved without affecting the probability value of recognizing a pedestrian as a pedestrian while reducing the probability of recognizing an object other than a pedestrian as a pedestrian.

5, the controller 20 separately compares the pre-stored basic pedestrian model into a plurality of communities, compares the pedestrian information with pedestrians acquired from the image processing unit 11, Let's learn again in plural communities.

6, the pedestrian candidate recognizing unit 210 sets a region of interest (ROA) in the acquired image in the image processing unit 11, and when the pedestrian in the ROI is detected, It can be recognized as a pedestrian only if it does not exceed the limit of the height (Height). This is for carrying out a re-scaling operation of a plurality of pedestrian models shown in Fig. 5 and a recognized object before comparison with a certain size.

6, assuming that the pedestrian P1 and the pedestrian P2 are the same size (elongation), the image pickup section 10 of the vehicle 2 is moved in accordance with the distance from the vehicle 2, The size of the pedestrian in the image acquired by the user is different.

That is, the pedestrian P1 is larger than the pedestrian P2, and the size of the pedestrian P1 as seen in the image capturing unit 10 is larger than the pedestrian P2.

Therefore, the image processing unit 11 performs a re-scaling operation so that it can be compared with the pedestrian model shown in Fig. 5 in proportion to the detected distance to the vehicle 2 of the pedestrian.

5 is a schematic diagram illustrating a pedestrian model cluster stored in a controller according to an embodiment of the present invention.

As shown in FIG. 5, the first group is divided into a first group and a third group. The first group includes a group that learns a front view of a pedestrian, and the second group includes a side view of a pedestrian , And the third cluster is a cluster that learns the appearance of a pedestrian including the profile of a pedestrian moving to the right.

However, the present invention is not limited to this, and it is possible to further include the clusters in more detail than the first to third clusters. For example, a cluster may be subdivided into a stationary front view, a stationary left view, a stationary right view, and a group of pedestrians on the move.

Accordingly, when an external image is photographed through the image capturing unit 10, a plurality of pedestrians included in the image are recognized as a pedestrian by the pedestrian candidate recognition unit 210, and an obstacle other than a plurality of pedestrians is recognized as a pedestrian The pedestrian model training unit 220 newly learns when the pedestrian is recognized as a pedestrian (True Positives).

In addition, when the external image is photographed through the image capturing unit 10, the pedestrian candidate recognizing unit 210 recognizes a plurality of pedestrians included in the image as a pedestrian, and recognizes an obstacle other than a plurality of pedestrians as a pedestrian The pedestrian model training unit 220 newly learns about false positives when it recognizes an obstacle that is not a pedestrian as a pedestrian.

3, the pedestrian candidate determining unit 230 includes a pedestrian model comparing unit 231, an LBP comparing unit 232, and an entropy comparing unit (not shown).

First, as described above, the pedestrian model comparator 231 calculates a score, which is an average value in agreement with a plurality of pedestrian models.

Next, when the LBP comparison unit 232 recognizes an object that is not a pedestrian as a pedestrian (false positive), the LBP comparison unit 232 again checks whether the pedestrian is a pedestrian using the Local Binary Pattern (LBP) method. The LBP method recognizes the texture of the image using the difference information between the center pixel and the surrounding pixels of the image. A pixel having a large brightness value based on the center pixel is set to 1, a small pixel is set to 0, Use the bit pattern as a feature point to check whether you are a pedestrian.

In one embodiment, 256 (2 ⁸ ) values may be set for individual pixels divided for the pedestrian, but are not necessarily limited thereto. Accordingly, pedestrian accuracy is calculated on the basis of the pedestrian texture information as another value is given according to the brightness.

Thereafter, the pedestrian candidate determining unit 230 calculates the output value of the linear model based on the score obtained by the pedestrian model comparing unit 231 and the accuracy secured by the LBP comparing unit 232, If the threshold value is larger, it is determined to be a pedestrian.

In addition, the pedestrian candidate determining unit 230 may calculate entropy when the pedestrian recognition system 1 according to the present invention recognizes an object other than a pedestrian as a pedestrian (false positive).

In this case, entropy is a stochastic measure of randomization information on image texture.

Next, Fig. 7 is an added process for removing noise.

As shown in FIG. 7, the pedestrian recognition system 1 according to the present invention performs elliptical noise removal steps in order to reduce noise in pedestrian recognition in operation.

First, determine the major axis and minor axis of the ellipse that touches the outline of the pedestrian.

Thereafter, it is possible to set a parallelogram which is in contact with the ellipse. 7, each side of the set parallelogram can be formed as an intersection point of the diagonal lines L1 to L4, and the diagonal line of the parallelogram is defined by the major axis D1 and the minor axis D2 of the ellipse contacting the pedestrian contour .

Thereafter, points other than the pedestrian edge are deleted by selecting points determined as noise based on the lines L1 to L4, thereby removing the noise.

Next, the pedestrian tracking unit 240 tracks the pedestrian determined by the pedestrian candidate determining unit 230 at predetermined time intervals.

The display unit 30 indicates that a user of the vehicle 2 is recognized as a pedestrian. When the pedestrian is recognized as a pedestrian, a bounding box can be displayed squarely on the outside of a plurality of pedestrians, May be displayed together with the distance information.

The main processor 21 stores the information processed by the pedestrian candidate recognizing unit 210, the pedestrian model training unit 220, the pedestrian candidate determining unit 230, and the pedestrian tracking unit 240 in the memory 22.

That is, the memory 22 stores a control program and control data for controlling the operation of the pedestrian recognition system 1, various images including the pedestrian recognition information processed by the main processor 21 and the image acquired by the image processing unit 11 The control signal and the like can be temporarily stored.

Particularly, the memory 22 may be a volatile memory such as a S-RAM or a D-RAM, as well as a flash memory, a read only memory, an erasable programmable read only memory (EPROM) And non-volatile memory such as Electrically Erasable Programmable Read Only Memory (EEPROM).

 Specifically, the nonvolatile memory may semi-permanently store control programs and control data for controlling the operation of the pedestrian recognition system 1, and training information of the pedestrian model, and the volatile memory may store control programs and control data And can temporarily store the image received by the image processing unit 11 and the pedestrian information processed by the control unit 20. [

The configuration of the pedestrian recognition system 1 according to the embodiment has been described above.

Hereinafter, the operation of the pedestrian recognition system 1 according to the embodiment will be described.

8 is a schematic view illustrating a pedestrian recognition method of a pedestrian recognition system according to an embodiment of the present invention.

The pedestrian recognition system 1 according to the present embodiment detects pedestrian candidates (S10). Specifically, the image processing unit 10 can detect a plurality of pedestrian candidates through the images photographed by the image photographing unit 10.

At this time, the image processing unit 11 transmits information of the extracted object including the image information acquired by the image capturing unit 10 to the control unit 20 at predetermined time intervals.

The control unit 20 recognizes the pedestrian and determines whether the recognized pedestrian is a true positives or a false positives (S20). Specifically, it means a case where a pedestrian is recognized as a pedestrian (True Positives), and a case where an obstacle other than a pedestrian is recognized as a pedestrian (False Positives).

At this time, if the value calculated by the HOG method exceeds a preset threshold value, the pedestrian is recognized as a pedestrian (True Positives). If the value calculated by the HOG method is less than a predetermined threshold value, (False Positives) as a pedestrian (S30).

Next, since the false positives can be recognized as an obstacle rather than a pedestrian in spite of being a pedestrian, it is determined again whether or not the pedestrian is a pedestrian through a linear combination of a plurality of pedestrian recognition methods.

First, it is compared with a plurality of pedestrian community models shown in FIG. 5 (S1) (S40). this is

 In step S200, an object judged as an obstacle by the sensed pedestrian candidate can be identified as a pedestrian, and an object determined as an obstacle other than the pedestrian as a pedestrian candidate is compared with the basic pedestrian model stored in the pedestrian recognition system 1 (S200). That is, the control unit 20 calculates a score which is an average value of degree of agreement with the basic pedestrian models.

When the pedestrian recognition system 1 according to the present invention recognizes an object that is not a pedestrian as a pedestrian (false positive), the pedestrian recognition system 1 again checks whether the pedestrian is a pedestrian by using a Local Binary Pattern (hereinafter referred to as "LBP" ) (S50).

Thereafter, the pedestrian recognition system 1 according to the present invention calculates a result value through a predetermined linear model for the values obtained in S1 to S2 (S70), and compares the calculated result value with a predetermined threshold value If it is determined that the pedestrian is not pedestrian, the pedestrian is determined as a pedestrian (false positive) (S80).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limited to the embodiments set forth herein; It will be understood that various modifications may be made without departing from the spirit and scope of the invention.

1: pedestrian recognition system
2: vehicle

Claims (12)

Capturing an external image of the vehicle;
Sensing a pedestrian candidate in the photographed image;
Recognizing the pedestrian candidate as a false positive or a pedestrian (True Positives) rather than a pedestrian;
Training a pre-stored pedestrian model based on the true positives;
Calculating a score based on the pre-stored basic pedestrian model for the non-pedestrian obstacles (False Positives);
Calculating an accuracy according to an LBP (Local Binary Pattern) method for an obstacle other than the pedestrian;
And determining that the pedestrian is a pedestrian if the calculated result based on the score and the accuracy is greater than a predetermined first threshold value. The method according to claim 1,
If the result of the Histogram of Oriented Gradient (HOG) method is less than a second predetermined threshold value for the pedestrian candidate, the pedestrian candidate is determined to be False Positives. 3. The method of claim 2,
Wherein the score is an average value corresponding to one or more pre-stored pedestrian models. The method of claim 3,
Wherein the one or more pre-stored pedestrian models are trained based on the obstacles (True Positives) and obstacles other than the pedestrian exceeding the first threshold value. 5. The method of claim 4,
Wherein the pre-stored basic pedestrian model is trained in accordance with a pose or direction of the pedestrian candidate determined as the pedestrian. 6. The method according to any one of claims 1 to 5,
And displaying a bounding box on the outer periphery of the determined pedestrian when the pedestrian is determined to be a pedestrian, and displaying the bounding box to the user. An image capturing unit for capturing an external image of the vehicle;
An image processor for detecting a pedestrian candidate in the photographed image;
Recognizes the pedestrian candidate as a false positives or a pedestrian (True Positives) rather than a pedestrian,
Training a pre-stored pedestrian model based on the true positives,
Calculating a score based on the pre-stored basic pedestrian model for the non-pedestrian obstacles (False Positives), calculating an accuracy according to the LBP (Local Binary Pattern) method for the obstacles other than the pedestrian, And determining that the pedestrian is a pedestrian if the result calculated based on the score and the linearity model exceeds a preset first threshold value. 8. The method of claim 7,
The control unit may further include a pedestrian candidate recognizing unit for determining that the result of the Histogram of Oriented Gradient (HOG) method is less than a second threshold value that is set in advance and is a pedestrian candidate (False Positives) Pedestrian recognition system. 9. The method of claim 8,
Wherein the score is an average value corresponding to one or more pre-stored pedestrian models. 10. The method of claim 9,
The control unit
Calculating an accuracy according to a LBP (Local Binary Pattern) method for an obstacle that is not a pedestrian among the pedestrian candidates, and calculating a result based on the score and the accuracy based on the linear model exceeds a preset first threshold And a pedestrian candidate determining unit determining that the pedestrian is a pedestrian. 11. The method of claim 10,
Wherein the controller is configured to train the one or more pre-stored pedestrian models based on obstacles other than the pedestrian exceeding the first threshold and the True Positives, or the pre-stored basic pedestrian model is judged as the pedestrian And a pedestrian model training unit that trains the pedestrian model according to a pose or a direction of the pedestrian candidate. 12. The method according to any one of claims 7 to 11,
And displaying a bounding box on the outer periphery of the determined pedestrian when the pedestrian is determined to be a pedestrian, and displaying the bounding box to a user.

Images