KR101770924B1 - System and method for Active Pedestrian Protection - Google Patents

Abstract

The present invention relates to an active pedestrian protection system and a method thereof for improving real-time property. The active pedestrian protection method includes: a candidate generation step of extracting a pedestrian candidate object from an input image; a risk assessment step of calculating region of danger (ROD) through statistical analysis of the running data and filtering the candidate object of the pedestrian based on the ROD; and a candidate verification step of performing a candidate verification on each of the filtered candidate objects to detect a pedestrian in a dangerous state.

Description

[0001] The present invention relates to a system and method for active pedestrian protection,

The present invention relates to an active pedestrian protection system and method, and more particularly, to an active pedestrian protection system and method that can improve system real time by minimizing the time required for pedestrian detection.

 The active pedestrian protection system is a system that anticipates pedestrian pedestrian accidents and prevents them by emergency braking.

The existing active pedestrian protection system generally performs three stages of Hypothesis Generation (HG), Hypothesis Verification (HV), and Risk Assessment (RA) in sequence.

HG is implemented as a relatively lightweight algorithm, and candidates are selected for areas where pedestrians are expected to exist in the entire image. The HV is implemented with a relatively heavy but accurate algorithm and evaluates whether the generated candidate is a pedestrian. The RA assesses the risk of an accident by evaluating whether the detected pedestrian is on the route of the vehicle.

Such an active pedestrian protection system uses a non-irreversible means such as emergency braking as compared with a system that provides only an alarm, so that the cost due to malfunction is very large. Malfunctions are caused by false detection of pedestrians as false pedestrians and path prediction errors that are considered to be dangerous to safe pedestrians by mistakenly predicting the route of the vehicle. Therefore, in order to secure practicality and consumer acceptability, it is necessary to minimize malfunction.

A typical example of a method for minimizing the detection of pedestrian misfire is a method of performing HV through CNN (Convolutional Neural Network). CNN has been reported to outperform existing systems in various recognition fields and has a very low false detection rate in pedestrian recognition. However, CNN is considerably heavier than existing systems and has a lot of difficulties in real-time implementation. Therefore, there have been proposed methods of using high-speed hardware such as GPU (Graphic Processing Unit) or FPGA and reducing the number of pedestrian candidates by using relatively light detector such as ACF (Aggregated Channel Feature) as HG, It has a disadvantage that it is difficult to secure.

Therefore, it is expected that the real - time improvement of HV implemented through CNN will be an important means of minimizing the detection of pedestrian error for the time being.

David Gerbenimo, Antonio M. L. Pez, Angel D. Sappa, and Thorsten Graf, "Survey of Pedestrian Detection for Advanced Driver Assistance Systems," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 7, Jul. 2010, pp. 1239-1258.

In order to solve the above problems, the present invention extracts only a dangerous pedestrian candidate through a risk assessment (RA), and performs a HV (Hypothesis Verification) so that a relatively heavy HV processing burden And to provide an active pedestrian protection system and method for improving the real-time performance of the HV.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method for extracting a pedestrian candidate object from an input image, A risk assessment step of calculating ROD (Region of Danger) through statistical analysis of the running data and filtering the pedestrian candidate object based on the ROD; And a candidate verification step of performing a candidate verification on each of the filtered candidate candidate objects to detect a pedestrian in a dangerous state.

The risk evaluation step may include calculating a region of a prediction path (ROPP), which is an area where a future route of the vehicle may exist, through statistical analysis of the driving data; Expanding the left and right outer regions of the ROPP according to a predetermined criterion to calculate the ROD; And selecting and outputting only the pedestrian candidates located in the ROD among the pedestrian candidate objects extracted through the candidate generation step.

" 및 "

"를 근거로 예측시간 T 동안의 예상 경로 끝점(x(T), y(T))을 계산하는 단계; "Φ(T)=tan^-1(y(T)/x(T))"의 식에 따라 상기 예상 경로 끝점의 위상각(Φ(T))을 계산하는 단계; "

"에 근거해 최대 방향 오류(ΔΦmax(v))를 계산하는 단계; 상기 최대 방향 오류(ΔΦ_max(v))를 기반하여 "Φ(T)-ΔΦ_max(v) ~ Φ(T)+ΔΦ_max(v)"를 ROPP의 위상각 범위로 정의하는 단계; "

"에 근거해 최대 가속도(a_max(v))를 계산하는 단계; "

"에 근거해 최대 도달거리(s_max(v))를 계산하는 단계; 및 반지름이 s_max(v)이고 위상각이 Φ(T)-ΔΦ_max(v) 범위인 부채꼴로 정의되는 ROPP를 산출하는 단계를 포함하며, 상기 x 및 y는 차량 전방과 좌측을 x축과 y축으로 하는 좌표계 상의 좌표를, 상기 v은 속도를, 상기 ω는 각속도를, 상기 t는 시각을, 상기 T는 예측 시간을, 상기 a, b, c, d는 함수의 형태와 최대값, 최소값을 결정하는 매개변수 각각을 나타내는 것을 특징으로 한다. The step of calculating the ROPP may be "

"And"

"Calculating a predicted path end point (x (T), y ( T)) for the prediction time T on the basis of;" Φ (T) = tan -1 (y (T) / x (T)) " of Calculating a phase angle (? (T)) of the expected path endpoint according to the equation:?

"Direction to the maximum error (ΔΦmax (v)) calculating, based on; and based on the direction of maximum error _{(ΔΦ max (v))"} Φ (T) -ΔΦ max (v) ~ Φ (T) + ΔΦ _max (v) "as the phase angle range of ROPP;

(A _max (v)) based on "

Calculating a maximum reaching distance (s _max (v)) on the basis of the radius s _max (v) and a phase angle Φ (T) -ΔΦ _max (v) Wherein x and y are coordinates on a coordinate system having a front and a left side of the vehicle as an x axis and a y axis, v is a velocity,? Is an angular velocity, t is a time, And a, b, c, and d respectively represent a function type, a parameter for determining a maximum value and a minimum value, respectively.

The step of calculating the ROD includes: connecting lateral ends of a sector corresponding to the ROPP to obtain a vehicle lateral width (W); And obtaining the ROD by extending each of the left and right lateral outer regions of the ROPP by half (W / 2) of the vehicle lateral width (W), respectively.

Selecting and outputting only the pedestrian candidates located in the ROD may include projecting the ROD onto the input image; And comparing the ROD position projected on the input image with the position of the candidate pedestrian object and selecting and outputting only the pedestrian candidate object positioned in the ROD as a verification object.

"에 따라 ROD의 차량 중심 좌표계를 입력 영상 좌표계로 변환하며, 상기 x(u, v, 1)는 입력 영상 좌표계, 상기 X(X, Y, Z, 1)는 차량 중심 좌표계, 상기 P는 카메라의 투사 매트릭스인 것을 특징으로 한다. Wherein projecting the ROD to the input image comprises:

"Converts the vehicle center coordinate of ROD to the input image coordinate system, the x (u, v, 1) is the input image coordinate system, the X (X, Y, Z, 1) is a vehicle central coordinate system, wherein P is the camera according to the Is a projection matrix of < / RTI >

According to another aspect of the present invention, there is provided, as a means for solving the above-mentioned problems, a method for generating a pedestrian candidate object, A risk assessment unit for calculating the ROD (Region of Danger) through statistical analysis of the running data and filtering the pedestrian candidate object based on the ROD; And a candidate verifying unit for performing candidate verification on each of the filtered candidate candidate objects to detect a pedestrian in a dangerous state.

Wherein the risk evaluation unit includes: an ROPP calculation unit for calculating a region of prediction path (ROPP), which is an area where a future route of the vehicle can exist, through a statistical analysis of the traveling data; An ROD calculation unit for calculating the ROD by expanding the left and right outer regions of the ROPP according to predetermined criteria; And a pedestrian candidate selecting unit for selecting only the pedestrian candidates located in the ROD among the pedestrian candidate objects extracted through the candidate generating unit and transmitting the selected pedestrian candidates to the candidate verifying unit.

In the present invention, when detecting a pedestrian for active pedestrian protection according to the three steps of candidate generation, candidate verification, and risk evaluation, the risk evaluation is performed before the candidate verification, considering that the processing load of the candidate verification is relatively large. As a result, the processing load of the candidate verification is minimized and the processing time is reduced, thereby improving the real-time performance of the entire system.

1 is a view for explaining an active pedestrian protection method according to an embodiment of the present invention.
2 is a diagram illustrating a problem of a physics-based path prediction using a CTRV motion model that can be generated at the time of vehicle rotation.
Fig. 3 is a view for explaining a range of longitudinal motion states varying according to the relationship between velocity and acceleration. Fig.
4 is a diagram for explaining the range of the lateral motion state which changes according to the relationship between the velocity and the direction error.
5 is a diagram for explaining the definition of a directional error required for analysis of a lateral motion state range.
6 is a diagram for explaining the ROPP calculation step according to an embodiment of the present invention in more detail.
7 and 8 are views for explaining the ROD calculation step according to an embodiment of the present invention in more detail.
9 and 10 are views for explaining the step of selecting a candidate for a pedestrian according to an embodiment of the present invention in more detail.
11 is a view for explaining an active pedestrian protection system according to an embodiment of the present invention.

It is noted that the technical terms used in the present invention are used only to describe specific embodiments and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be construed in a sense generally understood by a person having ordinary skill in the art to which the present invention belongs, unless otherwise defined in the present invention, Should not be construed to mean, or be interpreted in an excessively reduced sense. In addition, when a technical term used in the present invention is an erroneous technical term that does not accurately express the concept of the present invention, it should be understood that technical terms can be understood by those skilled in the art. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Furthermore, the singular expressions used in the present invention include plural expressions unless the context clearly dictates otherwise. In the present invention, terms such as "comprising" or "comprising" and the like should not be construed as encompassing various elements or various steps of the invention, Or may further include additional components or steps.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to the same or similar elements, and redundant description thereof will be omitted.

In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings.

1 is a view for explaining an active pedestrian protection method according to an embodiment of the present invention.

Referring to FIG. 1, the active pedestrian protection method of the present invention includes a candidate generation (HG) step S10 for extracting a pedestrian candidate object from an input image, an ROD (Region of Danger) from the driving data, A risk evaluation (RA) step S20 for selecting candidates for possible pedestrian candidates and a candidate verification operation based on a CNN (Convolutional Neural Network) are performed for each of the selected candidate pedestrian objects to detect a pedestrian in a dangerous state And a candidate verification (HV) step S30 to notify.

That is, in the present invention, the RA step S20 is performed before the HV step S30, unlike the conventional method. If the operation sequence is changed, a larger number of inputs must be processed in the RA step S20 than in the conventional case, but a smaller number of inputs are processed in the HV step S30 than in the conventional case. However, since the calculation request amount of the HV step S30 is much larger than that of the RA step S20, the method of the present invention can have a shortened operation time as compared with the conventional method.

For example, it is assumed that each of the HV step (S30) and the RA step (S20) consumes 10 ms and 0.1 ms to process one pedestrian candidate (this means that the calculation requirement of the HV step (S30) Assuming that it is 100 times as large as it is, it generally has a larger value).

In this case, if 10 pedestrian candidates are output at the HG level, HV outputs 4 pedestrians, and 2 pedestrians are output at the RA stage, HV and RA are operated 10 times and 4 times, respectively, The time consumed is "10 ms × 10 + 0.1 ms × 4 = 100.4 ms".

On the other hand, if 10 pedestrian candidates are output at the HG level, the RA outputs 4 dangerous pedestrian candidates, and the HV outputs 2 dangerous pedestrians, the present invention operates RA 10 times and HV 4 times, Is "0.1 ms x 10 + 10 ms x 4 = 41 ms ".

That is, since the execution time of the HV step is much larger than the execution time of the RA step, the system execution time is approximately proportional to the number of inputs processed by the HV step.

Therefore, the present invention reduces the information processing amount in the HV step, thereby shortening the operation time of the entire system, and consequently improving system real-time performance.

1, the RA step S20 of the present invention includes a step S21 of calculating a region of a prediction path (ROPP) which is an area where a future route of the vehicle can exist through statistical analysis of the traveling data of the vehicle (Step S22) of calculating a dangerous area ROD (Region of Danger) based on the ROPP, and a step of selecting only a pedestrian candidate located in the ROD among a plurality of pedestrian candidates as a candidate pedestrian candidate , And transmitting it to the HV step S30 (S23).

Hereinafter, the RA step S20 according to an embodiment of the present invention will be described in more detail with reference to FIG. 2 to FIG.

First, the present invention utilizes a physics-based path prediction method to grasp ROPP.

For reference, vehicle path prediction can be largely performed in three ways: physics-based, maneuver-based, and interaction-based. The physics-based prediction method is to predict the path by extending the current motion state such as speed, acceleration, and angular velocity. This is a relatively light algorithm that is easy to implement in real time. However, it has a limited predictable time and a large error Lt; / RTI > The reason for this uncertainty is that the vehicle movement is basically steered continuously according to the intention of the driver. The intention-based prediction method has the advantage of reducing the uncertainty by grasping the driver's intention. However, since it takes a burden to process various large-volume information such as map, forward image, and driver image in order to understand the intention of the driver, Is still limited. The mutual-based prediction method is a method to consider the interaction between vehicle and other vehicle, and it is still in the early stage of development.

Accordingly, the present invention predicts ROPP using a physics-based prediction method.

Physics-based path prediction is based on a predicted path endpoint (x (t), y (t)) based on the Constant Turn-Rate and Velocity (CTRV) motion model assuming that the vehicle velocity and angular velocity are constant over a short period of time. Can be calculated.

[Equation 1]

&Quot; (2) "

In this case, x and y are coordinates on the coordinate system having the x-axis and the y-axis as the front and the left sides of the vehicle, v is the vehicle speed, ω is the angular speed, and t is the time. t can be increased from the current time point 0s to the maximum predicted time T (for example, 3 seconds).

On the other hand, the CTRA motion model frequently used for vehicle pose estimation has a problem that it is not suitable for path prediction including low-speed travel. It is because the CTRA motion model has a problem of predicting the future route in the vehicle backward direction.

Also, the CTRV motion model has a problem of generating a large error during rotation. In contrast to the curved line which keeps the steering angle constant, the driver gradually increases the steering angle gradually and then adjusts it gradually smaller. At this time, since the turning radius becomes smaller as the steering angle becomes larger, Because a large error is generated when the future path is predicted.

2 is a diagram illustrating a problem of a physics-based path prediction using a CTRV motion model that can be generated at the time of vehicle rotation.

In FIG. 2, it is assumed that the blue dotted line expresses the actual path during the period of t = 0 to T, the red dotted line represents the predicted path, and the vehicle rotates along the green path from the left to the upper right.

2 (a) is a view showing the actual path and the predicted path of the vehicle immediately before the left turn. In the actual path, the vehicle speed is decreased and the direction change is started upward. However, in the physics-based path prediction method, The straight line path that maintains the current speed is predicted.

FIG. 2 (b) is a diagram showing an actual path and a predicted path of the vehicle in a situation where the left turn is almost completed. In the actual path, the vehicle speed increases and starts to go straight upward. However, Because it is in the path, it predicts the curved path that maintains the current turning radius.

2 (c) is a graph of estimated curvature (assumed to be an actual curvature, blue solid line) and estimated curvature (red solid line) calculated by fitting a real path and a predicted path to a circle during a left turn, It can be seen that it is difficult to compensate it by a simple method because the estimated curvature is smaller than the estimated curvature and the size of the error is not constant.

The limitation of such a physics-based path prediction occurs because the driver constantly adjusts the vehicle motion. In the case of rotating at a low speed, the above-mentioned error necessarily occurs because the magnitude and rate of change of the steering angle are both large.

Therefore, if it is not possible to determine the precise future path, the next best way is to predict the area where the future path can exist as small as possible. In the present invention, the probability of occurrence of the error is minimized by statistically analyzing the traveling data to predict the ROPP, which is a region where a future route can exist, in a sector shape having a minimum size.

The vehicle basically includes an electronic control chassis control system such as an electronic stability control (ESC), etc., and the ESC can provide driving data including a speed and an angular speed.

Therefore, the present invention predicts the vehicle path based on the speed and the angular speed included in the running data. In particular, the driver models the range of longitudinal / lateral motion according to the vehicle speed considering the considerably different driving behavior according to the vehicle speed, and then calculates the area ROPP in which the future route can exist based on the model.

At this time, the range of the longitudinal motion state is determined according to the relation between the velocity and the acceleration, and is determined according to the relationship between the lateral motion state range velocity and the direction error.

First, referring to FIG. 3, a description will be made of a range of longitudinal motion state that changes according to the relationship between velocity and acceleration.

3 (a) and 3 (b) are views showing a speed-acceleration occurrence frequency extracted from raw data such as running data. More specifically, FIG. 3 (a) shows the acceleration range according to the speed, which is expressed as a point regardless of the accumulation value to show the overall range, (b) shows the frequency of occurrence of the speed- Is expressed in height and expressed in a different angle in three dimensions.

3 (a) and 3 (b), it can be seen that as the vehicle speed increases, the range of acceleration decreases, and that the maximum acceleration at low speed is also limited within a certain range. Such a range of the acceleration can be expressed by a function defined by the maximum value and the minimum value and inversely proportional to the speed as shown in Equation (3).

&Quot; (3) "

Where a _max ( v ) is the magnitude of the maximum acceleration at velocity v . a and b are parameters that determine the type of function, and c and d are parameters that represent the maximum and minimum values, respectively.

FIG. 3 (c) shows the maximum acceleration function according to the velocity, and the red dotted line corresponds to the form of a _max ( v ) when these parameters are properly set. If the magnitude of the maximum acceleration is determined, the maximum movement distance s _max ( v ) that can be moved during the maximum predicted time T based on the current velocity v can be calculated according to Equation (4).

&Quot; (4) "

Referring to FIG. 4, the range of the lateral motion state varying according to the relationship between the velocity and the direction error will be described.

If the vehicle if this is assumed that the situation to rotate in the upper left, depending on the blue line 5, t = 0 ~ T between the actual path and the predicted path is represented as blue dotted line and the red dotted line, the direction error (ΔΦ) is It is defined as the angle between the actual path at the current location and the line segment connecting the end of the expected path. If we know the maximum directional error at a certain point in time, we can say that there is a future path within a certain range based on the predicted path direction although the exact future path is unknown.

4 (a) and 4 (b) are views showing the frequency of occurrence of the speed-direction error extracted from the running data (row data). More specifically, FIG. 4 (a) shows a range of directional error, and is expressed as a point regardless of the cumulative value to show the entire range. FIG. 4 (b) It is expressed from a different angle in three dimensions.

4A and 4B, it can be seen that, similarly to the maximum acceleration, the range of the direction error becomes smaller as the vehicle speed becomes larger, and the maximum error at low speed is also limited within a certain range .

Therefore, the range of the direction error can be expressed by a function limited to the maximum value and the minimum value as shown in Equation (5) and inversely proportional to the speed.

&Quot; (5) "

At this time, ΔΦ _max ( v ) represents the maximum directional error at the velocity v. As in Equation (3), a , b , c , and d are parameters for determining the type, maximum value, and minimum value of the function.

Fig. 4 (c) shows the maximum error function according to the velocity, and the red dotted line shows the form of ?? _max ( v ) when these parameters are properly set.

When such longitudinal / lateral motion state range modeling is completed, the present invention can calculate a region ROPP in which a future path can exist based on the longitudinal / lateral motion state range model through the ROPP calculation step S21.

FIG. 6 is a diagram for explaining the ROPP calculation step S21 according to an embodiment of the present invention in more detail.

First, the current speed v and the angular speed omega of the vehicle included in the running data are extracted and acquired (S211).

And then calculating equation (1) and equation (2) based on a predicted path end point (x (T), y ( T)) to (S212), the phase of the predicted path of each end point ^{(Φ (T) = tan -1} (y ( T ) / x ( T )) (S213).

And based on the equation (5) up to direction errors (ΔΦ _max (v)) after the calculation (S214), up to direction errors (ΔΦ _max (v)) the phase angle range of ROPP based (that is, Φ (T) - ΔΦ _max (v) ~ Φ and to calculate a _{(T) + ΔΦ max (v} )) (S215).

Then, the maximum acceleration a _max ( v ) is calculated based on Equation (3) (S216), and the maximum reaching distance s _max ( v ) is calculated based on Equation (4) (S217).

Finally, the ROPP defined as a sector whose radius is s _max ( v ) and whose phase angle is in the range of ? ( T ) - ?? _Max ( v ) is calculated (S218).

When the ROPP is calculated through the ROPP calculation step S21 of FIG. 6, the present invention calculates the ROD based on ROPP as shown in FIG. 7 and projects it to the input image to calculate the ROPP based on the pedestrian candidate generated in the pedestrian candidate generation step S10 Let the candidate objects be filtered.

FIGS. 7 and 8 are views for explaining the ROD calculation step S22 according to an embodiment of the present invention in more detail.

Referring to FIG. 7, in the ROD calculation step of the present invention, the lateral width W of the vehicle connecting the both ends of the sector corresponding to the ROPP is determined (S221), and the outer side region of the left side and the right side outer side region of the ROPP are defined as the vehicle lateral width (W / 2) of W, respectively, so as to obtain ROD (S222).

The red dotted line in FIG. 8 (a) represents the estimated path calculated based on the velocity and the acceleration, and the blue sector represents the ROPP calculated based on the maximum arrival distance and the maximum direction error. The blue region of FIG. 7 (b) represents the ROD which is extended to the left and right by half of the vehicle lateral width on both sides of the set ROPP.

Of course, the degree of expansion of both sides of ROPP may be variously adjusted depending on the detection resolution. For example, in the case of increasing the detection resolution, the degree of expansion of both sides of ROPP can be increased to "W / 2 + α". When the detection resolution is to be lowered, -α ". At this time, alpha may be a positive real number, and it may be preferable to have a value smaller than W / 2.

FIG. 9 and FIG. 10 are views for explaining the pedestrian candidate object selection step S23 according to an embodiment of the present invention in more detail.

Referring to FIG. 9, the step of selecting a candidate for a pedestrian according to the present invention includes a step of projecting an ROD to an input image (S231) by converting a vehicle center-of-gravity coordinate system of the ROD into an input image coordinate system Comparing the positions of the objects, selecting only the pedestrian candidate objects located in the ROD, and transmitting them to the HV step S30 (S232).

Since the ROD is calculated by statistical analysis of the running data, it has a coordinate system different from that of the input image, that is, a vehicle center coordinate system. Therefore, it is necessary to convert the vehicle center coordinate system of the ROD into the input image coordinate system using the projection matrix (P) of the camera.

If a camera projection matrix P is multiplied by a point X (X, Y, Z, 1) on the vehicle center coordinate system as in Equation 6, a point x (u, v, 1) on the image can be obtained, The ROD can be projected onto the input image.

&Quot; (6) "

At this time, the size of P is 3 × 4, and X and x use a homogeneous coordinate system. Since the ROD on the vehicle center coordinate system exists on the ground surface, Z is set to a negative -H of the camera height. The ROD on the image can be calculated by sampling the boundary of the ROD with a certain number of points and calculating the points on the image corresponding to the points.

In FIG. 10 (a), a rough fan-shaped figure drawn with a solid red line represents the ROD on the vehicle center coordinate system, and FIG. 10 (b) represents the ROD projected on the corresponding input image. However, although the ROD is roughly fan-shaped, the ROD projected on the input image may not appear as a sector shape due to perspective transformation.

Since the ROD means an area occupied by the vehicle within a predetermined time, the present invention can determine that the pedestrian is dangerous if any pedestrian exists in the ROD as shown in FIG. 10 (c). In addition, when using the vehicle center coordinate system ROD, it is possible to evaluate whether the pedestrian center coordinate (X, Y) exists in the ROD. In addition, when using the image coordinate system ROD, it is possible to evaluate whether the base center of the pedestrian zone rectangle exists in the ROD.

The red solid line in FIG. 10 (c) shows the ROD. Referring to this, it can be seen that when the vehicle speed is small, the maximum reachable distance is small and the ROD with the wide angle range is calculated. In the case of a pedestrian marked with a red square, it is classified as a dangerous pedestrian because the center of the bottom exists in the ROD, and a pedestrian marked with a blue square is classified as a non-dangerous pedestrian because it exists outside the ROD.

11 is a view for explaining an active pedestrian protection system according to an embodiment of the present invention.

11, the active pedestrian protection system of the present invention includes an input image acquiring unit 10 for acquiring and providing an input image through various image acquisition devices such as a front camera, an electronic control chassis control system, A candidate generating unit 30 for extracting a pedestrian candidate object from the input image, and an ROD (Region of Danger) from the running data to obtain a driving data, A candidate evaluation unit 50 for selecting each pedestrian candidate object based on a CNN (Convolutional Neural Network) or the like and selecting and reporting only a pedestrian, A vehicle control unit 60 that performs various vehicle control operations for preventing collision in consideration of the position of the pedestrian and the vehicle running state, The.

In addition, the risk evaluation unit 40 further includes an ROPP calculating unit 41 for calculating ROPP, which is an area where a future route of the vehicle can exist, through a statistical analysis of the running data of the vehicle, a ROPP calculating unit 41 for calculating ROPP, And a pedestrian candidate selecting unit 43 for selecting only the pedestrian candidate objects that are likely to be out of the vehicle based on the ROD and providing the pedestrian candidate objects to the candidate verifying unit 50. [

That is, considering that the processing load of the candidate verifying unit 50 is much larger than that of the risk evaluating unit 40, the active pedestrian protection system of the present invention is configured such that the processing load of the candidate verifying unit 40, So that the amount of information input to the memory 50 is minimized. As a result, the operation time of the system of the present invention is greatly reduced compared with the conventional system, and the system real time can be obtained.

The features, structures, effects and the like described in the foregoing embodiments are included in at least one embodiment of the present invention and are not necessarily limited to one embodiment. Further, the features, structures, effects, and the like illustrated in the embodiments may be combined or modified in other embodiments by those skilled in the art to which the embodiments belong.

Therefore, it should be understood that the present invention is not limited to these combinations and modifications. While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the present invention. It can be seen that various modifications and applications are possible. For example, each component specifically shown in the embodiments may be modified and implemented. It is to be understood that the present invention may be embodied in many other specific forms without departing from the spirit or essential characteristics thereof.

Claims (8)

A candidate generating step of extracting a pedestrian candidate object from an input image;
A risk assessment step of calculating ROD (Region of Danger) through statistical analysis of the running data and filtering the pedestrian candidate object based on the ROD; And
And a candidate verification step of performing a candidate verification for each of the filtered candidate pedestrian objects to detect a pedestrian in a dangerous state,
The risk assessment step
Calculating a fan-shaped Region of Prediction Path (ROPP) which is an area where a future route of the vehicle can exist through statistical analysis of the travel data;
Expanding the left and right outer regions of the ROPP according to a predetermined criterion to calculate the ROD; And
And selecting and outputting only the pedestrian candidates located in the ROD among the pedestrian candidate objects extracted through the candidate generation step. delete 2. The method of claim 1, wherein calculating the ROPP comprises:
"

"And"

(X (T), y (T)) for the predicted time T based on the estimated path endpoint
Calculating a phase angle? (T) of the expected path end point according to an equation of? (T) = tan ^-1 (y (T) / x (T));
"

Calculating a maximum directional error [Delta] [phi] _max (v);
Defining a direction of the maximum error (ΔΦ _max (v)) on the basis of _{"Φ (T) -ΔΦ max (} v) ~ Φ (T) + ΔΦ max (v)" of the phase angle range of ROPP;
"

"(A _max (v)) < / RTI >
"

(S _max (v)) on the basis of "
Calculating an ROPP defined by a sector whose radius is s _max (v) and whose phase angle is in the range of? (T) -?? _Max (v)
Wherein x and y denote coordinates on a coordinate system having x and y axes on the front and left sides of the vehicle, v is a velocity,? Is an angular velocity, t is a time, T is a predicted time, and b, c, and d respectively represent a function type, a maximum value, and a parameter for determining a minimum value, respectively. 4. The method of claim 3, wherein the step of calculating ROD comprises:
Recognizing the transverse width W of the vehicle by connecting both ends of the sector corresponding to the ROPP; And
And obtaining the ROD by expanding the left and right outer regions of the ROPP by half (W / 2) of the vehicle lateral width (W), respectively. 2. The method of claim 1, wherein selecting and outputting only the pedestrian candidates located in the ROD comprises:
Projecting the ROD onto the input image; And
Comparing the ROD position projected on the input image with the position of the candidate pedestrian object, and selecting and outputting only the pedestrian candidate objects located in the ROD as the object to be verified. 6. The method of claim 5, wherein projecting the ROD to the input image comprises:
"

(U, v, 1) is an input image coordinate system, P (X, Y, Z, 1) is a vehicle center coordinate system, v) is a point on the input image coordinate system, (X, Y, Z) is a point on the vehicle center coordinate system, and P is a projection matrix of the camera. A candidate generator for extracting a pedestrian candidate object from an input image;
A risk assessment unit for calculating the ROD (Region of Danger) through statistical analysis of the running data and filtering the pedestrian candidate object based on the ROD; And
And a candidate verification unit for performing a candidate verification on each of the filtered candidate candidate objects to detect a pedestrian in a dangerous state,
The risk assessment section
An ROPP calculation unit for calculating a fan-shaped ROPP (Region of Prediction Path) which is a region where a future route of the vehicle can exist through statistical analysis of the travel data;
An ROD calculation unit for calculating the ROD by expanding the left and right outer regions of the ROPP according to predetermined criteria; And
And a pedestrian candidate selecting unit for selecting only the pedestrian candidates located in the ROD among the pedestrian candidate objects extracted through the candidate generating unit and transmitting the selected pedestrian candidates to the candidate verifying unit. delete

Images