WO2018190362A1

WO2018190362A1 - Method and device for detecting pedestrian around vehicle

Info

Publication number: WO2018190362A1
Application number: PCT/JP2018/015194
Authority: WO
Inventors: 依若戴; 楊張; 孝一照井; 健志磨
Original assignee: 日立オートモティブシステムズ株式会社
Priority date: 2017-04-12
Filing date: 2018-04-11
Publication date: 2018-10-18
Also published as: JPWO2018190362A1; CN108694363A; JP6756908B2

Abstract

Provided are a method and a device for detecting pedestrians around a vehicle, the method and the device being capable of quickly discriminating, among pedestrians, a plurality of riders from a plurality of pedestrians, and detecting a plurality of riders in real time. A device for detecting pedestrians around a vehicle comprises: an acquisition unit for acquiring image information around the vehicle from an imaging device in the vehicle to detect a current vehicle speed of the vehicle; an initial candidate frame parameter calculation unit for calculating, on the basis of the image information, initial candidate frame parameters of pedestrians around the vehicle, the parameters each including an orientation that is the angle of the pedestrian with respect to the imaging device; an adjustment unit for adjusting the initial candidate frame parameters on the basis of the orientations to acquire adjusted candidate frame parameters; a classification unit for classifying a plurality of pedestrians into riders and pedestrians on the basis of the adjusted candidate frame parameters to acquire rider detection frame parameters of the riders; and a danger coefficient setting unit for setting a danger coefficient for each rider on the basis of the rider detection frame parameter, state information on each rider, and change frequency of the orientation.

Description

Method and apparatus for detecting pedestrians around a vehicle

The present invention relates to a method and apparatus for detecting pedestrians around a vehicle.

The development of automobiles has brought great convenience to human beings, but traffic accidents caused by it have been a major problem for people. In addition to accidents caused by breach of motorists' own rules and bad habits, pedestrian traffic violations are very serious on most roads in China. In particular, riders (including light vehicles such as motorcycles, mopeds, electric bicycles, bicycles, tricycles) that occupy the city occupy the lanes of automobiles, do not comply with traffic signal regulations, Traffic violations such as crossing and running backwards are not uncommon. In China, Article 58 of the Road Traffic Safety Act states that “the maximum speed must be 15 km or less when electric wheelchairs and electric bicycles for the physically challenged are traveling on light vehicle roads.” In practice, the actual speed of an electric bicycle is generally around 40-50 km, far exceeding the maximum speed specified by law. In 2016, China owned about 390 million light vehicles, of which about 220 million were electric bicycles. Because there are so many cars, the hidden dangers of light vehicles on road traffic cannot be ignored.

At the same time, due to the rapid development of sensors and control technology, active safety technology for automobiles to prevent accidents is gaining more and more attention. The standardization of collision avoidance related to automobile safety is also progressing around the world, and since 2018, the European New Car Assessment Program (EuroNCAP) is expected to include the ability to avoid collisions with bicycles in the evaluation of automobiles. Continental Corporation used radar and high-precision global positioning system sensors to achieve synchronized movement of the car and bicycle, monitoring the bicycle, and automatic braking. Volvo, Swedish sporting goods company and Ericsson have jointly developed a two-way communication system between driver and rider that can prevent collisions with pedestrians, and Jaguar Land Rover is also developing bicycle collision warning technology .

US Patent Application No. 9,087,263B2

In the rider collision warning technology, sensor-based rider detection and tracking algorithms play an important role. The following patents, “Detection of pedestrians and bicycle riders based on vision” (“Vision based Pedestrian and Cyclist Detection Method”, US Patent Application No. 9,087,263B2, published on July 21, 2015, Nakayama, Taiwan) (Science Research Institute application) (Patent Document 1) In the on-board camera as a sensor, first a human and a bicycle part are detected, and a rider detection algorithm for determining whether or not a rider is based on the spatial positional relationship of both Is disclosed. However, in Patent Document 1, since it is necessary to detect a circular wheel of a bicycle, it cannot be applied when the bicycle wheel is not visible, and it cannot be determined whether or not a pedestrian is a rider.

Furthermore, the operating environment and conditions of the in-vehicle sensor are very severe, and the collision avoidance time of the vehicle running at high speed is extremely short. Therefore, the in-vehicle collision avoidance system is required to have high real-time characteristics and high processing speed. In an actual traffic situation, there are many light vehicles, motor vehicles, and many stationary objects such as pedestrians and barricades. Therefore, a plurality of riders may be detected. However, when detecting a plurality of riders, an algorithm that is complicated in computation and has a large calculation amount often cannot satisfy real-time characteristics. Therefore, according to the prior art, it becomes difficult to detect a plurality of riders in real time.

Therefore, the present invention provides a method and apparatus for quickly distinguishing a plurality of riders and a plurality of pedestrians among pedestrians and detecting a pedestrian around the vehicle that can detect a plurality of riders in real time.

In order to solve the above-described problem, an apparatus for detecting pedestrians around a vehicle according to the present invention acquires image information around the vehicle from an imaging device for the vehicle, and an acquisition unit that detects the current vehicle speed of the vehicle; Based on the image information, an initial candidate frame parameter calculation unit that calculates an initial candidate frame parameter including an orientation that is an angle of the pedestrian with respect to the imaging device at each time point of the plurality of pedestrians around the vehicle; An adjustment unit that adjusts the initial candidate frame parameters based on the orientation to obtain the adjusted candidate frame parameters at each time point, and a plurality of pedestrians based on the adjusted candidate frame parameters. Riders at each point in time of each of the one or more riders, classified into one or more riders and one or more pedestrians Based on the classification unit for obtaining the detection frame parameter, the rider detection frame parameter, the state information of each of the one or more riders, and the change frequency of the direction, each rider of the one or more riders And a risk factor setting unit for setting a risk factor.

The initial candidate frame parameter further includes a frame centroid coordinate, a frame width w, and a frame height h, and the adjusted candidate frame parameter includes the frame centroid coordinate, the adjusted frame width w ′, and the frame height. h, and the orientation α, where

It is.
The classification unit sets the adjusted candidate frame parameter at each time point of each of the one or more riders as the rider detection frame parameter.

In this way, riders can be classified from pedestrians according to the adjusted candidate frame parameters.

The classification unit uses the initial candidate frame parameters of each of the one or more pedestrians as pedestrian detection frame parameters at the respective time points of the one or more pedestrians,
The risk factor setting unit is configured such that the pedestrian detection frame parameter, the state information of each of the one or more pedestrians, and the change frequency of the direction, each pedestrian among the one or more pedestrians. Set the risk factor to.

For each rider of the one or more riders, the risk factor setting unit is based on the frame center of gravity coordinates of the rider detection frame parameter at the previous time point and the frame center of gravity coordinates of the rider detection frame parameter at the current time point. The relative speed and the relative movement direction of each rider with respect to the vehicle are acquired.

The risk factor setting unit determines whether or not each rider is a new rider. If the rider is a new rider, the risk factor is set to 1. If the rider is not a new rider, the current rider detection frame is set. The risk factor is set based on the frame center-of-gravity coordinates and the direction of the parameter, the relative speed, the relative movement direction, the state information, and the change frequency of the direction.

The state information is a normal state or an abnormal state, and the risk factor setting unit determines that the rider is a child based on the frame height of the rider detection frame parameter, or from the image information When it is determined that the distance between the rider and another rider is smaller than a predetermined safe distance, or when it is determined from the image information that the rider is receiving interference from an external object, the state information is determined as an abnormal state.

According to another aspect of the present invention, there is provided a method for detecting a pedestrian around a vehicle, the first step of acquiring image information around the vehicle from a photographing device of the vehicle and detecting a current vehicle speed of the vehicle, and the image information. A second step of calculating an initial candidate frame parameter including an orientation that is an angle of the pedestrian with respect to the imaging device at each time point of the plurality of pedestrians around the vehicle, and based on the orientation, A third step of adjusting the initial candidate frame parameters to obtain adjusted candidate frame parameters at each time point, and one or a plurality of the pedestrians based on the adjusted candidate frame parameters A rider detection frame parameter at each time point for each of the one or more riders. A risk factor is set for each rider among the one or more riders based on the step, the rider detection frame parameter, the state information of each of the one or more riders, and the direction change frequency; These steps are included.

ADVANTAGE OF THE INVENTION According to this invention, it is possible to provide the method and apparatus which detect the pedestrian around a vehicle which can distinguish a some rider and several pedestrian among pedestrians rapidly, and can detect several riders in real time. It becomes.
Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

1 is a structural diagram of an apparatus for detecting pedestrians around a vehicle according to an embodiment of the present invention. 3 is a flowchart of a method for detecting pedestrians around a vehicle according to an embodiment of the present invention. It is a schematic diagram of an initial candidate frame at time t1 of one pedestrian among a plurality of pedestrians according to an embodiment of the present invention. It is a schematic diagram of the candidate frame after adjustment of one pedestrian shown in FIG. It is a schematic diagram of the initial candidate frame at the time t1 of another pedestrian among a plurality of pedestrians according to an embodiment of the present invention. It is a schematic diagram of the candidate frame after adjustment of the other pedestrian shown by Fig.4 (a). It is a schematic diagram which shows the definition of direction.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a structural diagram of an apparatus 10 for detecting pedestrians around a vehicle according to an embodiment of the present invention. As shown in FIG. 1, the device 10 for detecting pedestrians around a vehicle includes an acquisition unit 11, an initial candidate frame parameter calculation unit 12, an adjustment unit 13, a classification unit 14, and a risk coefficient setting unit 15.

FIG. 2 is a flowchart of a method for detecting pedestrians around a vehicle according to an embodiment of the present invention. As shown in FIG. 2, in step S <b> 21, the acquisition unit 11 acquires image information around the vehicle from the vehicle imaging device and detects the current vehicle speed of the vehicle. The photographing device (not shown) is one or a plurality of camera systems attached to appropriate positions of the vehicle (the upper end of the vehicle windshield, the rear end of the vehicle tail, or both sides of the vehicle body). Collect and store image information from the rear and both sides. In general, a camera system includes an optical system and a camera. The optical system may have a zoom function, an autofocus function, and the like, and a color CCD (charge coupled device) video camera may be used as the camera.

In step S22, the initial candidate frame parameter calculation unit 12 determines the angle of the pedestrian shooting device at each time point of the plurality of pedestrians around the vehicle based on the image information, that is, the pedestrian torso shooting device. An initial candidate frame parameter including the direction α which is an angle is calculated.

FIG. 5 is a schematic diagram showing the definition of the direction α. When the orientation of the pedestrian with respect to the imaging device is right-handed, α is 0, and when the orientation of the pedestrian with respect to the imaging device is front-facing, α is π / 2, and the orientation of the pedestrian with respect to the imaging device is Α is π or −π when the pedestrian is facing left, and α is −π / 2 when the pedestrian is facing the imaging device. Other directions are output according to the actual size and normalized to [−π, π]. Similarly, as shown in FIG. 5, the direction α divided into a certain region may be divided into the same angle of one class. As shown in FIG. 5, when the pedestrian's orientation with respect to the imaging device is right forward, it is classified into class I, α is π / 4, and when the pedestrian's orientation with respect to the imaging device is true forward, Class II, α is π / 2, and when the direction of the pedestrian with respect to the imaging device is left front facing, class III, α is π × 3/4, and pedestrian imaging If the orientation to the device is true left, classify as class IV, α is π or -π, and if the orientation of the pedestrian to the imaging device is left rearward, class A is classified as class V and α is- When π × 3/4 and the orientation of the pedestrian with respect to the imaging device is true rearward, it is classified into class VI, α is −π / 2, and the orientation of the pedestrian with respect to the imaging device is right rearward. In the case of class VII, α is −π / 4, and pedestrian shooting When the orientation with respect to the device is right-facing, it is classified into class VIII and α is 0.

The initial candidate frame parameters further include frame centroid coordinates, frame width w, and frame height h. FIG. 3A is a schematic diagram of the initial candidate frame F at the time point t1 of one pedestrian among a plurality of pedestrians according to an embodiment of the present invention. The parameters include frame centroid coordinates (x _t1 , y _t1 , z _t1 ), frame width w _t1 , frame height h _t1 , and orientation α _t1 (not shown). Here, the frame center-of-gravity coordinates are set as the world coordinate system of the vehicle. For example, the coordinate origin is a position directly below the middle of the vehicle head.

Here, for example, a feature extraction method (dense feature: DPM deformed part model, ACF total channel feature, HOG direction gradient histogram, dense edge, etc .; sparse feature: size, shape, sparse edge, body part, walking , Texture, gray scale / edge symmetry, etc.), contour template matching, and the like. First, an initial candidate frame including frame centroid coordinates, frame width, and frame height is calculated. For example, a deep learning algorithm of machine learning. Next, the orientation of the pedestrian is analyzed by a deep learning algorithm or a normal machine learning algorithm. For example, different models based on dedicated orientation detection networks or different directional gradient histograms can be used to divide the input pedestrian initial candidate frames into different orientations. The type of method is called a cascade model.

Further, for example, the frame center-of-gravity coordinates, the frame width, the frame height, and the direction can be collectively calculated by the same deep learning network or normal machine learning, that is, a single model algorithm. In the case of a deep learning network, the pedestrian orientation is trained and detected as part of the regression loss function in all connected layers at the end of the deep learning network.

As can be seen from the above, in step S22 of FIG. 2, the initial candidate frame parameter calculation unit 12 can identify all pedestrians from the image information by the above calculation. In this embodiment, the pedestrian includes a rider and / or a pedestrian.

2, the adjustment unit 13 adjusts the initial candidate frame parameter based on the orientation, and acquires the adjusted candidate frame parameter at each time point.

FIG.3 (b) is a schematic diagram of the candidate frame after adjustment of one pedestrian shown by Fig.3 (a). As shown in FIG. 3B, the adjustment unit 13 adjusts the initial candidate frame parameter of the initial candidate frame F at the time point t1 of the pedestrian to adjust the candidate frame after the adjustment at the time point t1 of the pedestrian. F 'is acquired. For example, the candidate frame parameters of the adjusted candidate frame F ′ are the frame centroid coordinates (x _t1 , y _t1 , z _t1 ), the frame width w ′ _t1 , the frame height h _t1 , and the orientation α _t1 (not shown). Including. here,

It is. That is, the adjustment unit 13 changes the frame width w _t1 of the initial candidate frame F based on the frame height h _t1 and the orientation α _t1 to obtain the adjusted frame width w ′ _t1 of the candidate frame F ′. Keep other parameters constant.
In this way, the candidate frame parameters of the adjusted candidate frame at the time t1 of each pedestrian among the plurality of pedestrians can be acquired.

When the orientation of the pedestrian with respect to the imaging device is forward or backward, the size of the initial candidate frame is not changed, that is, the adjusted candidate frame parameter and the initial candidate frame parameter are the same. When the orientation of the pedestrian with respect to the imaging device is leftward or rightward, the frame height of the initial candidate frame is kept constant, and the frame width is adjusted to be equal to the frame height. When the orientation of the pedestrian with respect to the imaging device is left rearward, right rearward, left frontward, or right frontward, the frame height of the initial candidate frame is held constant and the frame width is adjusted to half the frame height. The change (adjustment) is based on the premise that the frame center-of-gravity coordinates are held constant. As shown in FIG. 3B, once the frame width changes, the coordinates of the four vertices of the adjusted candidate frame also change correspondingly.

In step S24, the classification unit 14 classifies the plurality of pedestrians into one or a plurality of riders and one or a plurality of pedestrians based on the adjusted candidate frame parameters, and each of the one or a plurality of riders. Obtain rider detection frame parameters at each time point. Here, the classification unit 14 calculates the candidate frame parameters after adjustment of each pedestrian by, for example, a deep learning classification algorithm, and classifies based on the calculation result. Since the specific calculation and classification method is the same as the conventional method, a redundant description will not be given.

FIG. 4A is a schematic diagram of an initial candidate frame G at time t1 of another pedestrian among a plurality of pedestrians according to the embodiment of the present invention, and FIG. 4B is a schematic diagram of FIG. It is a schematic diagram of candidate frame G 'after adjustment of the other pedestrians shown in FIG. In step S24, the classification unit 14 classifies, for example, one pedestrian in FIG. 3B as a rider (hereinafter referred to as “rider A”) and walks the other pedestrians in FIG. 4B. (Hereinafter, the other pedestrians in FIGS. 4A and 4B are classified as “pedestrian B”).

The classification unit 14 uses the adjusted candidate frame parameter at each time point of each of one or more riders as the rider detection frame parameter. In this example, for example, the classification unit 14 uses the candidate frame parameter of the adjusted candidate frame F ′ at time t1 shown in FIG. That is, the rider detection frame parameters include frame barycentric coordinates (x _t1 , y _t1 , z _t1 ), frame width w ′ _t1 , frame height h _t1 , and orientation α _t1 (not shown).

Furthermore, the classification unit 14 sets the initial candidate frame parameter of each of one or more pedestrians as the pedestrian detection frame parameter at each time point of one or more pedestrians. In this example, for example, the classification unit 14 uses the initial candidate frame parameter of the initial candidate frame G at time t1 shown in FIG. 4A as the pedestrian detection frame parameter of the pedestrian.

Here, as shown in FIGS. 4A and 4B, in the case of pedestrian B, the pedestrian detection frame parameter and the initial candidate frame parameter are the same, that is, the frame width of the pedestrian detection frame is adjusted. It is smaller than the frame width of the subsequent candidate frame, thereby narrowing the detection range for pedestrians.

Next, in step S25, the risk factor setting unit 15 determines the rider detection frame parameter, the status information of each of one or more riders, and the change frequency of the direction, and the respective rider among the one or more riders. Set the risk factor to.
Here, for each rider among one or more riders, the risk factor setting unit 15 is based on the frame center of gravity coordinates of the rider detection frame parameter at the previous time point and the frame center of gravity coordinates of the rider detection frame parameter at the current time point. The relative speed and the relative movement direction of each rider with respect to the vehicle are acquired.

For example, taking the rider in FIG. 3B as an example, the previous time is t1, for example, and the current time is t2, for example. In the case of the rider A, the frame center-of-gravity coordinates of the rider detection frame parameter at _t1 are, for example, (x _t1 , y _t1 , z _t1 ), and the frame center of gravity coordinates of the rider detection frame parameter at _t2 are, for example, (x _t2 , y _t2 , Z _t2 ), and the relative speed v and the relative movement direction r of the rider A with respect to the vehicle are acquired from the amount of change in (x _t2 , y _t2 , z _t2 ) and (x _t1 , y _t1 , z _t1 ). The relative speed and relative movement direction of each of the other riders can be obtained by a similar method.

The risk factor setting unit 15 determines whether or not each rider is a new rider. If the rider is a new rider, the risk factor setting unit 15 sets the risk factor to 1. If the rider is not a new rider, the current risk detection frame parameter is set. A risk coefficient is set based on the frame center-of-gravity coordinates and direction, relative speed, relative movement direction, state information, and direction change frequency.

For example, taking the rider A in FIG. 3B as an example, the risk coefficient setting unit 15 first determines whether or not it is a new rider. Specifically, it is determined whether or not the rider detection frame parameter exists at the previous time point t1 for the rider A, and if it is determined that the rider detection frame parameter does not exist, the rider A is a new rider, that is, a rider newly appearing at the current time t2. The risk factor W is set to 1 and the next rider is determined. If it is determined that it exists, it is determined that the rider is not a new rider, the risk factor W is initialized to 0, and the frame center of gravity coordinates and direction of the rider detection frame parameter at the current time t2, relative speed, relative movement direction, and state information The risk factor is set based on the direction change frequency.

In this example, for example, when it is determined that the rider A is not a new rider, the risk factor W is initialized to 0, and the risk factor is set for the rider A based on the following condition 1 to condition 5. Hereinafter, Condition 1 to Condition 5 will be described in detail.

Condition 1: It is determined whether the alarm safety distance d _w less than or not the frame center coordinates of the rider detection frame parameters at the current time t2 is calculated from the following equation (1).

In the formula (1), v is the relative speed of the rider A, t _f is the braking time required for the vehicle, t _d is the response time of the vehicle driver, μ is the coefficient of friction of the road where the vehicle is located, and g is the acceleration of gravity. , Ν _h is the current vehicle speed detected by the acquisition unit 11 in step S21.

Here, for example, as shown in the following equation (2), the value of the friction coefficient μ of the road can be determined based on the photographed image information. *

As described above, the coordinate origin is a position directly below the middle of the vehicle head, and in this case, the distance between the frame center of gravity coordinates and the coordinate origin is the distance between the rider and the middle position of the vehicle head. Rider frame center coordinates of the rider detection frame parameters at the moment t2 of A is a _(x _t2, y _{t2, z t2),} thereby obtaining the distance D between the intermediate position of the rider A and headway. Compared with alarm safety distance d _w calculated as described above the distance D, the distance D determines whether the alarm safety distance d _w smaller.

Condition 2: From the relative movement direction of rider A, it is determined whether rider A is approaching the vehicle. Here, for example, whether or not the rider A is approaching the vehicle is determined based on the calculated relative movement direction r.

Condition 3: Whether or not the state information of rider A is in an abnormal state.
The state information is a normal state or an abnormal state, and the risk coefficient setting unit 15 determines that the rider is a child based on the frame height of the rider detection frame parameter, or the rider and other riders from the image information. When it is determined that the distance is smaller than the predetermined safe distance, or when it is determined from the image information that the rider is receiving interference from an external object, the state information is determined as an abnormal state.

For example, if the frame height of the rider detection frame parameter is h _t2 at present t2 rider A, since the frame height is h _t2, the rider A is equal to or a child, children If it is, it is determined that the state is abnormal.

For example, when the present time t2 example, riders A and other riders, such as the distance between the rider C is the distance D _AC the frame centroid coordinates and rider C of the frame barycentric coordinates of the rider A. The predetermined safety distance d _s is obtained from the following equation (3), for example.

In equation (3), k is the relative speed between rider A and rider C, t ′ _f is the time required for the rider's brake, t _d ′ is the rider's response time, μ is the coefficient of friction of the road where rider A is located ( For example, obtained from the above equation (2)), g is the gravitational acceleration. Here, for example, k can be acquired from the amount of change between the frame center of gravity coordinates of the rider A and the frame center of gravity coordinates of the rider C.

Then, the distance _{D AC} with a predetermined safety distance _{d s,} the distance _{D AC} determines whether a predetermined safety distance _{d s} is less than. If it is smaller than the predetermined safety distance d _s , it is determined that the state is abnormal, and if it is equal to or longer than the predetermined safety distance d _s , it is determined that the state is normal.

Further, it is determined from the image information whether or not the rider A is interfered by an external object, for example, whether or not an unknown object appears within a predetermined angle range starting from the direction α of the rider A. If it does appear, it is determined that it is in an abnormal state, and if it does not appear, it is determined that it is in a normal state.

If all the above three determination results are NO, it is determined that the rider is in a normal state.

Condition 4: It is determined whether or not the rider A can see the vehicle based on the direction α of the rider A at the current time t2.
Here, for example, the direction α of the rider is any one of true rearward, left rearward, right rearward, true leftward, and rightward, that is, the direction α is the class VI, class V, class VII, class shown in FIG. If it belongs to either IV or class VIII, it is determined that the vehicle cannot be seen by the rider, and otherwise, it is determined that the vehicle is visible.

Condition 5: Whether the direction change frequency is greater than a predetermined threshold.

Here, for example, in the case of the rider A, 10 orientations α at 10 consecutive time points before the current time t2 are extracted, and the number of changes in these orientations, that is, the direction change frequency is calculated. . The predetermined threshold value is obtained based on, for example, statistical data obtained by a large number of riders. Next, it is determined whether or not the direction change frequency is greater than a predetermined threshold.

Hereinafter, taking rider A as an example, the setting of the risk factor for rider A based on condition 1 to condition 5 will be described in detail.

First, if the determination result of condition 1 or 2 is yes, that is, if the frame center-of-gravity coordinates of rider A at the current time t2 is smaller than the warning safety distance, or if rider A is approaching the vehicle, it is initialized. If 1 is added to the risk factor 0, that is, W = 0 + 1 = 1, whereas if both the judgment results of the conditions 1 and 2 are no, the risk factor is still the initialized risk factor That is, W = 0. In this example, for example, the determination result of condition 1 is yes and W = 1.

Next, if the determination result of condition 3 is yes, that is, if the current state is abnormal, 1 is added to the risk factor determined based on conditions 1 and 2, and if no, the condition 1 The risk factor determined based on condition 2 is maintained. In this example, for example, the determination result of condition 3 is yes, and W = 1 + 1 = 2.

Finally, when the determination result of condition 4 or 5 is yes, that is, when it is determined that the vehicle is visible to the rider or the change frequency of the direction is greater than a predetermined threshold, the risk coefficient determined based on condition 3 is further increased. 1 is added. On the other hand, when both the determination results of the condition 4 and the condition 5 are no, the risk coefficient determined based on the condition 3 is held. For example, when both the determination results of the condition 4 and the condition 5 are no, the risk factor determined based on the condition 3 is held, that is, W = 2.

In this way, each rider's risk factor at each point in time can be obtained, each rider can be tracked according to the risk factor, and an alarm can be issued to the vehicle if necessary.

Furthermore, the risk coefficient setting unit 15 further includes a pedestrian detection frame parameter, each state information of one or more pedestrians, and a change frequency of the direction, and each pedestrian among one or more pedestrians. Set the risk factor to. For example, the risk factor can be set for the pedestrian B by the above-described risk factor setting method for the rider.

As described above, according to the present embodiment, a plurality of riders and pedestrians among pedestrians can be quickly distinguished, and a plurality of riders can be detected in real time.

Although the present invention has been described with reference to the embodiments, it is obvious that those skilled in the art can make various substitutions, changes and modifications to the above contents. Accordingly, such substitutions, changes and modifications should belong to the present invention without departing from the spirit and scope of the appended claims.

Further, the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described.

DESCRIPTION OF SYMBOLS 10 ... Apparatus which detects the pedestrian around a vehicle, 11 ... Acquisition unit, 12 ... Initial candidate frame parameter calculation unit, 13 ... Adjustment unit, 14 ... Classification unit, 15 ... Risk factor setting unit

Claims

A device for detecting pedestrians around a vehicle,
An acquisition unit that acquires image information around the vehicle from a vehicle imaging device and detects a current vehicle speed of the vehicle;
Based on the image information, an initial candidate frame parameter calculation unit that calculates an initial candidate frame parameter including an orientation that is an angle of the pedestrian with respect to the imaging device at each time point of the plurality of pedestrians around the vehicle;
An adjustment unit that adjusts the initial candidate frame parameter based on the orientation to obtain the adjusted candidate frame parameter at each time point;
Based on the adjusted candidate frame parameters, the plurality of pedestrians are classified into one or more riders and one or more pedestrians, and the rider detection at each time point of the one or more riders. A classification unit for obtaining frame parameters;
A risk coefficient setting unit that sets a risk coefficient for each of the one or more riders based on the rider detection frame parameter, the state information of each of the one or more riders, and the change frequency of the direction. And a device for detecting a pedestrian around the vehicle.
The initial candidate frame parameters further include frame centroid coordinates, frame width w, and frame height h,
The adjusted candidate frame parameters include the frame centroid coordinates, the adjusted frame width w ′, the frame height h, and the orientation α, where

And
The walking around the vehicle according to claim 1, wherein the classification unit uses the adjusted candidate frame parameter at each time point of each of the one or more riders as the rider detection frame parameter. A device that detects a person.
The classification unit uses the initial candidate frame parameters of each of the one or more pedestrians as pedestrian detection frame parameters at the respective time points of the one or more pedestrians,
The risk factor setting unit is configured to determine whether the pedestrian detection frame parameter, the state information of each of the one or more pedestrians, and the change frequency of the direction of each of the one or more pedestrians. The apparatus for detecting a pedestrian around a vehicle according to claim 1, wherein a risk factor is set for the person.
For each rider of the one or more riders, the risk factor setting unit is based on the frame center of gravity coordinates of the rider detection frame parameter at the previous time point and the frame center of gravity coordinates of the rider detection frame parameter at the current time point. The apparatus for detecting a pedestrian around a vehicle according to claim 2, wherein a relative speed and a relative movement direction of each rider with respect to the vehicle are acquired.
The risk factor setting unit determines whether or not each rider is a new rider. If the rider is a new rider, the risk factor is set to 1. If the rider is not a new rider, the current rider detection frame is set. 5. The vehicle periphery according to claim 4, wherein the risk coefficient is set based on the frame center-of-gravity coordinates and the direction of the parameter, the relative speed, the relative movement direction, the state information, and the change frequency of the direction. Detecting pedestrians.
The state information is a normal state or an abnormal state,
The risk factor setting unit determines that the rider is a child based on the frame height of the rider detection frame parameter, or the distance between the rider and another rider is a predetermined safety distance based on the image information. The vehicle surroundings according to claim 5, wherein the state information is determined as an abnormal state when it is determined that the vehicle is smaller than the image information or when it is determined that the rider receives interference from an external object. Detecting pedestrians.
A method for detecting pedestrians around a vehicle,
A first step of acquiring image information around the vehicle from a vehicle photographing device and detecting a current vehicle speed of the vehicle;
A second step of calculating an initial candidate frame parameter including an orientation that is an angle of the pedestrian with respect to the photographing device at each time point of the plurality of pedestrians around the vehicle based on the image information;
A third step of adjusting the initial candidate frame parameters based on the orientation to obtain adjusted candidate frame parameters at each time point;
Based on the adjusted candidate frame parameters, the plurality of pedestrians are classified into one or more riders and one or more pedestrians, and the rider detection at each time point of the one or more riders. A fourth step of obtaining frame parameters;
A fifth step of setting a risk factor for each of the one or more riders based on the rider detection frame parameter, the state information of each of the one or more riders, and the direction change frequency; The method of detecting the pedestrian around the vehicle characterized by including these.
The initial candidate frame parameters further include frame centroid coordinates, frame width w, frame height h,
In the third step, the adjusted candidate frame parameters include the frame centroid coordinates, the adjusted frame width w ′, the frame height h, and the orientation α, where

And
The vehicle surroundings according to claim 7, wherein, in the fourth step, the adjusted candidate frame parameter at each time point of each of the one or more riders is set as the rider detection frame parameter. To detect pedestrians in the city.
In the fourth step, the initial candidate frame parameter of each of the one or more pedestrians is set as a pedestrian detection frame parameter at each time point of the one or more pedestrians,
In the fifth step, based on the pedestrian detection frame parameter, the state information of each of the one or more pedestrians, and the change frequency of the direction, each of the one or more pedestrians walking 8. The method for detecting a pedestrian around a vehicle according to claim 7, wherein a risk factor is set for the person.
Based on the frame center of gravity coordinates of the rider detection frame parameter at the previous time point and the frame center of gravity coordinates of the rider detection frame parameter at the current time point for each of the one or more riders, the vehicle of each rider The method for detecting a pedestrian around a vehicle according to claim 8, wherein a relative speed and a relative movement direction with respect to the vehicle are acquired.
In the fifth step, it is determined whether or not each rider is a new rider. If the rider is a new rider, the risk factor is set to 1. If the rider is not a new rider, the current rider detection frame is set. The vehicle surroundings according to claim 10, wherein the risk factor is set based on the frame center-of-gravity coordinates and the direction of the parameter, the relative speed, the relative movement direction, the state information, and the change frequency of the direction. To detect pedestrians in the city.
The state information is a normal state or an abnormal state,
Based on the frame height of the rider detection frame parameter, it is determined that the rider is a child, or it is determined from the image information that the distance between the rider and another rider is smaller than a predetermined safe distance, or 12. The method for detecting pedestrians around a vehicle according to claim 11, wherein when it is determined from the image information that the rider is receiving interference from an external object, the state information is determined as an abnormal state.