KR102160511B1 - Pedestrian recognition method for pedestrian waiting area and pedestrian recognition apparatus using the same - Google Patents

Abstract

The present invention relates to a method and an apparatus for visually detecting a pedestrian through a camera and determining the pedestrian′s situation. According to the present invention, the method for recognizing a pedestrian in a crosswalk waiting area comprises: a detection area setting area (S10) of setting an immediate area (IA) along a border between a sidewalk and a road where there is a crosswalk based on an image obtained from a camera (10), and setting a rectangular waiting area (WA) adjacent to the immediate area (WA); a pedestrian position determination step (S20) of determining a position of a pedestrian in the immediate area (IA) or the waiting area (WA); and a pedestrian pattern recognition step (S30) of identifying a probability that the pedestrian waits in the waiting area (WA) to cross at the crosswalk.

Description

Pedestrian Recognition Method and Device in Pedestrian Crossing Waiting Area {PEDESTRIAN RECOGNITION METHOD FOR PEDESTRIAN WAITING AREA AND PEDESTRIAN RECOGNITION APPARATUS USING THE SAME}

The present invention relates to a method and apparatus for recognizing a pedestrian situation in a pedestrian crossing waiting area in order to transmit a pedestrian guide signal or warning, and more particularly, a method for detecting a pedestrian by using a camera and determining the situation of a pedestrian, and It relates to the device.

As the performance of cameras and sensors is improved, and artificial intelligence technologies such as deep learning are developed, the development of devices for the safety of pedestrians on the road is becoming active. Briefly, a variety of pedestrian safety devices have appeared, ranging from a device that displays a walking guide signal by determining the presence or absence of a pedestrian on a crosswalk, to a device that predicts the number of pedestrians or the intention to walk and issues an appropriate guidance signal or alarm.

An example of such a pedestrian guide device is shown in'Patent Document 1'. The conventional pedestrian guidance device disclosed in'Patent Document 1'consists of a pedestrian detection device, a control device, a camera, and a vehicle detection device, and transmits light to the waiting area for pedestrians by using the first and second detection sensors of the pedestrian monitoring device. Pedestrians are detected by whether or not the reflected light is received from the pedestrian crossing, and when there are pedestrians entering the crosswalk through the crosswalk waiting area and pedestrians entering from the outside of the crosswalk waiting area while the pedestrian traffic light is red, unauthorized crossing is prohibited. Provide guidance.

However, since such a conventional pedestrian guidance device detects a pedestrian with a sensor such as light, ultrasonic waves, or infrared rays, it is easy to know whether a pedestrian is present, but it is difficult to know the exact movement pattern of the pedestrian. In addition, since the warning is issued by determining whether there are pedestrians in the waiting area for crossing, even if there are only pedestrians who do not intend to cross the street, if the pedestrian traffic light is red, a crossing prohibition guide is issued, causing inconvenience to pedestrians due to noise. There is a possibility of complaints occurring in nearby shopping malls.

KR 10-1904929 B1 (September 28, 2018)

The present invention was conceived to solve the above problems, and the problem to be solved by the present invention is that it is possible to extract an accurate movement pattern of a pedestrian, and by grasping the walking intention of a pedestrian, unnecessary warning announcements are generated near the pedestrian crossing. It is to provide a method and apparatus for recognizing pedestrians in a waiting area for a pedestrian crossing that can minimize

In the method for recognizing pedestrians in a pedestrian crossing waiting area according to the present invention, an instantaneous area is set long along the boundary of a roadway and a sidewalk with a crosswalk based on an image acquired from a camera, and a rectangular waiting area in contact with the immediate area is set. A detecting area setting step of setting; It is characterized in that it comprises a pedestrian position determination step of determining the position of a pedestrian in the immediate area or the waiting area, and a pedestrian pattern recognition step of grasping the possibility of a pedestrian waiting to cross the pedestrian crossing in the waiting area. .

The apparatus for recognizing pedestrians in the waiting area for a pedestrian crossing according to the present invention is a technical feature that includes a camera and a computing device for performing the method for recognizing pedestrians in the waiting area for a pedestrian crossing according to the present invention.

The pedestrian recognition method of the pedestrian crossing waiting area according to the present invention can prevent unnecessary warnings or announcements from being issued by distinguishing the immediate warning and the waiting announcement according to the pedestrian being recognized in the immediate area and the waiting area. have.

1 is a flowchart of a method for recognizing a pedestrian in a waiting area for a pedestrian crossing according to the present invention
2 is a vision recognition area set in the present invention
3 is a result of measuring the NW of a person waiting to walk while changing τ using a traffic control camera
In FIG. 4, a waiting area including a curved section instantaneous area and a sector area, and an arrangement of a camera for defining the same
5 is a configuration diagram of a pedestrian recognition apparatus in a waiting area for a pedestrian crossing according to the present invention

Hereinafter, a method and apparatus for recognizing a pedestrian in a waiting area for a pedestrian crossing according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart of a method for recognizing a pedestrian in a waiting area for a pedestrian crossing according to the present invention, and FIG. 2 is a vision recognition area set in the present invention. The method for recognizing a pedestrian in a waiting area for a pedestrian crossing according to the present invention includes the step of setting a detection area ( S10), a pedestrian position determining step (S20) and a pedestrian pattern recognition step (S30).

The detection area setting step (S10) is a step of setting an area to acquire an image with the camera 10 around the crosswalk. In the present invention, two detection areas are set. The first detection area is an area in which an alert or guidance should be issued immediately upon entering a pedestrian (hereinafter referred to as an “immediate area (IA)”), and is set long along the border of a roadway and sidewalk with a crosswalk. For example, a road boundary stone may be set as the immediate area IA as it is, and an area extending a predetermined width of 10 cm to 1 m to both sides of the road boundary may be set as the immediate area IA. The length of the immediate area IA may be the length of the area in which the crosswalk is drawn. The determination of the immediate area IA can be obtained by learning the image acquired by the camera 10 as a road edge image, and this method is applicable to the second detection area to be described below.

Immediately, an immediate warning or guidance is required because the immediate area (IA) is an area where pedestrians can immediately invade the roadway. However, even in the immediate area IA, since there is no need to warn pedestrians crossing from the roadway to the sidewalk, it is necessary to determine whether to issue the warning by calculating the position change of the pedestrian.

The second detection area is an area in which pedestrians are normally waiting for crossing (hereinafter referred to as'waiting area WA'), and the waiting area WA can be defined as a rectangular area in contact with the immediate area IA. have. For example, the horizontal length of the waiting area WA may be the same as the length of the immediate area IA, and the vertical length may be the width of the sidewalk. And staying in the waiting area WA for more than a certain period of time means that the pedestrian is willing to cross the crosswalk, and thus, a waiting announcement or an announcement indicating that the crossing should be crossed when the signal is switched can be performed.

The pedestrian positioning step S20 is a step of determining the position of a pedestrian in the immediate area IA or the waiting area WA. The position of the pedestrian can be obtained from the image captured by the camera 10. In the present invention, the position of the pedestrian is determined by recognizing the pedestrian's shoe. In the case of shoes, pedestrians or pedestrians rarely take off, and the shape is somewhat constant, so it is easy to judge by learning. In the case of recognizing a shoe, it becomes a question which point of the shoe is to be determined as the pedestrian's position, but ideally, the center of gravity of the shoe can be used. However, in the present invention, since the position change is more important than accurately determining the position of the pedestrian, even if the nearest point from the camera 10 is determined as the pedestrian position, there is no big problem. Since the camera 10 looks at the ground in a diagonal direction, the position of the pedestrian can be obtained by matching the coordinates of a specific point of the shoe in the image acquired by the camera 10 in the X-Y plane coordinate system. This is possible because when the line of sight of the camera 10, that is, an azimuth and an elevation angle, is determined, the coordinates of the corresponding image can be made one-to-one correspondence with the X-Y plane coordinate system.

However, even if a pedestrian is detected in the waiting area WA, there are cases in which it is necessary to determine that the person is simply passing by the waiting area WA or is a person who stays for a while due to a phone call or the like, and therefore does not intend to cross the crosswalk. Accordingly, in the present invention, a pedestrian pattern recognition step (S30) is performed to determine the possibility of a pedestrian waiting to cross a pedestrian crossing in the waiting area WA, and it is determined whether or not a waiting announcement or the like should be performed according to the value.

For intuitive understanding, we will assume several situations. First, if a pedestrian simply passes through the waiting area WA, the pedestrian's speed vector will not change much, and when the pedestrian is moving to wait in the waiting area WA, the size of the speed vector decreases or the direction is more frequent, It will change a lot. And if it is in full standby state, the velocity vector is zero. Therefore, a pedestrian's velocity vector V expressed in the XY coordinate system can be used as an index representing the pedestrian's movement. Since the camera image is obtained in units of frames, the velocity vector V(t) at a specific time t can be defined as shown in [Equation 1] below. However, t _f is the inter-frame time, and S(t) is the position vector of the pedestrian at the time t, that is, the pedestrian position determined in the pedestrian position determining step S20.

The autocorrelation function of V is used to determine the degree to which the pedestrian's velocity vector V is maintained. When observing from time 0 to T with respect to the time interval τ, V is a vector amount, so the greater the [Equation 2] below, the lower the pedestrian's waiting degree (hereinafter referred to as'atmospheric index W'), and the closer to 0, the longer the waiting degree. Is high. However, from t = 0 to t = T -Adding in the τ interval means t = 0, t _f , 2t _f , 3t _f ,... , T-τ, which means that the frames obtained at intervals of time t _f are added discretely.

However, the waiting degree is calculated for pedestrians with a large speed vector such as adults, and the waiting degree is calculated for pedestrians with a small speed vector such as children and the elderly. It is difficult to see as representing. Therefore, in order to eliminate the effect of the velocity vector, it is necessary to go through some kind of normalization, and the value obtained by dividing the right side of [Equation 2] by the sum of squares of the velocity vector is defined as the normalized atmospheric index NW, and based on this It is more desirable to judge the degree to which pedestrians are waiting.

In the above [Equation 2] and [Equation 3], τ refers to the time interval in observing the autocorrelation. If τ is large, the change in the velocity vector V is large, and when τ is small, the change in the velocity vector V is small. Depending on τ, the atmospheric index W or the normalized atmospheric index NW is affected.

3 shows a result of measuring NW for 5 seconds of a person waiting for a walk while changing τ in a section from 1t _f to 30t _f using a traffic control camera. Currently, most common traffic control cameras are 30fps (t _f = 0.033sec), and the optimal range of τ experimentally derived by the present inventors in this environment is 8t _f to 18t _f . The reason for this judgment is that when τ is close to 0, the walking waiter and the passerby cannot be distinguished, and if τ is large, the walking waiter stops and suddenly moves, and the denominator of [Equation 3] is very small, and the inner product value is a numerator. This is because, as this increases, it has a great influence on the entire NW, so the deviation becomes very large, and when judging by the NW value alone, the possibility of misjudgement as a passerby increases. It is expected to be possible). Empirically, it is deduced that a τ value that satisfies the NW = 0.5±0.1 section, in which the deviation for the change in the τ value is somewhat small and the deviation between the walking waiter and the passerby occurs, is appropriate. In the above range, the larger the size of the pedestrian speed and the greater the change, the smaller the τ value (for example, 8t _f ), and the smaller the size of the pedestrian speed, the greater the size (for example, 18t _f ), that is, the observation time. Inversely proportional to the average size of the pedestrian's velocity vector V (the average of the magnitude of V between each frame or a set number of frames) or the average rate of change (the average of the absolute value of the rate of change of V between each frame or between a set number of frames) It is judged that it is good to be set.

In the present invention, the immediate area IA and the standby area WA may also be defined in a curved section. In general, road boundaries have many straight sections, but in the case of sidewalks adjacent to intersections, curves or straight road boundaries change the angle little by little, so that the section-sections are made of straight lines to form a curvature as a whole (this is also defined as a'curved section' There is). In particular, in the case of the curved section instantaneous area (IA), since the field of view of the camera is often obscured by a person waiting to walk, etc., the purpose of immediately warning when entering a pedestrian may not be achieved.

However, in most cases there are traffic lights on both sides of a curved section at an intersection, and therefore, the camera 10 is often located on both sides of a curved section. The atmospheric region WA including the curved section immediate region IA and the sector region to be defined in the present invention is based on such an environment, and the atmosphere including the curved section instantaneous region and the sector region in FIG. 4 The sex area and the arrangement of the cameras to define it are shown. That is, the curved section immediate area IA is defined by the first camera 11 looking at the curved section from one side of the curved section and the second camera 13 looking at the curved section from the other side of the curved section. 1, After each instantaneous region IA is created by the second camera 13, the instantaneous region IA is defined as an image acquired from the remaining cameras when the field of view is restricted by a pedestrian or a pedestrian in a specific camera. .

If there is no limit on the field of view in all of the instantaneous regions IA, respectively, the method of creating the instantaneous region IA in the XY coordinate system is to use the XY obtained from the instantaneous region IA generated by the first camera 11. It interpolates the coordinates and the XY coordinates obtained from the instantaneous region IA generated by the second camera 13. For simplicity, two X-Y coordinates can be averaged to define the curve segment immediacy region (IA). Alternatively, an instantaneous area (IA) may be defined as the curve section by putting a weight on the distance between the X-Y coordinate and the camera respectively derived. For example, the distance between the XY coordinates of the immediate area IA derived from the first camera 11 and the first camera 11 is a, and the immediate derived from the second camera 13 and the second camera 13 If the distance between the XY coordinates of the sex area (IA) is b, the coordinates of the curve section immediacy area (IA) are XY of the immediacy area (IA) derived from the first camera 11 as shown in [Equation 4] below. It may be a value obtained by multiplying the coordinates by a and multiplying the XY coordinates of the immediate area IA derived from the second camera 13 by b and dividing by a+b.

When there is a curved section immediate area IA, the waiting area WA is generally formed in a sector shape corresponding to the curved section. Also in this case, as in the coordinate processing in the immediate area IA, the fan-shaped waiting area WA has a first camera 11 looking at the sector area from one side and a second camera 11 looking at the sector area from the other side of the curved section. After creating a waiting area WA defined by the camera 13 and including a sector shape in the first and second cameras 13, respectively, when the field of view is restricted by a pedestrian or a pedestrian in a specific camera, the rest The waiting area WA is defined as an image acquired from the camera. In addition, if there is no restriction on the field of view, the XY coordinates obtained from the atmospheric area WA generated by the first camera 11 and the XY coordinates obtained from the atmospheric area WA generated by the second camera 13 are averaged or weighted. By averaging, the waiting area WA can be defined with the final XY coordinates.

5 is a block diagram of an apparatus for recognizing a pedestrian in a waiting area for a pedestrian crossing according to the present invention. The apparatus for recognizing a pedestrian in the waiting area for a pedestrian crossing according to the present invention includes a camera 10 and a computing device 20 for performing a method for recognizing a pedestrian in the waiting area for a pedestrian crossing according to the present invention. The camera 10 is installed on the sidewalk where two crosswalks meet and may be composed of first and second cameras 11 and 13 looking at the curved area of the sidewalk in a vertical direction. In this case, by a person waiting for a walk or a pedestrian Even if some fields of view are obstructed, the XY coordinates of the curved section immediate area (IA) and the waiting area (WA) including the sector section can be derived, and when the field of view is not obstructed, the first and second cameras (11, 13) It is possible to derive more accurate XY coordinates than using a single camera by interpolating the image acquired in.

S10 detection area setting step S20 pedestrian positioning step
Step S30 Pedestrian Pattern Recognition
10 camera 11 first camera
13 Second camera 20 Operation unit

Claims (4)

According to the image acquired from the camera 10, an instantaneous area (IA) is set along the border of a roadway and a sidewalk with a crosswalk, and a rectangular waiting area (WA) in contact with the instantaneous area (IA) is set. A detection area setting step (S10);
A pedestrian position determining step (S20) of determining the position of the pedestrian in the immediate area IA or the waiting area WA, and
Consisting of a pedestrian pattern recognition step (S30) of grasping the possibility of a pedestrian waiting to cross the pedestrian crossing in the waiting area WA,
Pedestrian pattern recognition step (S30) is a pedestrian crossing waiting, characterized in that it is determined that the probability of waiting for the pedestrian to cross the crosswalk in the waiting area WA as the smaller the value of [Equation 2] is larger. Area pedestrian recognition method (however, t _f is the interframe time of the image acquired by the camera 10, V(t) is the pedestrian's velocity vector at time t, and τ is the time interval of the autocorrelation, Addition in the interval from t = 0 to t = T·t _f is discretely added to the frames obtained at intervals of time t _f from t = 0 to t = T-τ}.
[Equation 2]

The method according to claim 1,
Crossing, characterized in that the pedestrian's position is derived by mapping the coordinates of a specific point of the pedestrian shoe photographed by the camera 10 to an XY plane coordinate system according to an azimuth and elevation angle at which the camera 10 looks at the ground. How to recognize pedestrians in sidewalk waiting areas.
delete Camera 10 and
A pedestrian recognition device in a pedestrian crossing waiting area, comprising a computing device (20) for performing the pedestrian recognition method in the pedestrian crossing waiting area according to claim 1 or 2.

Images