JP2006082618A - Fallen article detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a fallen article detection device capable of monitoring the entire platform by cameras of the minimum number, and suppressing improper reaction caused by a fallen article other than a person. <P>SOLUTION: The fallen article detection device to detect a fallen article based on a screen image-picked up by an image pickup means to pick up an image of a predetermined place comprises an optical flow measurement point setting means to set a plurality of optical flow measurement points, an optical flow detection means to detect the optical flow of the fallen article based on the set measurement point, a fallen article detection means to detect a lump of the fallen article based on the detected optical flow, a falling locus tracing means to trace the falling locus of the identified lump, and a fallen article determination means to determine whether or not the fallen article is a predetermined one based on the lump of the fallen article and the falling locus of the fallen article, which are provided in a predetermined area on the image picked-up screen. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an apparatus for automatically detecting from a screen obtained by a TV camera a person who intentionally descends or unintentionally falls on a track surface or the like from a station platform or the like.

  It is forbidden by law that passengers on railways get off the railway platform from the station platform. Nevertheless, there are no endless passengers who accidentally fall down or deliberately get off the track. In order to ensure the safety of personnel, station staff must be aware of the occurrence of such a situation as much as possible, and depending on the situation, must take measures including an immediate stop of the train.

  However, there is a problem that it is difficult to constantly monitor the entire length of a long platform by a small number of station staff.

  Therefore, an automatic detection device for fallers has been developed. For example, there is a switch mat that detects the weight of a fallen person. This is superior to other methods in that humans can be distinguished from light animals such as small animals depending on the firmness of the switch, but it is installed over the entire length of the platform due to resource issues that require installation and maintenance. It is not possible to do so, and at present, it is only installed in the form of dots in the vicinity corresponding to the entrance / exit of the train. As a result, it is not so popular.

For this reason, the use of screen information photographed by a camera has been studied as being capable of covering the entire length of the station platform with a small number of sensors. A system has been announced that detects a foreign object on a track as "an object higher than the track" by using a binocular stereo camera mounted so as to hang under the platform roof. (For example, Patent Document 1)
JP 2003-246268 A (page 3, FIG. 3)

  However, there is also a possibility that sufficient detection performance cannot be obtained in detecting a human body that has been flattened after dropping. In addition, since a binocular stereo camera is used, the cost of the camera body increases. More importantly, there is a problem that the length range of the platform that can be guarded by one binocular camera cannot be taken so wide as to make stereoscopic viewing function reliably.

  The present invention has been made from such a point of view, and provides a fallen object detection apparatus that monitors the entire platform with as few cameras as possible and suppresses erroneous reactions caused by fallen objects other than people (such as dust) as much as possible. The purpose is to do.

The present invention is a falling object detection device for detecting a falling object based on a screen imaged by an imaging means for imaging a predetermined place, and sets a plurality of optical flow measurement points in a predetermined area on the screen. Optical flow measurement point setting means, optical flow detection means for detecting the optical flow of the photographed falling object based on the set measurement point, and falling object for detecting the falling object based on the detected optical flow Detection means, and a fall trajectory tracking means for tracking the fall trajectory of the detected fallen object;
It has a configuration including falling object discriminating means for discriminating whether or not the object is a predetermined falling object based on the detected falling object and the locus of falling object falling.

  With this configuration, it is possible to determine whether or not the falling object is a predetermined falling object by tracking the locus of the falling object based on the optical flow at the measurement point.

  Also, the optical flow measurement point setting means is sensitive to the magnitude of the optical flow at each measurement point on the screen, dense and highly sensitive to a landscape far from the photographing means, and close to the photographing means. Is a sparse and low sensitivity setting.

  With this configuration, the optical flow can be measured with a single photographing unit with higher accuracy and covering a wide range.

  Moreover, the falling object detection means classifies the optical flow detected at the measurement point into those close to the horizontal direction and those close to the vertical direction, and determines whether or not the lump of falling objects in each direction has a predetermined size. When it is determined and the size is a predetermined size, it is determined whether the fallen object is a fall of a person or an object other than that.

  With this configuration, it is possible to identify a person falling in the vertical direction and, for example, passing a train in the horizontal direction.

  Further, the falling object detection means is configured to determine a false optical flow caused by the vibration of the imaging means when the distribution of the optical flow detected at the measurement point in the entire area of the screen has a predetermined size.

  With this configuration, it is possible to exclude the occurrence of a false optical flow due to the vibration of the imaging unit.

  The present invention can more reliably detect falling objects such as people falling on a wider range of station platforms, etc., with a smaller number of cameras than conventional methods such as stereo vision and switch mats. It becomes.

In the present invention, by focusing on the extraction of “motion” information of falling objects from a monocular screen, it is possible to detect a fallen person with a smaller number of cameras, and whether the falling object is a person or a non-person. Judgment was also possible. The outline is as follows.
(1) Continue to wait for something to move down across the edge of the platform,
(2) When such movement is detected, the trajectory of the moving object is tracked,
(3) Determine whether the fallen object is human or not from the characteristics of the fallen object and the fall trajectory.

  (1) continues to monitor the optical flow at a large number of measurement points arranged on the monitoring screen, and when a certain condition is satisfied regarding the distribution of the generated flow and the characteristics of each flow, it is judged as `` detecting a falling object '' It is realized by doing.

  (2) is realized by obtaining the falling trajectory as the sum of the entire optical flow caused by the detected falling object.

  The determination in (3) was realized by processing the spatio-temporal characteristics of the fall trajectory and the properties of the optical flow group that is the basis of the characteristics with a discriminator created in advance by data mining processing.

  The block diagram of the falling object detection apparatus of an Example is shown in FIG.

  The falling object detection device 1 includes an imaging unit 2, a screen processing unit 3, a falling object detection processing unit 4, a falling object tracking processing unit 5, a falling object discrimination processing unit 6, an alarm processing unit 7, a display unit 8, and an input unit 9. And an alarm unit 10.

  The imaging unit 2 is a TV camera or the like that monitors a station platform.

  The screen processing unit 3 includes a display processing unit that displays a captured screen on the display unit 8, a storage unit that stores the screen for each frame, and the like.

  The falling object detection processing unit 4 continues to monitor the optical flow at a large number of measurement points arranged on the screen of the display unit 8, and when certain conditions are satisfied with respect to the distribution of the generated flow and the properties of the individual flows. Judged as falling object detection.

  The falling object tracking processing unit 5 obtains a falling locus of the optical flow caused by the detected falling object.

  The fallen object discrimination processing unit 6 discriminates whether or not the fallen object is a person from the characteristics of the obtained fallen object and the fall trajectory.

  The alarm processing unit 7 notifies the user of the presence of a fallen object through the display unit 8 and the alarm unit 10.

  The display unit 8 displays a captured screen, a measurement point, a fall locus, an alarm, and the like.

  The input unit 9 inputs measurement point setting information such as a home boundary line.

  The alarm unit 10 notifies the station staff of the danger by voice or sound.

  FIG. 2 shows a flowchart of processing of the falling object detection device of the embodiment.

  First, a camera is installed (step S1).

  In order to reliably detect “falling from the platform to the track”, whether it is a person or garbage, using the movement, the camera of the present system is installed on the opposite side across the track as shown in FIG.

  FIG. 5 shows a camera layout.

  In FIG. 5, (1) to (3) and (1 ') to (3') indicate the camera and the respective viewing angles.

  The camera arrangement is shown schematically.

  When the platform is in a straight line, one camera can cover a distance of 70m to 140m, so that one 200 meter long platform can be covered with three cameras.

  Due to the high selectivity of the focal length of the lens attached to the camera, what is important here is not the actual distance but the composition of the home in the camera field of view. In other words, if the image of the platform on the screen is full up and down the screen in the foreground, just half of the top and bottom in the distant view, the actual distance is not a problem as long as it spreads. The value of 70 m described above has a large degree of arbitraryness because it becomes a problem only to select a lens having an appropriate focal length so that the home screen reflected in the field of view is like that.

  Next, the platform edge line and the track position are specified (step S2).

Next, all measurement point positions are determined, and “speed sensitivity” for each measurement point is determined (step S3).
The “measurement point” is set by arranging optical flow measurement points (hereinafter referred to as “measurement points”) in accordance with the perspective of the platform image on the screen.

  FIG. 6 is an explanatory diagram of the arrangement of measurement points on the screen.

  This figure is a schematic diagram, and the number of measurement points does not match the actual value. It is also possible to hide the measurement points and boundary lines on the actual screen.

  The user falls within the trapezoidal trapezoidal measurement point distribution range, the platform edge line (hereinafter referred to as “boundary line” for the sake of convenience) and the lower end of the line substantially along the line and both of them. Specifies the horizontal end position.

  Based on these pieces of information, the top edge of the measurement point distribution range, the position of each measurement point, the “speed sensitivity” for each measurement point, and the like are automatically determined.

  As described above, in addition to the coordinates on the screen, each measurement point has a “speed sensitivity” that is applied when the optical flow is measured at that point or when falling object tracking is started from that point.

  In the distance, only small movements are sensitive to small movements, and in the foreground, only sufficiently large movements generate optical flow.

  Each measurement point also includes information on whether it is above or below the boundary line (platform edge line) designated by the user.

  Next, drop alert processing is performed (step S4 in FIG. 2).

  FIG. 3 shows a flowchart of the drop alert process.

  First, the state is set to the “falling object detection mode” and the falling object detection process is performed (step S11).

  FIG. 4 shows a flowchart of the falling object detection process.

  In the falling object detection process, the optical flow is measured every two to three frames. Hereinafter, this is referred to as one cycle.

  Falling objects are extracted from the distribution of the measured optical flow.

  That is, using the features such as the vector direction of the optical flow and the degree of gathering, a fall event that may cause a person to fall is identified.

  First, the optical flow is calculated at all measurement points (step S21).

  Next, it is confirmed whether or not an optical flow has occurred (step S22).

  If no optical flow has occurred, the process returns to step S21.

  On the other hand, when an optical flow in a direction close to the downward direction with a certain size or more is shown, the measurement point is identified as “falling object candidate movement”.

  The other measurement points indicating non-zero movement are identified as “train candidate movement”.

  Thereafter, the same processing is performed in parallel on the points indicating “train candidate motion” and the measurement points indicating “falling object candidate motion”.

  By removing the vector magnitude information from the measurement points that show a motion exceeding a certain threshold, a binary matrix (part 1) is created in which the motion points are 1 and the non-motion points are 0 ( S23 step).

  Since the measurement result of optical flow is inherently unstable, spatial “hole filling” is performed using this matrix.

  Specifically, when a certain measurement point is “with movement”, this is realized by assuming that adjacent measurement points around it are also “with movement”.

  The binary matrix (part 1) subjected to spatial “hole filling” is further multiplied in time by being attenuated by multiplying by an appropriate time constant (step S24).

  For each of “candidate train movement” and “candidate object fallen”, the lateral and temporal movements at the position of a measurement point or It is determined whether or not there is downward motion, and a binary matrix (part 2) is created (step S25).

  Neighborhood labeling is performed on each of the binarization matrices of “train candidate motion” and “falling object candidate motion”.

  What is obtained by the labeling is the distribution status of the “candidate motion block of train” and the “candidate block of falling object motion” (step S26).

  Based on the distribution status of each of these blocks, “falling object detection”, “train entry detection”, and “camera vibration detection” are identified.

  Specifically, among “falling object candidate motion blocks”, those that satisfy the following conditions are identified as “falling object detection” from the platform (step S27).

The purpose of these conditions is to ensure that the “falling object candidate motion mass” is not a noise caused by train or camera vibrations or a small object such as a bird.
(a) There is a certain extent in the horizontal and vertical directions.
(b) The horizontal spread does not exceed a certain limit value.
(c) Cutting below the “borderline (platform edge)”.
(d) It protrudes above the “boundary line”.
(e) Apart from the condition (a), a certain number of “measurement points” are constituent elements.
(f) There is no “train candidate motion lump” in the vicinity of the center of gravity of the “falling object candidate motion lump”.

  Conditions (a) and (e) reflect the fact that “too small fallen objects cannot be humans”.

  Condition (b) is that when a “falling object candidate motion lump” having a large spatial extent is observed, it is highly likely that the entire field of view is moving due to vertical vibration of the camera itself.

  In the case of a person, it is not larger than a certain spread.

  Conditions (c) and (d) together require that the “falling object candidate motion mass” is just falling across the edge of the platform.

  The condition (f) is to make sure that the “falling object candidate movement lump” is not a false downward movement that appears by chance due to the lateral movement of the train in the field of view.

  Also, the width and height in the screen of an object obtained by ORing the distribution of “the train of candidate trains” obtained by the labeling that has a certain extent or more are obtained. When this exceeds a certain value, it is identified as “train entry detection” where the train is moving on the screen.

  In addition, when the width and height of the result of performing the same processing for both the “candidate of movement of train candidates” and “the lump of candidate movement of falling objects” exceed a certain value, the entire camera vibrates. It is identified as “camera vibration detection”.

  In either case, records of the occurrence of train entry events and camera vibration events are retained for a certain period of time and used later as one of the materials for determining whether or not a person drop warning can be issued.

  When the “falling object” is detected in this way, the state is changed to the “falling object tracking mode”, and the falling locus is traced from the next processing.

  First, the fall object tracking start process is performed (step S12 in FIG. 3).

  Trajectory tracking is started for the detected “falling object”.

  Immediately after a fallen object is detected, list all the measurement points in a fairly wide range centered on the “lumps” that can clearly read the downward flow, and list the measured points of the fallen object. The initial position of the “tracking point” used for the trajectory tracking is set, and the tracking process is started.

  Further, although the center of gravity of the measurement points constituting the “falling object” is used as the initial value of the position of the falling object, this is not essential. This is because, in the fallen object discrimination, attention is paid to various other attributes rather than the locus position.

  Next, a fallen object tracking process is performed (step S13).

  Tracking is done by an average flow of all “tracking points” where motion is clearly detected.

  Thereafter, the average movement of the “tracking points” is obtained, and the positions of the falling objects are continuously estimated by shifting the positions of all the “tracking points” according to the result. As long as falling object tracking continues, a “tracking point” is newly added for each cycle by picking up a measurement point having a downward movement in the vicinity of the current center of gravity of the falling object.

  The tracking ends when the number of “tracking points” reaches 0 or when 15 cycles have been tracked.

  As a result of the tracking, falling object trajectory information for a maximum of 15 cycles is obtained.

  Falling object trajectory information includes the following information for each cycle.

  The position of the falling object, the amount of movement of the falling object per cycle, the number of tracking points that could be used for tracking the falling object, and the motion vector for each tracking point used for tracking the falling object.

  It is checked whether or not the tracking process has been completed normally (step S14).

  Specifically, if the number of tracking points reaches 0 without waiting for at least 5 cycles to be tracked, tracking of the falling object has failed or the falling object has fallen firmly enough to be tracked. Returning to the fallen object detection process, it is like noise that does not continue.

  If the tracking of the falling object is successful, the state is set to the “falling object discrimination mode”, the process proceeds to the falling object discrimination process, and the falling object discrimination process is performed (step S15).

  It is discrimination of the type of fallen object from the fall trajectory.

  In the fallen object discriminating process, the obtained fallen object trajectory information is analyzed according to the procedure described below, and it is discriminated whether the fallen object is “person” or “non-person”.

  FIG. 7 shows an explanatory diagram of the fall trajectory.

  First, the trajectory is scanned from the start point, and a portion moving relatively linearly as shown in FIG. 7 is cut out. This is called a “straight period” for convenience. In many fallen objects, the fall trajectory during this period exhibits a clean straight line facing downward.

  When the trajectory scan of the fallen object is completed, the fall trajectory, the straight line period of the fall, and the movement history of all the tracking points of the fall are displayed on the screen.

  Further, the attribute used for discrimination of the fallen object is extracted from the fall trajectory information and the optical flow information of the measurement points constituting the fallen object immediately before the start of the trace tracking.

An example of the attribute is shown below.
(a) Absolute value [deg] of the inclination of the locus of the straight line period when the vertical direction is 0.
(b) The degree of variation in the horizontal direction from the regression line fitted to that part of the fall trajectory during the straight line period.
(c) Vertical length of the trajectory during the straight period.
(d) The percentage of upward movement included in the straight line period.
(e) The average value of downward movement during the straight line period.
(f) Others The various attributes of the fallen object and its fall trajectory are input to the discriminator, and the class of whether the fallen object is most likely to fall under “person” or “non-person” as its output. Get the discrimination result.

  As a discriminator, a “decision table” that is widely known as one of data mining methods is used.

  A discriminator is implemented by incorporating a decision table produced in advance using a large number of teacher data into the program.

  In addition, another determination method having a low ability to determine a person, for example, determination based on Mahalanobis general distance is performed in parallel, and the determination result is output for reference to the user.

  In this case, those that could not be identified for safety are output as a person.

  As a result of the determination, when it is determined that the fallen object is “person”, after confirming that there is no occurrence of an event that seems to be a train intrusion or camera vibration immediately before and after, an alarm is displayed on the display unit 8, The alarm unit 10 notifies the station staff by voice or sound.

  As described above, compared to conventional methods such as stereo vision and switch mats, it is possible to more reliably detect falling objects such as people falling on a wider range of station platforms with a smaller number of cameras. Become.

Regarding the embodiment including the first example, the following additional notes are disclosed.
(Appendix 1) A falling object detection device for detecting a falling object based on a screen imaged by an imaging means for imaging a predetermined place, wherein a plurality of optical flow measurement points are set in a predetermined area on the screen. Optical flow measurement point setting means, optical flow detection means for detecting the optical flow of the photographed falling object based on the set measurement point, and falling object for detecting the falling object based on the detected optical flow Detection means, fall trajectory tracking means for tracking the fall trajectory of the detected fallen object, and fallen object discrimination for discriminating whether the object is a predetermined fallen object based on the detected fallen object and the fall trajectory of the fallen object And a falling object detecting device.
(Supplementary note 2) The optical flow measurement point setting means is sensitive to the magnitude of the optical flow at each measurement point on the screen, close to the scenery far from the photographing means, and close to the photographing means. The fallen object detection device according to appendix 1, wherein the falling object detection device is set to be sparse and low sensitivity.
(Supplementary note 3) The fallen object detection means classifies the optical flow detected at the measurement point into those close to the horizontal direction and those close to the vertical direction, and whether or not the lump of fallen objects in each direction has a predetermined size. The fallen object detection device according to appendix 1, wherein the fallen object is determined to be whether the fallen object is a fall of a person or the passage of another object when the size is a predetermined size.
(Supplementary note 4) The fallen object detecting means determines that the optical flow detected at the measurement point is a false optical flow caused by the vibration of the photographing means when the distribution of the optical flow in the entire area of the screen is a predetermined size. Falling object detection device.
(Appendix 5) A falling object detection method for detecting a falling object on the basis of a screen imaged by an imaging means for imaging a predetermined place, wherein a plurality of optical flow measurement points are set in a predetermined area on the screen. Optical flow measurement point setting step, optical flow detection step for detecting the optical flow of the captured fallen object based on the set measurement point, and fallen object for detecting the fallen object based on the detected optical flow A detection step, a fall trajectory tracking step for tracking the fall trajectory of the detected fallen object, and a fallen object discrimination for discriminating whether or not the fallen object is a predetermined fallen object based on the detected fallen object and the fall trajectory of the fallen object And a fallen object detection method.
(Appendix 6) A falling object detection program for detecting a falling object on the basis of a screen imaged by an imaging means for imaging a predetermined place, wherein a plurality of optical flow measurement points are set in a predetermined area on the screen. Optical flow measurement point setting step, optical flow detection step for detecting the optical flow of the captured fallen object based on the set measurement point, and fallen object for detecting the fallen object based on the detected optical flow A detection step, a fall trajectory tracking step for tracking the fall trajectory of the detected fallen object, and a fallen object discrimination for discriminating whether or not the fallen object is a predetermined fallen object based on the detected fallen object and the fall trajectory of the fallen object Falling object detection program for causing a computer to execute steps.
(Supplementary note 7) The fallen object discriminating means includes the absolute value of the slope of the trajectory in the straight line period, the horizontal variation from the regression line applied to that part of the fall trajectory in the straight line period, the vertical length of the straight line period, the straight line period The fallen object detection device according to appendix 1, wherein the fallen object detection device is determined based on a ratio of upward motion included therein and an average value of downward motion during a linear period.

  INDUSTRIAL APPLICABILITY The present invention can be applied to a use for providing a falling object detection device that detects a fallen person in a station platform.

Configuration diagram of falling object detection device of the embodiment Flow chart of processing of the falling object detection device of the embodiment Flow chart of fall alert processing Flow chart of falling object detection process Camera layout Illustration of measurement point arrangement on the screen Illustration of the fall trajectory

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Falling object detection apparatus 2 Imaging | photography part 3 Screen processing part 4 Falling object detection processing part 5 Falling object tracking processing part 6 Falling object discrimination | determination processing part 7 Alarm processing part 8 Display part 9 Input part 10 Alarm part

Claims (5)

A falling object detection device that detects a falling object based on a screen imaged by an imaging means for imaging a predetermined place,
Optical flow measurement point setting means for setting a plurality of optical flow measurement points within a predetermined area on the screen,
An optical flow detection means for detecting an optical flow from the captured screen based on the set measurement points;
Falling object detection means for detecting a falling object based on the detected optical flow;
A fall trajectory tracking means for tracking the fall trajectory of the detected fallen object;
A falling object detection device comprising: a falling object determining means for determining whether or not the object is a predetermined falling object based on the detected falling object and the locus of the falling.   The optical flow measurement point setting means is sensitive to the magnitude of the optical flow at each measurement point on the screen, dense and sensitive to scenery far from the photographing means, and sparse to scenery close to the photographing means. 2. The falling object detection device according to claim 1, wherein the sensitivity is set to low sensitivity.   The fallen object detection means classifies the optical flow detected at the measurement point into those that are close to the horizontal direction and those that are close to the vertical direction, and determines whether or not the lump of fallen objects in each direction has a predetermined size. 2. The falling object detection device according to claim 1, wherein when it is a predetermined size, it is determined whether the falling object is a fall of a person or an object other than that.   2. The falling object detection unit according to claim 1, wherein the falling object detection unit determines that the optical flow detected at the measurement point in the entire area of the screen has a predetermined size as a false optical flow caused by vibration of the imaging unit. apparatus. A fallen object detection method for detecting a fallen object based on a screen shot by a shooting means for shooting a predetermined place,
An optical flow measurement point setting step for setting a plurality of optical flow measurement points in a predetermined area on the screen;
An optical flow detection step for detecting the optical flow of the captured fallen object based on the set measurement points;
Based on the detected optical flow, a falling object detection step for detecting a falling object,
A fall trajectory tracking step for tracking the fall trajectory of the detected fallen object;
A fallen object detection method comprising: a fallen object discrimination step for discriminating whether or not a fallen object is a predetermined fallen object based on the detected fallen object and a fall trajectory of the fallen object.

Images