JP2002197445A - Detector for abnormality in front of train utilizing optical flow - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a detector of abnormality in front of a train utilizing an optical flow, the detector of which can detect a passenger making abnormal movement which may possibly results in falling from a platform, not only at straight line but also at curves. SOLUTION: The detector of abnormality in front of a train consists of a monitor camera 11 for consecutively photographing the scene of a monitoring object, an A/D conversion part 12, a time-sequential image storage part 13 for storing time sequential images, a first image first derivative gradation part 14 for taking out a first image in a time-sequential 2 image to take first derivative to each pixel, a regularization parameter table citing part 15 for citing the first derivative value of each pixel of the first picture to give an optimal regularization parameter, an arithmetic processing part 16 and a warning part 17. The part 16 consists of an image processing means for calculating the estimated of the optical flow, an estimated speed selection means for selecting the estimated speed of the optical flow entering the angle width of a dangerous direction with respect to the moving direction of the train from among the estimated speed of optical flows, a judging means for judging the presence and absence of a moving object which can clearly be judged to be dangerous, based on the estimated of the optical flow in a direction vertical to the moving direction of the train.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention processes a time-series image taken by an ITV camera mounted on a train and monitoring a traveling direction to process a person or a moving object (hereinafter referred to as a passenger) from a platform.
The present invention relates to a train forward abnormality detection device that uses an optical flow to detect an abnormality such as a fall of a moving object (if necessary).

[0002]

2. Description of the Related Art Various devices for detecting a fall of a passenger from a platform have been developed or proposed. For example, Japanese Patent Application Laid-Open Nos. 7-329772 and 9-21856.
JP, JP-A-9-26472, JP-A-9-19
No. 3789, JP-A-9-193803 and JP-A-2000-95099. However, the fall detection devices disclosed in these patent publications are:
Both are fall detection devices installed on the ground. Therefore, when the fall detecting device detects a fall of a passenger or the like from the platform, a communication means for notifying the train of the fall is necessary.
That is, a wireless communication system that connects the fall detection device installed on the ground and the driver's seat of the train is required. When the driver was informed that a passenger had fallen off the platform in the driver's seat,
Actions such as operating the emergency brake of the train will be taken. In this way, the driver will eventually take some action, but in view of the reliability of the actual wireless communication system and the responsiveness of the entire system including the driver, communication is interrupted or instantaneous. However, there is a problem that there is no danger that the driver's response will be delayed.

Conventionally, there has been an idea to detect an obstacle in front of the train by template matching from an image obtained by an ITV camera mounted on the train and monitoring the traveling direction. However, if the detection target is known, detection by template matching is possible, but it is impossible to detect various objects such as a fall of a passenger from a platform. In addition, the effect of the movement of the own train cannot be excluded only from the difference between consecutive frames,
The target velocity field cannot be determined. Therefore, an obstacle detection device that detects an obstacle in front of the train by template matching from an image obtained by an in-vehicle ITV camera has not been developed. An abnormality monitoring device that is not mounted on a vehicle but registers a track under the platform in a template image and determines that a person is getting off the track when the degree of similarity decreases is disclosed in Japanese Patent Application Laid-Open No. 9-282455. ing.

There is a device that emits a laser beam from a laser radar attached to the front of a vehicle, calculates a distance based on reflected light from an object, and detects an obstacle, but when a person moves in a complicated manner like a station platform. In addition, it is difficult to determine whether or not there is an abnormality.

[0005]

The problem to be solved by the present invention is to utilize an optical flow which can detect an abnormally moving passenger who may fall off a platform not only on a straight road but also on a curve. An object of the present invention is to provide a train forward abnormality detection device.

[0006]

In order to solve the above problems, an ITV camera mounted near the driver's seat of a train monitors a train traveling direction, and a regularization method is applied to a time-series television image taken by the ITV camera. The image processing used is used to estimate the optical flow for each pixel, and further, by selecting the estimated velocity vector of the optical flow at an angle that falls within the dangerous direction angle width with respect to the train traveling direction, it is clear that the dangerous flow ahead of the train The presence of a moving object that can be determined to be present is detected.

In the image processing, the moving amount (deviation) of background information occurring between successive image frames is replaced with instantaneous velocity field vector information of an optical flow by a gradient method, and the background data is used using the direction data. The moving amount is obtained, the background information is corrected by matching the background information between the successive image frames, and the estimated speed of the moving object existing between the successive image frames after the background moving correction is obtained. This overcomes the difficulty of image processing of background difference and time difference when background information always moves for each image frame.

Further, in the image processing, a first differentiation process is performed on a first image of a continuous two frames and a first image of the second image, and a first differentiation value corresponding to a pixel to be operated is referred to. An optimal regularization parameter is selected from the regularization parameter table, and the first image and the second image of two consecutive frames are subjected to image processing using the selected regularization parameter to obtain a gradient method optical flow.

[0009]

BEST MODE FOR CARRYING OUT THE INVENTION In-vehicle IT
As shown in FIG. 2, the optical flow extracted from the two time-series images obtained by shooting with the V camera becomes an estimated velocity vector radially spreading from one point FOE (focus of expansion) on the image. The FOE is also called an infinity point or a vanishing point. In FIG. 2, six radial straight lines with arrows indicate optical flows, and two thick straight lines without arrows indicate rails.

In the present invention, a passenger who has a possibility of falling is defined as follows. That is, a passenger who may fall from the platform is a person who enters at a certain speed or more in a direction perpendicular to the traveling direction. A situation in which the vehicle rushes toward the traveling train instead of a direction perpendicular to the traveling direction is also assumed, but in such a case, the component of the velocity vector always includes a component in the transverse direction from the platform to the track. In order to stably detect a passenger who may fall down on a straight line or a curve, it is necessary to always recognize the FOE in a time-series image.

FIG. 3 is a diagram in which the divergent velocity field is divided into four parts with the FOE as the center. The direction of the estimated velocity vector is determined for each of the first, second, third, and fourth regions divided into four. That is, for example, four regions in FIG. 3 are divided by orthogonal XY axes, and Y at the boundary between the first region and the fourth region.
When the axis is 0 ° and the angle is clockwise, FOE
The optical flow diverging around the center is from 0 ° to 90 ° in the first region, from 90 ° to 180 ° in the second region, and from 180 ° to 27 ° in the third region.
To an angular width of 0 °, and from 270 ° to 3 in the fourth area
All within an angle width of 60 °.

Therefore, the first area and the fourth area in the screen shown in FIG.
In the track image in which the platform of the train is shown in the area, it is determined that the optical flow in the angular direction between the two solid arrows is dangerous. That is, in the first area, the velocity field angle width α, that is, (θ ₁ to θ ₁₎ sandwiched between the arrow of the upper left angle θ _{1 and} the arrow of the lower left angle θ _2. θ ₂ ) when there is an optical flow having an angle θ, or in the fourth area, an arrow having an upper right angle θ ₃ and an angle θ having a lower right angle θ
Velocity field angular width defined between the ₄ arrow β i.e. (theta ₃ ~
When there is an optical flow having an angle θ that falls within θ ₄ ), it is determined that there is a dangerous moving object in the train traveling direction, particularly a passenger who may fall.

The velocity field angular width in the first region (θ ₁ 〜
θ ₂ ) is a dangerous direction angle width with respect to the train traveling direction, that is, an angle width falling within a predetermined angle width including a direction perpendicular to the train traveling direction. The direction perpendicular to the traveling direction of the train in the first area is a direction perpendicular to the dotted line on the right side indicating the track position, and is therefore approximately 260 °, as indicated by the dotted arrow slightly downward and to the left. Therefore, the predetermined angle width is set to, for example, 60 °.
When equal angular width with respect to the to and dotted arrows, theta ₁ is 230 °, θ ₂ becomes 290 °. Therefore, when it is detected that an optical flow having an angle of 230 ° to 290 ° exists in the first area, it is determined that a dangerous moving object exists in the train traveling direction.

[0014] The velocity field angular width in the fourth region (θ ₃ ~
θ ₄ ) is an angle width in a dangerous direction with respect to the train traveling direction, that is, an angle width falling within a predetermined angle width including a direction perpendicular to the train traveling direction. The direction perpendicular to the train traveling direction in the fourth area is a direction perpendicular to the dotted line on the left side indicating the track position, and is therefore approximately 100 ° as indicated by the dotted arrow slightly downward and to the right. Therefore, the predetermined angle width is set to, for example, 60 °.
When equal angular width with respect to the to and dotted arrows, theta ₁ is 70 °, θ ₂ becomes 130 °. Therefore, from 70 ° to 1
When it is detected that an optical flow having an angle of 30 ° exists in the fourth area, it is determined that there is a dangerous moving object in the train traveling direction.

Optical flow refers to the apparent velocity distribution of a certain object on the screen that is perceived by the observer. The basic formula for estimating the optical flow is derived from two assumptions. One is that the gray value of the target physical point is kept constant during exercise. Let P (x, y, t) be the gray value of a point (x, y) on the screen at time t,
After a short time Δt, the physical point moves to the point (x + Δx, y + Δy), and its gray value is expressed as P (x + Δx, y + Δy, t +
Δt), the following equation 1 is established.

[0016]

(Equation 1) When this is Taylor-expanded and Δt is limited to 0, Equation 2 is obtained.

[0017]

(Equation 2) Where u and v are apparent velocity vectors, and P _x ,
_Py and _Pt are partial derivatives with respect to x, y and t, respectively.

Another assumption is that the apparent speeds u and v change smoothly. Smoothness E _c of the speed, as in Equation 3, to evaluate the sum of the Laplacian of the velocity vector.

[0019]

(Equation 3)

For the actual calculation, P _x ,
_{P y,} P _t results in the error is approximated by a finite-difference. Equation 3
Needs to be discretized, and the smoothness of the velocity distribution is actually evaluated by Expressions 4 and 5. Here, _Eb represents a constraint condition of the optical flow based on the assumption that the gray value of the target physical point is kept constant during the movement.

[0021]

(Equation 4)

(Equation 5)

However, the apparent velocities u and v of the vector display in Equation 5 are the average values of the neighboring velocity vectors as shown in Equations 6 and 7.

(Equation 6)

(Equation 7)

When Equations 4 and 5 are integrated by introducing a regularization parameter α, which is a weighting coefficient for smoothness, an evaluation equation for the error E as shown in the following Equation 8 is given.

(Equation 8)

An equation for calculating the velocity distribution is obtained under the condition that the error E takes the minimum value. After all, if the following equations 9 and 10 are repeatedly converged, an estimated speed can be obtained. Here, k is a repetition parameter.

(Equation 9)

(Equation 10)

The present invention relates to an I-type vehicle mounted near the driver's seat of a train.
The time-series television image captured by the TV camera is subjected to the image processing using the above-described regularization method to perform optical flow estimation for each pixel, and further, to a predetermined angle range from the direction perpendicular to the train traveling direction or from the direction perpendicular thereto. By selecting an estimated velocity vector of an optical flow existing in a certain dangerous direction angle width, it is detected that a moving object that can be clearly determined to be dangerous exists in front of the train.

That is, as shown in the block diagram of FIG. 1, a train forward abnormality detecting apparatus using an optical flow according to an embodiment of the present invention has a surveillance camera 11 for continuously photographing a scene to be monitored. A / D conversion unit 12 that converts the converted analog image into a digital image, a time-series image storage unit 13 that stores the time-series image input from the A / D conversion unit 12, and a time-series image that is stored in the time-series image storage unit 13. A first image primary differentiator 14 and a first image primary differentiator 1 that take out the first image from the series 2 images and perform the first differentiation for each pixel
4 includes a regularization parameter table reference unit 15, an arithmetic processing unit 16, and an alarm unit 17 that refer to the primary differential value of each pixel of the first image, which is the output of No. 4, and provide an optimal regularization parameter.

The arithmetic processing section 16 includes a time-series image storage section 13
The optical flow of the image of the monitoring target scene is estimated by using the first image and the second image in the two time-series images accumulated in the storage unit and the regularization parameters from the regularization parameter table reference unit 15 as inputs, and the estimated optical This is a device that performs a predetermined calculation process on a flow to select an estimated optical flow speed in a direction perpendicular to the train traveling direction, and determines whether a dangerous moving object exists in front of the train.

In terms of component configuration, the arithmetic processing unit 16 includes, for example, a CPU for performing various operations and control, a ROM for storing programs, a RAM for storing various data, a keyboard, a display, and a printer for inputting data and the like. And so on.

In terms of the functional configuration, the arithmetic processing unit 1
6, an image processing means for calculating an optical flow estimated speed, an estimated speed selecting means for selecting an optical flow estimated speed in a direction perpendicular to the train traveling direction from the optical flow estimated speed calculated by the image processing means,
Determining means for determining the presence or absence of a moving object which can be clearly determined to be dangerous based on the estimated optical flow speed in a direction perpendicular to the train traveling direction selected by the estimated speed selecting means.

Next, the operation of the train forward abnormality detecting device using the optical flow according to the embodiment of the present invention shown in FIG. 1 will be described with reference to the flowchart of FIG.

In FIG. 6, an image photographed by the surveillance camera 11 is converted into a digital signal by an A / D converter 12, and
It is input to the time-series image storage unit 13. Time-series two images of the first image and the second image are stored in the time-series image storage unit 13 (101).

Subsequent to step 101, the first image primary differentiator 14 performs a first differentiation process on the first image (102). The first differentiation is performed for every pixel of the first image for each pixel.

Subsequent to step 102, the regularization parameter table section 15 refers to the regularization parameter table and selects an optimal regularization parameter α for the first derivative of each pixel of the first image (103). ).

After step 103, the arithmetic processing unit 16
Performs an optical flow estimation operation on all pixels for each pixel (104). The optical flow estimation calculation is performed using the above equations 8 to 10 for each pixel.
The data used for the calculation is the image data of the first image and the second image stored in the time-series image storage unit 13, and the regularization parameter α of the regularization parameter table unit 15. Note that the regularization parameters will be described in detail again after the description of the flowchart in FIG. 6 ends.

Following step 104, the arithmetic processing unit 16
Generates an angle sequence table of estimated velocity vectors (105). The angle sequence table of the estimated velocity vector, as shown in FIG. 5, describes the angle θ of the estimated velocity vector of the optical flow in the left column, and displays the position of the optical flow having this angle by coordinates X and Y. It is. The position of the optical flow is, in other words, the position on the XY screen of the pixel that has generated the optical flow.

FIG. 5 shows that the boundary between the first area and the second area is 92
°, the boundary between the second region and the third region is 185 °, and the boundary between the third region and the fourth region is 270 °. This is FO
This is because the position of E constantly fluctuates as the train advances. In the four regions of FIG. 5 divided by such a boundary angle, the optical flow diverging around the FOE has an angular width of 0 ° to 92 ° in the first region and 92 ° to 185 ° in the second region. To the angular width, 18 in the third area
From 5 ° to 270 °, and 2 in the fourth area
It falls entirely within the angle range of 70 ° to 360 °.

However, if, for example, the optical flow to be included in the angular width of the first area has an angle of 250 °, this optical flow will be arranged in the third area in FIG. Similarly, for example, if an optical flow to be included in the angular width of the fourth region has an angle of 80 °, this optical flow will be arranged in the first region in FIG.

Following step 105, the arithmetic processing unit 16
Is a boundary of four regions centered on the FOE, that is, a first region,
A process for determining boundaries between the second, third, and fourth areas is performed (106).

The FOE is determined as follows. The arithmetic processing unit 16 determines the direction distribution state of the estimated velocity vector and geometrically determines the vicinity of the FOE. After statistically determining the reliability of the extracted estimated speed vector by the reliability analysis process, the arithmetic processing unit 16 sets a point where the virtual extension line intersects in the direction of the starting point of each estimated speed vector as FOE. Although the intersection of a plurality of virtual extension lines is not a single point but a specific distribution is assumed, the FOE is determined by the least squares method.

When the FOE is determined, a boundary between the first area and the fourth area is determined. The boundary angle between the first area and the fourth area is determined by the angle of view of the surveillance camera and the level of the track (rail). If the train-mounted surveillance camera captures a sufficiently distant image, the area boundary is determined without considering the level.

Following step 106, the arithmetic processing unit 16
Deletes the optical flow generated in the scenery in the train traveling direction from the angle sequence table of the estimated speed vector (107). The optical flows that occur in the scenery in the train traveling direction are radial with the FOE at the center, and can be specified by angle in the angle sequence table of the estimated speed vector. The arithmetic processing unit 16 searches the angle sequence table of the estimated speed vector, identifies an optical flow generated in the scene in the train traveling direction, and deletes the optical flow.

Following step 107, the arithmetic processing unit 16
Generates a table of optical flows generated by moving objects existing in the train traveling direction (108). When step 107 is completed, the optical flows left in the angle sequence table of the estimated speed vector do not include the optical flows generated in the scenery in the train traveling direction. Therefore, in step 108, the arithmetic processing unit 1
6 shows an angle sequence table of the estimated speed vector as a first
By editing the area and the fourth area, a table of an optical flow generated by a moving object existing in the train traveling direction is generated.

Subsequent to step 108, the arithmetic processing unit 16
Determines whether there is an estimated speed vector existing in the dangerous direction angle width (109). That is, the arithmetic processing unit 1
6 searches a table of optical flows generated by the moving object existing in the train traveling direction generated in step 108, and finds an estimated velocity vector existing in the first area space, and calculates an angle in the range of θ1 to θ2. A dangerous estimated velocity vector having a scalar value equal to or larger than a set value is extracted.
Subsequently, the arithmetic processing unit 16 searches the table and finds that the estimated velocity vector existing in the fourth region space is θ3
A dangerous estimated speed vector having an angle in the range of θ4 and a scalar value equal to or larger than a set value is extracted.

If a dangerous estimated speed vector is extracted in step 109, the arithmetic processing section 16 supplies a signal indicating this to the alarm section 17 to perform an alarm process (11).
0). If no dangerous estimated speed vector is extracted in step 109, the process returns to step 101.

Next, the selection of the optimum value of the regularization parameter performed in step 103 of FIG. 6 will be described in detail below.

It is disclosed in Japanese Patent Application Laid-Open No. 9-297851, for example, that the selection of an appropriate regularization parameter α from the state around the operation pixel, that is, the state of the luminance distribution and the spatial frequency in the optical flow estimation is required. This is a known fact. In the optical flow estimation, the extraction sensitivity of the estimated velocity field can be controlled by selecting the regularization parameter α, and it can be considered that a function as a low-pass filter for disturbance control is provided.

The train forward abnormality detecting device according to the present invention detects an abnormal speed at which a direction can be identified in advance. It detects only the behavior speed of a dangerous person traveling in the normal direction on the track from the train to the FOE assumed forward in the traveling direction.

As described above, since the directionality of the estimated speed is limited, as the preprocessing of the optical flow estimation, the edge intensity and the edge direction of the calculation target pixel in the first image in the time series are grasped by the first derivative, and the edge direction is determined. If it is parallel to the line direction or included in a fixed angle range centered on the parallel, a large value is set in the regularized parameter reference table arranged corresponding to the calculation target pixel. Depending on the edge direction, an extremely large value is set in the regularization parameter reference table. In short, in the pixel whose density gradient matches the moving direction that the moving object considers dangerous in the optical flow estimation calculation, the regularization parameter α
Is an optimum value and gives a large value at other angles.

Since an optical flow is selected even in the background as the train travels, it is necessary to eliminate the influence of the optical flow generated in the train traveling direction in order to estimate the movement vector of the object. Furthermore, on crowded homes, a large number of people are moving in free directions, so that the estimated speed vector that appears on the entire screen is more complicated.

In step 103 of selecting the optimum value of the regularization parameter, the sensitivity is increased only in the dangerous directional component to be detected in the optical flow estimation calculation process in consideration of such a situation in the field. Naturally, the vector in the unnecessary direction is excluded by the angle filter function from the estimated vector extracted after the optical flow estimation.

The first derivative is based on the luminance value of 3 × 3 pixels.
This is an arithmetic process for calculating the gradient of the center point. This temporary differentiation processing is also called gradient processing, and the direction of the edge is recognized by this processing. Pixel position in image
When the (coordinates) are known, the vector in the direction perpendicular to the line direction corresponding to the (coordinates) can be easily calculated, so that the most suitable regularization parameter value α can be determined.

In the process of step 103, the amount of displacement (shift) of the background information occurring between successive image frames is replaced with the instantaneous velocity field vector information of the optical flow by the gradient method, and the amount of background movement is calculated using the direction data. , The background information between consecutive image frames is matched to perform background movement correction, and the estimated speed of a moving object that is actually present between consecutive image frames after the background movement correction is obtained.
This overcomes the difficulty of image processing of background difference and time difference when background information always moves for each image frame.

[0053]

According to the train forward abnormality detecting device using the optical flow according to the present invention, it is possible to detect from the vehicle a passenger having an abnormal movement that may fall off the platform not only on a straight road but also on a curve. It became so. Therefore, it is possible to take a quicker response on the vehicle than when a conventional abnormally-located train-front abnormality detecting device detects a passenger having an abnormal movement that may fall from the platform.

[Brief description of the drawings]

FIG. 1 is a block diagram of a train forward abnormality detection device according to an embodiment of the present invention.

FIG. 2 is a diagram showing an optical flow radially diverging from one point FOE on an image.

FIG. 3 is a diagram showing four regions obtained by dividing a divergent velocity field, that is, a first region, a second region, a third region, and a fourth region.

FIG. 4 is a diagram showing a regularization table.

FIG. 5 is a diagram showing an angle sequence table of an optical flow.

FIG. 6 is a flowchart showing a flow of an operation of the train forward abnormality detection device according to the embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Surveillance camera 12 A / D conversion part 13 Time series image accumulation part 14 First image primary differentiation part 15 Regularization parameter table reference part 16 Arithmetic processing part 17 Alarm part

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Kenichiro Nakamura 3-13-8 Kanda Jimbocho, Chiyoda-ku, Tokyo F-Term Co., Ltd. F-term (reference) 5B057 AA16 BA11 CE02 CH01 CH01 CH07 DA06 DA15 5H161 AA01 MM05 MM15 NN10 NN12

Claims (3)

[Claims] 1. An ITV camera mounted near a driver's seat of a train for monitoring a train traveling direction, an image processing means for calculating an estimated optical flow speed extracted from a time-series image taken by the ITV camera, and the image processing Estimated speed selecting means for selecting an optical flow estimated speed of a predetermined angle width including a direction perpendicular to the train traveling direction or a direction perpendicular to the train traveling direction from the optical flow estimated speed calculated by the means,
Determining means for determining whether there is a moving object that can be clearly determined to be dangerous based on the estimated optical flow speed at an angle falling within the dangerous direction angle width with respect to the train traveling direction selected by the estimated speed selecting means. Train forward abnormality detection device using the obtained optical flow. 2. The image processing means according to claim 1, wherein said moving amount of background information occurring between continuous frames is replaced with instantaneous velocity field vector information of an optical flow by a gradient method, and a background moving amount is obtained by using said direction data. And means for performing background movement correction by matching background information between consecutive frames and obtaining an estimated speed of a moving object that is actually present between consecutive frames after the background movement correction. A train forward abnormality detection device using the optical flow of No. 1. 3. The image processing means includes: an arithmetic processing means for performing a first-order differentiation process on a first image of two consecutive frames; and a first-order differentiation value corresponding to a pixel to be operated, and Regularization parameter selecting means for selecting an optimal regularization parameter, and optical flow calculation for obtaining a gradient method optical flow by performing image processing on the first image and the second image of two consecutive frames using the optimal regularization parameter 2. A train forward abnormality detecting device using an optical flow according to claim 1, wherein the train forward abnormality detecting device comprises: