JP4753320B2 - Escalator monitoring system - Google Patents

Description

  The present invention relates to an escalator monitoring system that monitors a passenger who uses an escalator using a camera.

  An escalator (also called a man conveyor) installed in a building or a station is usually automatically operated in a place where there is no supervisor. For this reason, there is a possibility that the passenger may overlook it even if it is in a dangerous state such as going out from the handrail. In addition, even when a passenger falls, there is a problem that it takes time for the communication to enter the monitoring staff, and the response is delayed.

  Conventionally, in order to solve such a problem, there is an escalator monitoring system in which a camera is installed near an escalator, and an image taken by the camera is used to detect a behavior such as a passenger entering or going backward. It has been proposed (see, for example, Patent Documents 1 and 2).

This type of escalator monitoring system is generally based on person detection by monocular image processing using a single camera. That is, a two-dimensional image photographed by one camera is analyzed and a passenger who uses an escalator is detected.
JP 2000-34088 A Japanese Patent Laid-Open No. 5-319762

  The escalator is a vehicle having a very long depth unlike an elevator or the like, and therefore information on the depth is necessary for monitoring using a camera. However, since the conventional escalator monitoring system monitors using one camera as described above, it is difficult to obtain detailed information about the depth from a two-dimensional image captured by the camera. There is a problem that passenger detection accuracy is very low.

  That is, on a two-dimensional image, the escalator and passengers can only be captured on a flat surface. For example, if there are passengers near the handrail of the escalator, the passenger may go outside the handrail depending on the shooting angle of the camera. It may not be possible to correctly determine whether the vehicle is on the inside of the handrail.

  In addition, even when a passenger falls, there is a problem that the response is delayed because the location cannot be accurately specified in the depth direction of the escalator.

  In addition, if the passengers and passengers overlap each other during congestion (this is called occlusion), it is difficult to accurately detect individual passengers from the captured images, and they can be overlooked even if there is an accident. There is sex.

  The present invention has been made in view of the above points, and by using a camera for monitoring, it is possible to accurately detect a passenger using an escalator, and to alert the dangerous behavior of the passenger by issuing an alarm. An object of the present invention is to provide an escalator monitoring system that can be prevented.

  The escalator monitoring system according to the present invention includes at least two cameras installed in a place where the entire escalator can be monitored, and three-dimensional information for acquiring three-dimensional information from images captured by the two cameras. An acquisition means and a passenger detection means for analyzing the three-dimensional information obtained by the three-dimensional information acquisition means and detecting the position of the passenger using the escalator including the depth direction of the imaging range. Features.

  The escalator monitoring system according to the present invention further includes behavior detecting means for detecting the behavior of the passenger detected by the passenger detecting means based on the operation information of the escalator.

  The behavior detection means performs “embarkation detection”. That is, the operation information of the escalator includes information related to a preset danger area, and the behavior detection means is provided on the escalator when the passenger detected by the passenger detection means is in the danger area. It is characterized in that a state where a predetermined amount or more is struck outward from the handrail.

  The behavior detection means performs “overtaking detection”. That is, the operation information of the escalator includes information related to the driving direction of the escalator, and the behavior detection unit is configured to move forward while the passenger detected by the passenger detection unit moves in the driving direction of the escalator at a predetermined speed or more. It is characterized by detecting a state overtaking a passenger.

  The behavior detection means performs “pass-through detection”. That is, the operation information of the escalator includes information related to the driving direction of the escalator, and the behavior detection unit is configured to move forward while the passenger detected by the passenger detection unit moves in the driving direction of the escalator at a predetermined speed or more. It is characterized by detecting a state of passing between passengers.

  The behavior detecting means performs “falling detection”. That is, the operation information of the escalator includes information related to the driving direction of the escalator, and the behavior detecting means is a height of the head when the passenger detected by the passenger detecting means is moving in the driving direction of the escalator. It is characterized by detecting a fall state from the time fluctuation.

  The behavior detection means performs “stay detection”. That is, the operation information of the escalator includes information related to a preset danger area, and the behavior detection means is configured such that the passenger detected by the passenger detection means stays in the danger area of the escalator for a predetermined time or more. It is characterized by detecting a state of being.

  The behavior detection means performs “stay detection (including the number of people staying)”. That is, the behavior detecting means detects the number of passengers staying in the escalator's staying danger area for a predetermined time or more.

  The behavior detection means performs “tamper detection”. That is, the operation information of the escalator includes information related to a preset mischievous attention area, and the behavior detecting means stops the passenger detected by the passenger detecting means within a mischievous attention area of the escalator for a predetermined time or more. It is characterized by detecting a state of being.

  The behavior detecting means performs “tampering detection (including child determination)”. That is, the behavior detecting means detects a state where the head height of the passenger detected by the passenger detecting means is stopped for a predetermined time or more in the mischievous attention area of the escalator. It is characterized by doing.

  The escalator monitoring system according to the present invention further includes warning means for performing warning processing according to the behavior of the passenger detected by the behavior detection means. This warning means includes changing the warning level according to the behavior of the passenger detected by the behavior detection means and issuing a warning.

  Further, the escalator monitoring system according to the present invention is based on the time-series information of the position of the passenger detected by the passenger detection means and the operation information of the escalator, the moving speed, acceleration, moving direction, moving flow line of the passenger. An information measuring means for measuring information on the existence time; and a boarding / alighting state detecting means for detecting at least one of a boarding / alighting time, a boarding / alighting time, a residence time, and a moving state based on a measurement result of the information measuring means. It is characterized by that.

  ADVANTAGE OF THE INVENTION According to this invention, the passenger who utilizes an escalator can be detected correctly, and an accident can be prevented beforehand by issuing a warning about the dangerous behavior of the passenger.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of an escalator monitoring system according to an embodiment of the present invention. 11 in the figure indicates an escalator to be monitored.

  The escalator 11 includes a plurality of steps 12 and a pair of balustrade panels 13 installed on both sides of these steps 12. Each step 12 is a stepped pedestal formed of, for example, aluminum die cast, and circulates between the entrances and exits while being supported by a truss (not shown) with a predetermined inclination angle.

  The pair of balustrade panels 13 are formed of, for example, transparent glass or acrylic, and a handrail belt 14 whose periphery is covered with rubber or the like is wound around the periphery of the balustrade panel 13. The handrail belt 14 moves in synchronization with each step 12 described above.

  Furthermore, a triangular guard plate 15 is attached to the lower end of the ceiling or the like that intersects the escalator 11. The triangular guard plate 15 is for preventing an accident that is caught in a gap with the ceiling when the passenger 10 gets out of the handrail belt 14.

  Here, as a camera for monitoring, the stereo camera unit 21 is installed in a place where the entire escalator 11 can be monitored (for example, a ceiling surface in the vicinity of the escalator 11). In this stereo camera unit 21, two cameras 22 a and 22 b are arranged with a predetermined interval, and each image captured by these cameras 22 a and 22 b is transmitted through the transmission cable 23 to an image processing apparatus. 24 are sequentially transferred.

  The image processing device 24 is a main component of this system. The image processing device 24 is configured by a general PC (personal computer) or an image processing board provided with a CPU, a ROM, a RAM, and the like. The image processing device 24 detects the position of the passenger 10 using the escalator 11 including the depth direction of the imaging range by analyzing each image captured by the cameras 22a and 22b in a three-dimensional manner. A function of detecting the behavior of the passenger 10 is provided.

  The control device 25 controls the operation of the escalator 11 and is composed of a PC different from the image processing device 24. The control device 25 receives various signals output from the image processing device 24 and controls the speaker 26 and the display device 27 and controls the driving device 28 of the escalator 11.

  The speaker 26 is installed in the vicinity of the escalator 11 and is used when a warning is given to the passenger 10. The display device 27 is installed in a monitoring room or the like, and performs monitor display of captured images, warning level display, and the like.

  FIG. 2 is a block diagram showing a functional configuration of the image processing device 24 provided in the escalator monitoring system.

  Functionally, the configuration of the image processing device 24 of the present system is shown as follows: a capture processing unit 31, a three-dimensional information acquisition unit 32, a passenger detection unit 33, a behavior detection unit 36, a warning unit 37, a measurement unit 38, and a boarding / alighting state detection unit. 39.

  The capture processing unit 31 captures each image continuously captured by the two cameras 22 a and 22 b provided in the stereo camera unit 21. Each captured image is stored in an image memory (not shown) in the image processing device 24 together with time information.

  The three-dimensional information acquisition unit 32 acquires three-dimensional information (x, y, z) of the shooting range from each image captured by the capture processing unit 31.

  The passenger detection unit 33 analyzes the three-dimensional information obtained by the three-dimensional information acquisition unit 32 and detects the position of the passenger 10 using the escalator 11 including the depth direction of the imaging range. The passenger detection unit 33 includes a person extraction unit 34 and a person tracking unit 35. The person extraction unit 34 analyzes the distribution state of the three-dimensional information (x, y, z) obtained by the three-dimensional information acquisition unit 32 and extracts the passenger 10 who uses the escalator 11 from the captured image. The person tracking unit 35 tracks the passenger 10 extracted by the person extracting unit 33 between the images.

  The behavior detection unit 36 detects the behavior of the passenger 10 detected by the passenger detection unit 33 based on the operation information of the escalator 11. Examples of the behavior include “embarkation”, “passing”, “passing through”, “falling”, “staying”, and “mischievous”. In addition to the driving direction, driving speed, and the like of the escalator 11, the operation information includes information related to a loading danger area, a staying danger area, a mischievous attention area, and the like, which will be described later.

  The warning unit 37 performs warning processing by voice or display according to the behavior of the passenger 10 determined by the behavior detection unit 36.

  Further, the information measuring unit 38 is based on the time-series information of the position of the passenger 10 detected by the passenger detecting unit 33 and the driving direction and driving speed obtained as the operation information of the escalator 11. Measures the direction of movement, movement line, and duration, and controls and detects at least one of the number of boarding / alighting times, boarding / alighting time, dwell time, and moving state (stop / run / walk / reverse run) Output to the device 25.

  The operation of this escalator monitoring system will be described in detail below.

  In addition, area determination, determination of distance, height, etc. including passenger detection described later are performed by analyzing each image three-dimensionally by stereo image processing using the two cameras 22a and 22b described above. .

(Passenger detection)
First, “passenger detection” by the image processing device 24 of the present system will be described by focusing on one passenger 10 who uses the escalator 11.

  As shown in FIG. 2, the image processing device 24 used in this system sequentially caps each image continuously captured by the two cameras 22a and 22b, and from these images in the x direction of the capturing range. , Y direction, z direction three-dimensional information is acquired. Then, the distribution state of the three-dimensional information is analyzed, and a portion corresponding to the passenger 10 is extracted from the captured image.

  FIG. 3 shows a distribution example of the three-dimensional information (x, y, z). A black circle in the figure indicates the position of the three-dimensional information obtained for each photographing. From such a three-dimensional information distribution state, a mass close to a human shape is extracted as a portion corresponding to the passenger 10.

  Subsequently, the image processing device 24 tracks the extracted passenger 10 between the images obtained in time series. Specifically, as shown in FIG. 4, a person extracted from an image captured at time T0 and a person extracted from an image captured at time T1 are matched using feature quantities such as three-dimensional information. Process, identify a combination with a high degree of correlation as the same person, and track that person.

  By such stereo image processing using the two cameras 22a and 22b, the position of the passenger 10 can be accurately detected including the depth direction of the imaging range. Moreover, although it demonstrated paying attention to the one passenger 10 here, even if it is in the state where many passengers 10 are using the escalator 11, the position of each passenger 10 is correctly detected by the said stereo image processing. It is possible.

(Measurement of various information and detection of boarding / exiting status)
Next, measurement of various information and detection of the boarding / alighting state by the image processing apparatus 24 of the present system will be described.

  As shown in FIG. 5, the time-series information (x0, y0, z0, t0) of the position of the passenger 10 obtained by the passenger detection process (xn, yn, zn, tn) and the operation information of the escalator 11 Based on the driving direction De and the driving speed Ve obtained, information such as the moving speed, acceleration, moving direction, moving flow line, and existence time of the passenger 10 can be measured. In addition, a movement flow line is obtained by connecting the time series information of the position of the passenger 10 with a line. L1 in the figure indicates a movement flow line.

  Further, based on these pieces of information, it is possible to detect a state related to a boarding / alighting operation such as the number of boarding / alighting times, a boarding / alighting time, a residence time, and a moving state (stop / run / walk / reverse run) as follows.

(1) Number of getting on and off The number of getting on and off is the number of passengers 10 getting on and off the escalator 11 per unit time. This is obtained by counting the number of people who have passed through the entrance and exit of the escalator 11 during the time t0 to tn.

(2) Boarding / alighting time Boarding / alighting time is the time which the passenger 10 passed from the entrance of the escalator 11 to the exit as shown in FIG. In this case, when the movement flow line L1 of the passenger 10 and the entrance line L2 of the escalator 11 intersect, it is determined that the passenger 10 has passed the entrance of the escalator 11. Further, when the movement flow line L1 of the passenger 10 and the exit line L3 of the escalator 11 intersect, it is determined that the passenger 10 has passed the exit of the escalator 11.

  The time when the passenger 10 passes through the entrance is the time over the entrance line L2. In the example of FIG. 6, the time tb is obtained as the passage time of the entrance because it spans between the time ta and the time tb. Similarly, the time when the passenger 10 passes through the exit gate is a time spanning the exit line L3. In the example of FIG. 6, the time td is obtained as the passage time of the exit because the time spans between the time tc and the time td. Thereby, the boarding / alighting time is calculated | required from "td-tb".

(3) Residence time The residence time is the time during which the passenger 10 stops at the same place. In this case, a state where the time variation of the position (x, y) of the passenger 10 is small is regarded as a staying state. Therefore, when the time t1 when the passenger 10 starts to stop and the time t2 when the passenger 10 starts to move are set, the residence time is obtained from “t2−t1”.

(4) Moving state The moving state is a state in which the passenger 10 is moving, and includes stop / run / walk / reverse run. As shown in FIG. 6, the moving speed Vp of the passenger 10 is obtained from the moving time and distance from time tb to time td. If the moving speed Vp is the same as the operating speed Ve of the escalator 11 (Vp = Ve), it is determined that the passenger 10 is stopped on the step 12 of the escalator 11, that is, is in a stopped state.

  On the other hand, if the moving speed Vp of the passenger 10 is faster than the driving speed Ve of the escalator 11 and is within the walking reference value V0 (Ve <Vp ≦ V0), the passenger 10 is walking on the step 12 of the escalator 11. It is determined that the state is a walking state. When the moving speed Vp of the passenger 10 is faster than the walking reference value V0 (V0 <Vp), it is determined that the passenger 10 is running on the step 12 of the escalator 11, that is, the running state.

  Further, when the moving direction of the passenger 10 is opposite to the driving direction De of the escalator 11, the passenger 10 is moving in the reverse direction on the step 12 of the escalator 11, that is, in the reverse running state. It is judged that.

  Note that the above is for the passenger 10 focusing on one person, but the same applies even when a large number of passengers 10 use the escalator 11, and each passenger 10 is detected individually by the stereo image processing described above. By doing, the information regarding each passenger 10 can be obtained.

  Furthermore, as shown in FIG. 7, it is possible to detect the number of people who are moving on the escalator 11 and the number of people waiting at the entrance of the escalator 11 using the information about each passenger 10.

(Start-up detection)
Next, “embarkation detection”, which is one of behavior detection by the image processing apparatus 24 of this system, will be described.

  The “embarkation detection” is to detect a state in which the passenger 10 embarks outward from the handrail belt 14 of the escalator 11.

  Now, as shown in FIG. 8, the distance from the handrail belt 14 to the triangular guard plate 15 is a, the boarding amount of the passenger 10 is b, and the distance from the passenger 10 to the triangular guard plate 15 is c. Then, the time until the passenger 10 is sandwiched between the escalator 11 intersections can be calculated from the distance c and the driving speed Ve.

  When the passenger 10 has been detected, a warning to that effect is given. At this time, the warning level may be changed as follows from the relationship between the distance a, the amount b, and the distance c.

b <a⇒Caution b ≧ a⇒Warning c ≧ Stoppable distance⇒Caution c <Stoppable distance⇒Warning.

  In addition, it is possible to determine which part of the passenger 10 is on board or the luggage from the three-dimensional shape. At that time, the warning level may be changed as follows.

The loading site is luggage ⇒ Caution The loading site is arm / head ⇒ Warning.

  Further, it can be determined from the position of the passenger 10 and the moving speed whether the image of the lump shape in the vicinity of the handrail belt 14 is caused by the passenger 10 getting on or whether noise is reflected. In this case, the person 10 'existing behind the escalator 11 can be cited as noise.

  As shown in FIG. 9, in the monocular image, it cannot be distinguished whether the lump shape near the handrail belt 14 is due to the passenger 10 getting on or the reflection of the person 10 ′ existing behind the escalator 11. On the other hand, in the present system, it is possible to accurately detect that the person 10 ′ is behind the escalator 11 based on the three-dimensional information (x, y, z) obtained from each image.

  In the following, the “operation detection” processing operation of this system will be described in detail with reference to FIG.

  FIG. 10 is a flowchart showing the processing operation of “embarkation detection” by the image processing apparatus 24 of the present system. Note that the processing shown in this flowchart is executed when a CPU (not shown) mounted in the image processing device 24 reads a predetermined program stored in a memory such as a ROM.

  First, as an initialization process, the positional relationship between the stereo camera unit 21 (cameras 22a and 22b) and the escalator 11 is set in the image processing device 24, and information regarding the risk of entering the area 51 is set in the image processing device 24. (Step A11). As illustrated in FIG. 8, the unloading danger area 51 is an area up to and slightly before the intersection of the escalator 11 to which the triangular guard plate 15 is attached.

  When the image processing device 24 detects the passenger 10 using the escalator 11 by analyzing each image taken by the cameras 22a and 22b (step A12), the image processing device 24 obtains the distance from the intersection of the escalator 11 to the passenger 10. (Step A13). As a result, when the distance from the intersection of the escalator 11 to the passenger 10 is longer than a preset threshold value (No in step A14), the image processing apparatus 24 has entered the danger area 51 where the passenger 10 has entered. Judge that there is nothing, do nothing in particular, continue monitoring.

  On the other hand, when the distance from the intersection of the escalator 11 to the passenger 10 is equal to or less than a preset threshold value (Yes in step A14), the image processing device 24 causes the passenger 10 to enter the dangerous area 51. The amount of boarding of the passenger 10 in the boarding danger area 51 is determined (step A15). As a result, if the amount of boarding is equal to or greater than a preset threshold value (Yes in Step A16), the image processing device 24 performs a warning process for alerting the passenger 10 (Step A17).

  Specifically, by outputting a warning signal from the image processing device 24 to the control device 25, for example, a message such as “Don't get out because it is dangerous” is output through the speaker 26. Note that the warning method is not limited to voice, and for example, a display (not shown) may be installed near the intersection of the escalator 11 and a warning message may be displayed on the display.

  Further, the warning level may be changed stepwise according to the distance to the intersection and the amount of boarding, or the operation of the escalator 11 may be decelerated or stopped.

  Here, although it has been described that a warning is given when the passenger 10 gets out of the handrail belt 14 by a predetermined amount or more in the boarding danger area 51, the passenger 10 does not care about the boarding danger area 51. A warning may be issued when a predetermined amount or more is started.

  In addition, when the passenger 10 is detected as starting, the detection signal may be output to the control device 25 to temporarily reduce the operation speed of the escalator 11 or to stop the operation.

(Overtaking / passing through detection)
Next, “passing / passing through detection” which is one of behavior detections by the image processing apparatus 24 of the present system will be described.

  FIG. 11 is a diagram illustrating an example of overtaking. Pa, Pb, and Pc in the figure indicate passengers on the escalator 11. The overtaking detection is performed by monitoring the arrangement order of these passengers Pa, Pb, and Pc in time series.

  That is, it is assumed that the arrangement order on the escalator 11 changes with time as follows. By analyzing the captured images obtained in time series from the cameras 22a and 22b for such a change in arrangement order, it is possible to detect that the passenger Pc has overtaken the passenger Pb at time T1 and has overtaken the passenger Pa at time T2.

Time T0: Pa-Pb-Pc
Time T1: Pa-Pc-Pb
Time T2: Pc-Pa-Pb.

  FIG. 12 is a diagram showing an example of slipping through. Pa, Pb, Pc, and Pd in the figure indicate passengers on the escalator 11. As with passing, slip-through can be detected by monitoring the order of these passengers in time series.

  That is, it is assumed that the arrangement order on the escalator 11 changes with time as follows. By analyzing the captured images obtained in time series from the cameras 22a and 22b for such a change in arrangement order, it is possible to detect that the passenger Pc has passed between the passengers Pa and Pd at time T2.

Time T0: (Pa, Pd) -Pb-Pc
Time T1: (Pa, Pd) -Pc-Pb
Time T2: Pc- (Pa, Pd) -Pb.

  In addition, as an index for determining overtaking / passing, a movement flow line may be used as an index in addition to the order in which the passengers are arranged.

  In the following, the processing operation of “overtaking detection” of the modified system will be described in detail with reference to FIG.

  FIG. 13 is a flowchart showing the processing operation of “overtaking detection” by the image processing apparatus 24 of the present system. Note that the processing shown in this flowchart is executed when a CPU (not shown) mounted in the image processing device 24 reads a predetermined program stored in a memory such as a ROM.

  First, as initialization processing, the positional relationship between the stereo camera unit 21 (cameras 22a and 22b) and the escalator 11 is set in the image processing device 24, and information on the driving direction De and the driving speed Ve of the escalator 11 is imaged. It is set in the processing device 24 (step B11).

  When the image processing device 24 detects the passenger 10 using the escalator 11 by analyzing each image captured by the cameras 22a and 22b (step B12), is the passenger 10 stopped or walking? Is determined (step B13).

  In this case, the moving speed of the passenger 10 is obtained from a plurality of photographed images, and if the moving speed is the same as the driving speed Ve of the escalator 11, it is determined that the vehicle is stopped on step 12. If the moving speed of the passenger 10 is faster than the driving speed Ve of the escalator 11, it is determined that the passenger 10 is walking (or running) in the driving direction De of the escalator 11. When it is determined that the passenger 10 is stopped on Step 12 (No in Step B14), the image processing device 24 does not do anything and continues monitoring.

  On the other hand, when it is determined that the passenger 10 is walking (or running) (Yes in Step B14), the image processing device 24 calculates a relative position with other passengers existing on the escalator 11. The arrangement order of the passengers 10 is obtained from the relative position (step B15). Then, the image processing device 24 monitors the order of the passengers 10 in time series, determines that there is an overtaking when the passengers are replaced with the front passengers (Yes in step B16), and pays attention to the passengers 10. A warning process for prompting is performed (step B17).

  Specifically, by outputting a warning signal from the image processing device 24 to the control device 25, a message such as “Do n’t walk on the escalator because it is dangerous” is output through the speaker 26 as a voice. The passenger 10 is alerted by voice through the speaker 26. Note that the warning method is not limited to voice, and for example, a display (not shown) may be installed near the intersection of the escalator 11 and a warning message may be displayed on the display.

  In addition, although it demonstrated supposing overtaking here, also in the case of passing-through, it can determine similarly from the moving speed and arrangement order of the focused passenger. However, as shown in the flowchart of FIG. 14, it is determined that the passenger has passed through (Step C11) on the condition that the passenger has passed between the passengers ahead and the passenger (Step C11).

  Further, when it is detected that the passenger 10 has passed or passed, the detection signal may be output to the control device 25 to temporarily reduce the operation speed of the escalator 11 or to stop the operation.

(Fall detection)
Next, “falling detection” which is one of behavior detection by the image processing apparatus 24 of the present system will be described.

  “Falling detection” is performed by monitoring the height fluctuation of the passenger's head position. That is, as shown in FIG. 15, for example, when the head height of the passenger 10 changes from 1.75 m to 0.30 m in T0 to T3, which is a short time, it is regarded as a fall.

  In addition, with the addition of time elements, the height of 10 passengers at the start of detection is 1.75 m, the head height is 0.30 m from a certain time, and the head height of 0.30 m continues for a predetermined time or more. If you do, you can consider it a fall.

  The processing operation of “falling detection” of this system will be described in detail below with reference to FIG.

  FIG. 16 is a flowchart showing a fall detection processing operation by the image processing apparatus 24 of the present system. Note that the processing shown in this flowchart is executed when a CPU (not shown) mounted in the image processing device 24 reads a predetermined program stored in a memory such as a ROM.

  First, as initialization processing, the positional relationship between the stereo camera unit 21 (cameras 22a and 22b) and the escalator 11 is set in the image processing device 24, and information on the driving direction De and the driving speed Ve of the escalator 11 is imaged. It is set in the processing device 24 (step D11).

  When the image processing device 24 detects the passenger 10 using the escalator 11 by analyzing each image taken by the cameras 22a and 22b (step D12), the image processing device 24 is based on the driving direction De and the driving speed Ve of the escalator 11. Time fluctuation of the head height of the passenger 10 is obtained (step D13). As a result, if the temporal variation in the head height of the passenger 10 is equal to or less than the threshold value (No in Step D14), the image processing device 24 does not do anything and continues monitoring as it is.

  On the other hand, when the temporal variation of the head height of the passenger 10 exceeds the threshold value, that is, when the variation is large (Yes in step D14), the image processing device 24 determines that the passenger 10 has fallen. Then, a warning process is performed to call attention to surrounding people and monitoring personnel (step D15).

  Specifically, by outputting a warning signal from the image processing device 24 to the control device 25, a message such as “There is a person who has fallen” is output through the speaker 26 as a voice. The passenger 10 is alerted by voice through the speaker 26. Note that the warning method is not limited to voice, and for example, a display (not shown) may be installed near the intersection of the escalator 11 and a warning message may be displayed on the display.

  Further, for example, a warning message such as “There is a person who has fallen. Operation is stopped for safety” may be output to stop the operation of the escalator 11 through the control device 25.

(Studging detection)
Next, “stay detection” which is one of behavior detection by the image processing apparatus 24 of the present system will be described.

  As shown in FIG. 17, “stay detection” is performed by monitoring the movement of the passenger 10 near the entrance / exit of the escalator 11. That is, it is determined that the vehicle is staying when the passenger 10 has stopped for a certain period of time in the vicinity of the entrance / exit. At that time, if the number of people staying is too large to pass through the entrance / exit, a warning is given.

  The processing operation of “falling detection” of this system will be described in detail below with reference to FIG.

  FIG. 18 is a flowchart showing a fall detection processing operation by the image processing apparatus 24 of the present system. Note that the processing shown in this flowchart is executed when a CPU (not shown) mounted in the image processing device 24 reads a predetermined program stored in a memory such as a ROM.

  First, as an initialization process, the positional relationship between the stereo camera unit 21 (cameras 22a and 22b) and the escalator 11 is set in the image processing device 24, and the vicinity of the entrance / exit of the escalator 11 is set as the stay danger areas 52a and 52b. It is set in the image processing device 24 (step E11).

  When the image processing device 24 detects the passenger 10 using the escalator 11 by analyzing each image taken by the cameras 22a and 22b (step E12), whether or not the passenger 10 is in the stay danger area 52a or 52b. Is determined (step E13). As a result, when the passenger 10 is not in either of the stay danger areas 52a and 52b (No in step E14), the image processing device 24 does not do anything and continues monitoring.

  On the other hand, when the passenger 10 is in the stay danger area 52a or 52b (Yes in Step E14), the image processing device 24 obtains the moving speed of the passenger 10 and determines whether or not the passenger 10 is stopped (Step). E15). As a result, when the moving speed of the passenger 10 is slower than the predetermined speed (Yes in Step E16), the image processing device 24 determines that the passenger 10 is stopped and monitors the time there (Step E17). ). Thereby, when the state where the passenger 10 has stopped for a predetermined time or more is detected (Yes in Step E18), the image processing device 24 determines that the passenger 10 is staying near the entrance / exit.

  Here, the staying state of other passengers 10 is similarly checked (step E19). If the number of staying persons at the same place is equal to or larger than a predetermined number (Yes in step E20), the image processing device 24 Warning processing is performed to call the passenger 10 attention to those who are determined to be dangerous and stop there (step E21).

  The predetermined number in step E20 is the number of people that a person cannot easily pass through. For example, in the case of an escalator 11 in which two people can ride in one step, when two or more people stay in the same place. A warning shall be issued.

  Specifically, by outputting a warning signal from the image processing device 24 to the control device 25, for example, a message such as “It is dangerous to stop at the entrance / exit” is output through the speaker 26. The passenger 10 is alerted by voice through the speaker 26. Note that the warning method is not limited to voice, and for example, a display (not shown) may be installed near the intersection of the escalator 11 and a warning message may be displayed on the display.

  In addition, although it has been described here that a warning is issued when a predetermined number or more stays in the stay danger area (near the entrance / exit), it stays in the stay danger area (near the entrance / exit) regardless of the number of people. A warning may be issued when the state is detected, or the warning level may be changed according to the number of people.

  In addition, although the vicinity of the entrance / exit is set as a stay danger area here, other places can also be set as the stay danger area, and it is also possible to detect the stay there and issue a warning in the same manner as described above. .

  Moreover, when the stay of the passenger 10 is detected, the detection signal may be output to the control device 25 to temporarily reduce the operation speed of the escalator 11 or to stop the operation.

(Prank detection)
Next, “tamper detection” which is one of behavior detection by the image processing apparatus 24 of this system will be described.

  The “tamper detection” is targeted at mischief near the inlet of the escalator 11 (the entrance of the handrail belt). Further, a person to be detected is a small child. That is, as shown in FIG. 19, when four places near the inlet of the escalator 11 are defined as mischievous attention areas 53a, 53b, 53c, and 53d, and a passenger 10 whose head height is equal to or less than a threshold value has stopped for a predetermined time or more. It is determined that the child is pranking and a warning is given.

  The processing operation of “tamper detection” of this system will be described in detail below with reference to FIG.

  FIG. 20 is a flowchart showing the processing operation of “tamper detection” by the image processing apparatus 24 of the present system. Note that the processing shown in this flowchart is executed when a CPU (not shown) mounted in the image processing device 24 reads a predetermined program stored in a memory such as a ROM.

  First, as initialization processing, the positional relationship between the stereo camera unit 21 (cameras 22a, 22b) and the escalator 11 is set in the image processing device 24, and the mischievous attention areas 53a, 53b, 53c are located near the inlet of the escalator 11. , 53d are set in the image processing device 24 (step F11).

  When the image processing apparatus 24 detects the passenger 10 using the escalator 11 by analyzing each image taken by the cameras 22a and 22b (step F12), the passenger 10 is in a mischievous attention area 53a, 53b, 53c, It is determined whether or not any of 53d (step F13). As a result, when the passenger 10 is not in any of the mischievous attention areas 53a, 53b, 53c, and 53d (No in Step F14), the image processing apparatus 24 does not do anything and continues monitoring.

  On the other hand, when the passenger 10 is in any of the mischievous attention areas 53a, 53b, 53c, and 53d (Yes in Step F14), the image processing device 24 obtains the head height of the passenger 10 (Step F15). As a result, if the head height of the passenger 10 is equal to or less than the predetermined value (Yes in step F16), the image processing device 24 subsequently obtains the moving speed of the passenger 10 and determines whether or not the passenger 10 is stopped there. (Step F17).

  It is assumed that the predetermined value in step F16 is determined based on, for example, the average height of a child below elementary school age.

  When the moving speed of the passenger 10 is slower than the predetermined speed (Yes in Step F17), the image processing device 24 determines that the passenger 10 is stopped and monitors the time there (Step E17). Thereby, when the state where the passenger 10 has stopped for a predetermined time or more is detected (Yes in Step F18), the image processing device 24 determines that the child is pranking near the inlet (Step F19). ). In this way, when the state where the passenger 10 is mischievous is detected, the image processing device 24 performs a warning process for alerting the passenger 10 (step F21).

  Specifically, by outputting a warning signal from the image processing device 24 to the control device 25, for example, a message such as “Dangerous, please do not approach the entrance of the handrail belt” through the speaker 26. Output audio. The passenger 10 is alerted by voice through the speaker 26. Note that the warning method is not limited to voice, and for example, a display (not shown) may be installed near the intersection of the escalator 11 and a warning message may be displayed on the display.

  Here, although it has been described that tampering detection is performed for a child, a warning may be issued when it is detected that the child has stopped in a mischievous attention area (near the inlet) regardless of the child.

  Also, if there are adults around you, you can try to be mischievous. The presence or absence of an adult in the surroundings can be determined by detecting another passenger 10 whose head height is equal to or greater than a predetermined value within a predetermined distance from the passenger 10 stopping in the mischievous attention area. However, even if there is an adult nearby, the vicinity of the inlet is dangerous, so it is safer to issue a warning immediately if there is a person who stops near the inlet regardless of the presence of an adult or a child. preferable.

  Although the vicinity of the inlet is set as a mischievous caution area here, it is also possible to set other places as mischievous caution areas and detect mischief there in the same manner as described above to issue a warning.

  Further, when it is detected that the passenger 10 is tampering with the mischievous attention area, the detection signal may be output to the control device 25 to stop the operation.

  As described above, according to the present system, by monitoring the escalator 11 using the two cameras 22a and 22b, the position of the passenger 10 can be determined without being greatly affected by the installation location of the cameras 22a and 22b. It is possible to accurately detect the depth direction. Therefore, for example, when the passenger 10 is near the handrail belt 14 of the escalator 11, it is correctly determined whether the passenger 10 is riding on the outside of the handrail belt 14 or on the inside of the handrail belt 14. And take appropriate action.

  Further, even when the passenger 10 falls, the location can be accurately identified in the depth direction of the escalator 11 and can be appropriately handled.

  Further, even in a heavily congested environment, that is, an environment in which occlusion occurs, the position of each passenger 10 can be accurately detected from the captured image, and can be verified in detail when an accident occurs.

  In addition, by monitoring the passenger 10 in conjunction with the operation information of the escalator 11, it is possible to detect in real time behaviors such as “embarkation”, “passing”, “passing through”, “falling”, “staying”, and “mischievous”. By issuing warnings for dangerous behaviors, accidents can be prevented.

  In the above embodiment, the escalator 11 is monitored using the two cameras 22a and 22b. However, the escalator 11 may be monitored using three or more cameras. However, if the number of cameras is increased, the installation equipment becomes oversized and the cost is increased, and further, it takes time for image processing. Therefore, monitoring is performed by the two cameras 22a and 22b as in the above embodiment. It is preferable to do.

  In the above embodiment, the image processing device 24 and the control device 25 are configured separately as shown in FIG. 1, but the functions of the image processing device 24 are installed in the control device 25 so that only the control device 25 can be used. It is also possible to configure.

  In short, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. Moreover, various forms can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be omitted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

FIG. 1 is a diagram showing a configuration of an escalator monitoring system according to an embodiment of the present invention. FIG. 2 is a block diagram showing a functional configuration of an image processing apparatus provided in the escalator monitoring system according to the embodiment. FIG. 3 is a diagram showing a distribution example of the three-dimensional information (x, y, z) obtained by the image processing apparatus of the escalator monitoring system. FIG. 4 is a diagram for explaining person tracking processing between images by the image processing apparatus of the escalator monitoring system. FIG. 5 is a diagram for explaining various types of information related to passengers getting on and off by the image processing apparatus of the escalator monitoring system. FIG. 6 is a diagram for explaining a method for detecting a boarding / alighting time as various types of information related to a passenger boarding / alighting operation by the image processing device of the escalator monitoring system. FIG. 7 is a diagram showing an example when a large number of passengers are detected by the image processing apparatus of the escalator monitoring system. FIG. 8 is a diagram for explaining “embarkation detection” by the image processing apparatus of the escalator monitoring system. FIG. 9 is a diagram for explaining a problem of “start-up detection” using a conventional monocular image. FIG. 10 is a flowchart showing the processing operation of “embarkation detection” by the image processing apparatus of the escalator monitoring system. FIG. 11 is a diagram for explaining “passing detection” by the image processing apparatus of the escalator monitoring system. FIG. 12 is a diagram for explaining “pass-through detection” by the image processing apparatus of the escalator monitoring system. FIG. 13 is a flowchart showing the processing operation of “overtaking detection” by the image processing apparatus of the escalator monitoring system. FIG. 14 is a flowchart showing a “pass-through detection” processing operation by the image processing apparatus of the escalator monitoring system. FIG. 15 is a diagram for explaining “falling detection” by the image processing apparatus of the escalator monitoring system. FIG. 16 is a flowchart showing the processing operation of “falling detection” by the image processing apparatus of the escalator monitoring system. FIG. 17 is a diagram for explaining “stay detection” by the image processing apparatus of the escalator monitoring system. FIG. 18 is a flowchart showing the processing operation of “stay detection” by the image processing apparatus of the escalator monitoring system. FIG. 19 is a diagram for explaining “tamper detection” by the image processing apparatus of the escalator monitoring system. FIG. 20 is a flowchart showing the processing operation of “tamper detection” by the image processing apparatus of the escalator monitoring system.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Escalator, 12 ... Step, 13 ... Railing panel, 14 ... Handrail belt, 15 ... Triangular guard plate, 21 ... Stereo camera unit, 22a, 22b ... Camera, 23 ... Transmission cable, 24 ... Image processing device, 25 ... Control device, 26 ... Speaker, 27 ... Display device, 28 ... Drive device, 31 ... Capture processing unit, 32 ... 3D information acquisition unit, 33 ... Passenger detection unit, 34 ... Human extraction unit, 35 ... Human tracking unit, 36 ... Behavior detection unit, 37 ... Warning unit, 38 ... Information measurement unit, 39 ... Boarding / alighting state detection unit, 51 ... Boarding danger area, 52a, 52b ... Retention danger area, 53a, 53b, 53c, 53d ... Mischievous caution area .

Claims (10)

At least two cameras installed in a place where the entire escalator can be monitored;
Three-dimensional information acquisition means for acquiring three-dimensional information from each image captured by the two cameras;
Passenger detection means for analyzing the three-dimensional information obtained by the three-dimensional information acquisition means and detecting the position of the passenger using the escalator including the depth direction of the imaging range;
A behavior detecting means for detecting the behavior of the passenger detected by the passenger detecting means based on the operation information of the escalator;
The operation information of the escalator includes information related to a preset danger area and information related to the driving direction of the escalator,
The behavior detection means is
The passenger detected by the passenger detection means detects a state in which a predetermined amount or more of the passenger detected from the handrail provided on the escalator is outside in the exit danger area, and the passenger detected by the passenger detection means An escalator monitoring system for detecting a state of overtaking a passenger in front while moving in the driving direction of the escalator at a predetermined speed or higher . The behavior detection means is
Escalator according to claim 1, wherein the detecting the state of passengers detected by the passenger detecting means is slipping through between the front passenger and the passenger while moving at a predetermined speed or more operating direction of the escalator Monitoring system. The behavior detection means is
Escalator monitoring system of claim 1, wherein the passenger detected by the passenger detecting means detects the state of a fall from a time variation of the head height when moving in the driving direction of the escalator. The operation information of the escalator includes information on a pre-set residence risk area,
The behavior detection means is
Escalator monitoring system of claim 1, wherein the detecting the state of passengers detected by the passenger detecting means is retained for a predetermined time or more in a residence danger area of the escalator. The behavior detection means is
The escalator monitoring system according to claim 4, wherein the number of passengers staying in the escalator's stay danger area for a predetermined time or more is detected. The escalator operation information includes information related to a preset mischievous attention area,
The behavior detecting means, escalator monitoring system of claim 1, wherein the detecting the state of passengers detected by the passenger detecting means is stopped for a predetermined time or more in mischief attention area of the escalator. The behavior detecting means detects a state in which the head height of the passenger detected by the passenger detecting means is stopped for a predetermined time or more within a mischievous attention area of the escalator, for a person whose head height is a predetermined value or less. The escalator monitoring system according to claim 6 . The escalator monitoring system according to any one of claims 1 to 7 , further comprising warning means for performing warning processing according to the behavior of the passenger detected by the behavior detection means. 9. The escalator monitoring system according to claim 8 , wherein the warning means performs warning by changing a warning level in accordance with the behavior of the passenger detected by the behavior detection means. Information measuring means for measuring the information on the moving speed, acceleration, moving direction, moving flow line, and existence time of the passenger based on the time series information of the position of the passenger detected by the passenger detecting means and the operation information of the escalator. When,
The escalator monitoring according to claim 1, further comprising: a boarding / alighting state detecting means for detecting at least one of a boarding / alighting time, a boarding / alighting time, a residence time, and a moving state based on a measurement result of the information measuring means. system.

Images