JP2009120327A - Escalator control device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an escalator control device improved in safety of a passenger or giving the passenger a feeling of safety. <P>SOLUTION: The escalator control device has an image processing device 107 processing images took by an image taking device 106 such as a camera. The image processing device 107 comprises: a nonstationary behavior determination part 111 detecting a nonstationary behavior of passenger, by processing the image; and an abnormal behavior classification part 112 classifying the behavior of the passenger, based on temporal change in detected nonstationary behavior. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an escalator control device suitable for ensuring the safety of passengers riding on an escalator or for giving passengers a sense of security.

  Conventionally, ultrasonic waves, infrared sensors, or image processing technology has been applied to detect the fall of passengers riding on escalators, boarding on handrails, or congestion on the lap, and sending alert broadcasts or stopping escalators. Have been proposed for ensuring the safety of passengers. In particular, the method using image processing has the advantage of building a system that can monitor a wide range with a single camera, and can record the video at the time of abnormality at the same time as abnormality detection, so it is not only as an abnormality detection sensor, but also details of abnormal events It can also be used for simple analysis and marketing purposes.

  As described in Japanese Patent Application Laid-Open No. 11-349269 (Patent Document 1), as a conventional technique for performing escalator control based on an image processing result, an abnormal state relating to passengers of a man conveyor is based on an image. There is a supervisory control means for making a determination corresponding to the section and instructing the control (stop, deceleration) of the man conveyor based on the abnormal state.

JP-A-11-349269

  In the case of the above-described conventional technology, a function of controlling the speed of the step according to the location where the passenger is abnormal is provided. However, it is impossible to determine the type of abnormal event of the passenger and control the escalator according to the type.

  For example, if a child or elderly passenger loses balance near the center of the upper and lower floors of the escalator, if clothes or shoes of the passenger are likely to be drawn into the gap between the lap and the step part, The escalator operation cannot be flexibly controlled in accordance with the abnormal operation of the passenger, such as when the child runs the escalator in a mischief or when trying to put an object outside the riding standard such as a stroller or a large suitcase.

  An object of this invention is to provide the escalator control apparatus which can improve a passenger | crew's safety or a sense of security.

  An escalator control device according to the present invention includes an image processing device that processes an image acquired by an image acquisition device such as a camera, and the image processing device detects an unsteady state of passenger movement by processing the image. A non-steady state operation determination unit, and an abnormal operation classification unit that classifies the passenger's movement based on the detected temporal change in the non-steady state.

  Preferably, the unsteady motion determination unit detects the unsteady state based on the unsteady degree calculated from the spatial change and temporal change of the brightness of the image, and the abnormal action classifying unit detects the temporal change of the unsteady degree. By classifying passenger behavior.

  According to the control device of the present invention, since the escalator can be controlled from the passenger image according to the form of the passenger's unsteady operation, the passenger's safety or security can be improved.

  Hereinafter, embodiments of the present invention will be described.

  FIG. 1 is a functional block diagram showing an overall configuration of an escalator control device according to an embodiment of the present invention. In FIG. 1, 101 is an escalator handrail, 102 is an escalator tread, 103 is a drive device such as an inverter that controls the operation of the escalator, 104 is a recording device that stores an image of the escalator, and 105 is a drive device 103. It is an interface for controlling the recording device 104. Reference numeral 106 denotes an escalator including the handrail 101 and the tread board 102, and an image acquisition apparatus that captures images of passengers moving on the tread board 102 and acquires those images. In this embodiment, a camera is used. The video captured by the camera 106 is sent to the recording device 104 through the interface 105.

  Reference numeral 107 denotes an image processing apparatus, which may be incorporated in the camera 106 or installed outside the camera 106. The image processing device 107 processes the image acquired by the camera 106 to detect an unsteady state, and sends the result to the driving device 103 through the interface 105. The driving device 103 controls the escalator operation according to the image processing result. I do. The recording device 104 performs an adaptive recording process such as recording when an unsteady state occurs.

  Note that the recording device 104 enables efficient recording by recording only when an unsteady operation occurs, and monitoring image recording can be performed even in a system with a small storage capacity. Or, by attaching a tag according to the type of abnormal operation when an unsteady occurrence occurs, it becomes possible to efficiently search and verify an accident event in the monitoring video. It can also be applied to customer marketing based on analysis of recorded video.

  Next, internal functions of the image processing apparatus 107 will be described. The video sent from the camera 106 is sent to the A / D converter 108 where it is converted from an analog signal to a digital signal. If the camera 106 sends a digital format video, the A / D converter 108 can be omitted. The video data is stored in the image memory 109. When image processing is not performed, the digital signal is converted into an analog signal by the D / A converter 110 and displayed on the external monitor 119. Alternatively, a digital signal is sent to the outside from the interface 118 in the image processing apparatus 107 and is recorded by, for example, the external recording apparatus 104. The interfaces 105 and 118 are wired connection or wireless connection, and both can be used simultaneously. By using the wireless connection, even when the recording device 104 cannot be installed in the vicinity of the escalator, all the camera images installed in the escalator on each floor can be collected on the disk server installed in the center.

  In order to process an escalator image and detect an abnormal event of a passenger, first, the captured image sent to the image memory 109 is sent to the unsteady operation determination unit 111. The unsteady motion determination unit 111 processes the image acquired by the camera 106 and detects the unsteady motion of the escalator passenger. Here, the steady operation means that the passenger is not on the escalator tread that is operating at a constant speed, or that the passenger is standing upright on the step, and the movement of the tread and the passenger It shows a state in which the movement is steady. On the other hand, the non-stationary operation indicates an operation that does not belong to the steady operation. For example, boarding on handrails, entry of non-standard items in escalator boarding regulations such as carts, strollers and suitcases, reverse running on the tread board, forward running, forward running down, tripping, falling and jumping, The shoes and clothes are drawn into the gap between the tread board and the rail, the falls in the wrapping part, and the shoes and clothes are drawn into the drawn part of the tread board part. When these abnormal movements occur, movements different from the steady movements of the treads and upright passengers occur, so that it is possible to detect the unsteady movements of the escalator passengers by detecting the disturbance of movement during the steady movements. However, even if an unsteady motion can be detected, the unsteady motion determination unit 111 cannot classify the type of the motion. Therefore, if an unsteady motion is detected by the unsteady motion determination unit 111, the process is passed to the abnormal motion classification unit 112 as the next step.

  In order to perform adaptive control of the escalator according to the case of the abnormal boarding of the passenger, the abnormal operation classification unit 112 performs classification of the non-stationary operation detected by the non-stationary operation determination unit 111. That is, the abnormal motion classification unit 112 classifies the passenger's motion based on the temporal change of the unsteady state detected by the unsteady motion determination unit 111. Intentionally abnormal boarding specifically includes boarding on handrails, entry of non-standard items in the escalator boarding regulations such as carts, strollers and suitcases, reverse running on escalator treads, forward running and forward running. It is going down or jumping, and passengers are trying to ride or try to ride out of the regulations. On the other hand, unintentional abnormal boarding means tripping or falling on the tread, pulling of shoes or clothes into the gap between the tread and rail, falling over in the lap or pulling of shoes or clothes into the pulling part of the tread This happens regardless of the will of the passengers.

  The alerting unit 113 alerts the passenger according to the classification result output from the abnormal operation classifying unit 112. In addition, the control signal generation unit 114 generates a control signal for controlling the operation of the escalator according to the classification result output from the abnormal operation classification unit 112. The control signal is transmitted to the control device via the interfaces 118 and 105. When the control device receives the control signal, the control speed of the escalator is variably controlled according to the classification result. After the abnormality is detected, the normal return determination unit 115 detects whether or not the passenger returns to the normal state by his / her own response. When the watch returns to normal, the watch returns the escalator to normal operation manually after handling the abnormal boarding. Signals output from the alerting unit 113, the control signal generating unit 114, and the normal recovery determining unit 115 are recorded as metadata for searching for and analyzing abnormal events through the interfaces 118 and 105 in a paired state with video data. It is stored in the device 104. The monitor can see the image processing result analyzed by the image processing apparatus 107 by the monitor 119. Also, alarms can be issued only when abnormal boarding occurs, reducing the burden on the supervisor.

  Next, details of data processing in the image processing apparatus 107 will be described using the flowchart of FIG. First, in step 201, a monitoring image is acquired. Since time-series image data is necessary when performing motion determination in a later step, a moving image having a certain length of time is stored here. Next, in step 202, the unsteady operation is determined. In this process, for example, the degree of dispersion of gradient information (differential values) in the time direction and the spatial direction calculated from the moving image is evaluated at each pixel position on the image. Details of this method will be described later with reference to FIG.

  Based on the determination process in step 202, it is determined in step 203 whether or not the passenger's movement is non-stationary, and it is selected whether or not to perform movement classification as the next processing step. If the non-stationary degree is low, the process ends and the process proceeds to the next frame. If the degree of non-stationaryness is high, the next operation classification is performed. At this time, recording is started in step 204 using the detection of the unsteady operation as a trigger, and further, the occurrence of the unsteady event is notified to the monitor (step 205).

  In step 206, action classification processing is performed. Here, classification of intentional / unintentional abnormal boarding is performed. Details of this method will also be described later with reference to FIG.

  In step 207, the operation determined to be unsteady in step 203 is classified as an intentional abnormal operation or an unintentional abnormal operation. If it is an intentional abnormal operation, alerting and inverter control corresponding to this are performed (step 208), and if it is an unintentional abnormal operation, alerting and inverter control corresponding to this are performed similarly (step 208). 209). After this, it is determined whether or not the passenger boarding state has returned to normal (step 210). If it has returned to normal, the escalator operation is returned to normal operation (step 211), and if the abnormal operation continues. Continue alerting and inverter control.

  Next, the non-stationary determination process (step 202) in FIG. 2 will be described with reference to FIG. In the non-steady state determination process, the escalator passengers are in a non-steady state, specifically, entering the handrail, entering a cart, stroller or suitcase, reverse running on the step, forward running or forward running, Striking, falling, jumping, being pulled into the gap between the step and rail, falling on the lap and pulling feet or clothes into the pulling part of the step, Detection is based on the degree of unsteady movement calculated from the statistics, and first, the type of abnormality is unknown, but the situation that an abnormality has occurred is detected.

  In FIG. 3, first, the input image is the current frame and the immediately preceding N frame (step 301). In step 302, an inter-frame difference process is performed from the current frame and the immediately preceding frame, and threshold processing is performed in step 303. By this process, edge information of the moving object is extracted from the input moving image. In step 304, the brightness gradient in the spatial direction and the time direction at the position of the moving edge extracted in step 303 is calculated. This time-space luminance gradient is expressed by the following equation (1).

  Next, in step 305, a covariance matrix M of a spatiotemporal luminance gradient expressed by the following equation is obtained from the value of equation (1). M is also called a gram matrix.

  The sum of the formula (2) is calculated from the entire image or pixels in the designated area. Equation (2) is the variance and covariance in a region with a spatiotemporal luminance gradient, and expresses the degree of variation in the plot of equation (1). If the magnitude and direction of movement in the region varies, the rank of equation (2), that is, the number of non-zero eigenvalues of the gram matrix M increases (up to 3). At this time, if the person area in the designated area is small, the motion information of the person is buried during the calculation of equation (2) due to the influence of the background pixels. Therefore, in step 306, a gram matrix M of a scene in which only the background is visible, specifically, a scene in which only the operation of the escalator tread is visible, is calculated, and this is subtracted from the gram matrix of the input image as a background gram matrix. To do. This calculation makes it easier to detect unsteady movements of passengers even if the person area is small.

  Since the eigenvalue calculation (step 307) of the Gram matrix is actually less likely to be zero due to the influence of noise or the like, the continuous rank increment shown in the following equation is used as an index of abnormal operation in the specified region. Use degrees.

Here, λ _1, λ _2, λ ₃ eigenvalues of the Gram matrix M ^{_{(ascending), λ 1 ◇, λ 2}} ◇, λ 3 ◇ is the eigenvalues of the upper left 2 × 2 matrix of the Gram matrix M (ascending). After calculating (3) in step 308, this is subjected to threshold processing in step 309, and it is determined that an unsteady operation has been detected when a certain level abnormality value in (3) is obtained. Based on this signal, image recording is started or alerting is performed. If the level of the expression (3) does not exceed the threshold value for determining the unsteady motion even after a certain period of time has elapsed (step 310), it is determined in step 311 that the scene is a scene in which only the background is shown. Then, a background gram matrix is created and stored, and used for the gram matrix difference process in step 306. If the level of the expression (3) exceeds the threshold value of the non-steady motion determination within a certain time as determined in step 310, it is determined that the passenger's motion is unsteady, and the determination of a certain time has elapsed. The state is reset, and it waits again in step 309 until the expression (3) becomes equal to or less than the threshold value.

  After unsteady motion is detected in step 309, motion classification processing shown in step 206 in FIG. 2 is performed. For this purpose, the time analysis of equation (3) is performed. This is because, for example, when a small child runs backward on the escalator, the non-stationary degree of the expression (3) exceeds the threshold set in step 309 for a long time. Moreover, when a passenger falls down unintentionally, the operation | movement becomes sudden and the time waveform of (3) type | formula also becomes a sharply convex waveform. In this way, the type of abnormal boarding is determined by utilizing the fact that the time waveform of equation (3) varies depending on the type of abnormal boarding. For this determination, for example, Fourier transform or wavelet transform can be used. Abnormal rides are classified into intentional abnormal rides and unintentional abnormal rides, both of which are further classified. Details thereof will be described below.

  First, the classification of intentional abnormal boarding will be described. Examples of intentional abnormal operation include reverse running of a step, step rushing (forward running and forward running), and riding of nonstandard objects such as a stroller. When a passenger runs backward in a step, the value of equation (3) keeps a high level for a long time, and the value of equation (3) suddenly decreases at the moment the reverse runner disappears from view. Shows the waveform returning to level. This is because equation (3) shows a large value when there is an opposite movement in the field of view. In addition, the magnitude of the unsteadiness increases and decreases in synchronization with the operation of running down the steps. This is because the moment when the arms and legs are raised and lowered just like the step movement and the opposite moment are repeated alternately. The characteristics of the time waveform of the expression (3) generated by such an operation can be determined by, for example, Fourier transform or wavelet transform.

  When a passenger takes a step quickly, the same waveform as in the case of reverse running is shown, but the level of unsteadiness is lower than that of reverse running. This is because the direction of travel of the escalator footboard and the direction of passenger movement are the same, and the value of equation (3) is slightly smaller than in the case of reverse running.

  In the case of a stroller, the movement immediately before getting on the treadle becomes an unsteady movement. In this case, the time waveform of the expression (3) is a waveform that is upwardly convex like the fall, but is not as steep as the fall. Therefore, it is determined that a non-standard object such as a stroller is placed when a peak having a frequency lower than the fall occurs in the trapping portion by frequency analysis of the time waveform of equation (3).

  Next, classification of unintentional boarding will be described. Examples of unintentional abnormal operation include low-speed walking and congestion in the lap portion, falling over the tread portion and the lap portion, and being drawn into the escalator. The low speed in the trapping part is a case where a parent or a child with a small child or an elderly person is expecting a boarding timing because of a high step speed. In this case, the time waveform of the expression (3) continues for a time length in which there is a minute fluctuation due to passenger hesitation. When the tread board is unmanned, the time waveform of equation (3) has a low level and is flat, and this can also be determined by frequency analysis using Fourier transform or wavelet transform frequency. In the case of congestion, a waveform similar to that of low-speed walking occurs. In this case, the waveform of equation (3) slightly fluctuates over a longer time than low-speed walking. As described above, the overturn can be determined by detecting a steep peak of the time waveform of the expression (3). At this time, for example, when the detection area on the image is divided into strip areas for each escalator tread and an abnormal operation is determined in each area, it can be determined whether a fall has occurred on the tread or on the lap. With regard to the escalator being pulled in, it is expected that the passenger will make a violent movement trying to escape when the passenger is pulled in. Moreover, the operation cycle is considered to be irregular, unlike the above-described reverse running or rushing of the tread. Therefore, this can also be determined by frequency analysis using Fourier transform or wavelet transform. The above is the details of the action classification process in step 206 of FIG.

  Next, after the abnormal operation is classified into intentional or unintentional in step 207 of FIG. 2, a process of reflecting each case in the inverter control of the escalator will be described. First, the case where the abnormal operation is intentional (step 208) will be described. The process of step 208 will be described with reference to FIG. In step 401 of FIG. 4, a signal of intentional abnormal operation is input. In the case of an intentional abnormal operation, the escalator control method is not changed depending on the type of the abnormal operation in order to alert any intentional abnormal operation classified in step 206. The content of the notice broadcast may be changed according to the type of intentional action. Next, attention is immediately given (step 402). For this, broadcasting by a speaker, or LED embedded in a step is made to emit light in a predetermined alert color or blink. After a certain period of time has passed, this warning is determined to be in a dangerous state (step 403), and the escalator is slowly decelerated by inverter control (step 404). If the alerting has not reached a certain time, the process returns to step 402, and the alerting broadcast and LED lighting / flashing are continued. The above is the alerting & inverter control (step 208) of FIG. 2 performed for intentional abnormal operation.

  For unintentional abnormal operation, the processing of alerting & inverter control (step 209) in FIG. 2 is performed. The process of step 209 will be described with reference to FIG. In step 501 of FIG. 5, a signal indicating an unintentional abnormal operation is input. From here, unlike the case of intentional abnormal operation, the control method of the escalator is switched according to the type of abnormal operation.

  As described above, unintentional abnormal movements include low-speed walking with the elderly and children, congestion, falling, and being drawn. In the case of low-speed walking (step 502), the escalator is guided to decelerate (step 507), and then the escalator is gently decelerated by inverter control (step 508) so that the passenger can easily get on the escalator tread. In the case of congestion (step 503), a large number of people are staying in the trapping section. Therefore, the escalator is guided to accelerate (step 509), the escalator is gradually accelerated (step 510), and the congestion is alleviated. However, at this time, the speed of the escalator is accelerated so as not to exceed the legal speed and at an acceleration that does not cause a fall or trip during acceleration. When the passenger falls (step 504), first, it is determined whether the fall has occurred on the tread or the lap portion (step 511), and if it is on the tread, the deceleration guide (step 507) is given and then the escalator is slowly decelerated. (Step 508). This is because if the escalator is suddenly stopped when a fall occurs on the step, the passenger may fall over. If a fall occurs in the trapping section, the escalator is not stopped, but a safety confirmation broadcast is performed to confirm whether or not the passenger is injured (step 512). If the passenger cannot stand up, the safety confirmation broadcast is continued at step 210 in FIG. 2, and the escalator cannot return to normal operation. In this case, it does not return to normal operation until rescue is performed by the security guard. Next, if the passenger is drawn into the gap between the tread board and the rail, or the gap between the tread board and the trap (step 505), the escalator is immediately stopped (step 513) regardless of the place where the pull-in occurs. Step 506 determines that an exceptional operation has occurred when none of the abnormal operations defined in advance in the classification unit of Step 206 applies to the abnormal operation detected in Step 203 of FIG. The information is recorded (step 514). This is to verify the exception behavior afterwards.

  The above is a detailed description of the function of the image processing apparatus 107 in FIG. In addition to the above processing, the following processing can be used in combination. For example, the input image is reduced in stages, a multi-resolution image pyramid is created, the calculation of equation (3) is performed for each of them, and an operation based on the determination and time series determination is performed as an unsteady degree vector of the multi-resolution image. The classification is also calculated as a vector and processed for determination. In this way, when the dynamic range of passenger movement is wide, it is possible to express the passenger's unsteady movement well with images of any resolution, so parameter settings for image size and frame rate can be made. It is possible to make unsteady operation determination and abnormal operation classification easy to use without bothering.

  The unsteady motion determination / motion classification method based on the above data processing is fast because it does not require an abnormal motion template. In addition, since the calculation of equation (3) can be performed on the entire image without breaking the space-time image into small patches, the processing time is very fast and the implementation is not limited to implementation on a personal computer. It can also be mounted on equipment. In addition, due to this advantage, the above-described multi-resolution analysis can also be performed in real time.

  Note that the present invention is not limited to the above-described embodiments, and various embodiments are possible within the scope of the technical idea of the present invention.

It is a functional block diagram which shows the whole structure of the escalator control apparatus which is one Embodiment of this invention. It is a flowchart which shows the detail of the data processing in an image processing apparatus. It is a flowchart of unsteady operation detection. It is a flowchart of alerting to an intentional abnormal operation and escalator control. It is a flowchart of alerting to unintentional abnormal operation and escalator control.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Handrail 102 Tread 103 Drive device 104 Recording device 105, 118 Interface 106 Camera 107 Image processing device 108 A / D conversion unit 109 Image memory 110 D / A conversion unit 111 Unsteady operation determination unit 112 Abnormal operation classification unit 113 Unit 114 control signal generation unit 115 normal return determination unit 116 memory 117 CPU
119 monitor

Claims (5)

In an escalator control device comprising: an image acquisition device that acquires an image of an escalator and a passenger; and an image processing device that processes an image acquired by the image acquisition device.
The image processing apparatus includes:
An unsteady motion determination unit that detects the unsteady state of the motion of the passenger by processing the image;
Based on the temporal change of the unsteady state detected by the unsteady movement determination unit, an abnormal movement classification unit that classifies the movement of the passenger;
An escalator control device comprising:   The unsteady motion determination unit according to claim 1, wherein the unsteady motion determination unit detects the unsteady state based on a non-stationary degree calculated from a spatial change and a temporal change in brightness of the image, and the abnormal motion classification unit includes the abnormal motion classification unit. An escalator control device that classifies the movements of the passengers according to temporal changes in unsteadiness.   3. The escalator control device according to claim 1, further comprising a control signal generation unit that generates a control signal for controlling operation of the escalator in accordance with a classification result of the abnormal operation classification unit.   3. The escalator control device according to claim 1, further comprising: an alerting unit that alerts the passenger according to a classification result of the abnormal operation classifying unit.   The escalator control device according to any one of claims 1 to 4, further comprising a normal return determination unit that detects that the movement of the passenger has returned to a steady state.

Images