WO2020136513A1

WO2020136513A1 - Detecting object on escalator or moving walkway

Info

Publication number: WO2020136513A1
Application number: PCT/IB2019/061063
Authority: WO
Inventors: Ka Fai William LEE; Yin San Edmond YEUNG; Yiu Wah Colman YEUNG; Chun Long Chris WONG
Original assignee: Mtr Corporation Limited
Priority date: 2018-12-24
Filing date: 2019-12-19
Publication date: 2020-07-02

Abstract

A video content analysis module (40), a system, an apparatus and a method for detecting a dangerous object on an escalator (100) or moving walkway, which may damage the escalator (100) or moving walkway by jamming in the comb section (10), the video content analysis module (40) receives a video stream from a video camera (30) that monitors the comb section (10) and is configured to analyze the video stream to identify objects, determine whether any of the identified objects is a dangerous object and generate an alert in response to determine that an identified object is a dangerous object.

Description

DETECTING OBJECT ON ESCALATOR OR MOVING WALKWAY

BACKGROUND

Escalators are moving staircases which transport people from one level of a building to another level of a building. An escalator comprises a plurality of steps, which may be linked by a closed loop chain and a system of gears to move the steps in a loop between a first landing platform and a second landing platform. The first and second landing platforms are at different heights. For example the first landing platform may be on a lower floor of a building while the second landing platform may be on an upper floor of a building.

Moving walkways are for conveying passengers, luggage or another payload between landing platforms which are on the same level (without inclination) or different levels (with inclination) of a building. Moving walkways have a similar construction to escalators, but have a plurality of pallets, rather than steps. The pallets may be linked by a closed loop chain and a system of gears to move the pallets in a loop between a first landing platform and a second landing platform. Moving walkways are sometimes referred to as passenger conveyors, travellators, moving walks or autowalks.

The landing platforms of an escalator or moving walkway each include a comb section, which is to interface with the steps or pallets of the escalator or moving walkway. The comb section includes a plurality of spaced apart teeth or prongs which interface with grooves in the surface of each step or pallet of the escalator or moving walkway. In some examples the comb section comprises a comb plate and one or more combs, each with a plurality of spaced apart teeth, which are fixed to the comb plate.

The comb section at one end of the escalator or moving walkway receives the steps or pallets before they enter the underside of the loop (e.g. the comb on the upper landing platform of an escalator which is moving upwards). The comb section at the other end is a comb section from which the steps or pallets emerge when they return from the underside of the loop to the upper side. As the direction of an escalator or moving walkway may be reversed the combs at the top and bottom of an escalator or respective ends of a moving walkway may have the same design.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig.l shows an example in which an object is jammed in the comb section of an escalator; Fig. 2 shows an example of damage which may be caused by jamming of an object in a comb section of an escalator;

Fig. 3 is a schematic diagram showing an example of a system according to the present disclosure;

Fig. 4 is a schematic diagram showing an example of a video content analysis module according to the present disclosure;

Fig. 5 is a schematic diagram showing an example of a method of operation of a video content analysis module according to the present disclosure;

Fig. 6 is a schematic diagram showing an example of a system including a plurality of video content analysis modules according to the present disclosure;

Fig. 7A shows an example of an alert sent to a user according to the present disclosure;

Fig. 7B shows an example of an image sent in an alert generated by a video content analysis module according to the present disclosure;

Fig. 8 is a schematic diagram showing an example of a system according to the present disclosure;

Fig. 9 is a schematic diagram showing an example of an apparatus comprising a video camera and a video content analysis module according to the present disclosure;

Fig. 10 is a diagram showing an example according to the present disclosure in which a video camera installed in an escalator or moving walkway;

Fig. 11 is a diagram showing an example according to the present disclosure in which a video camera is positioned a distance D behind the comb section;

Fig. 12 is a diagram showing an example according to the present disclosure in which a first video camera is installed at a first side of an escalator and a second camera is installed at a second side of an escalator or moving walkway;

Fig. 13 is a diagram showing an example view of comb section as seen by a video camera in accordance with the present disclosure;

Fig. 14 is a schematic diagram showing an example structure of a video content analysis module according to the present disclosure; Fig. 15 is a schematic diagram showing an example method of training a video content analysis module according to the present disclosure; and

Fig. 16 is a diagram showing an example method of detecting a dangerous object which is likely to damage an escalator or moving walkway according to the present disclosure.

DETAILED DESCRIPTION

The teachings of the present disclose may be implemented on both escalators and moving walkways. The following description refers mainly to escalators, however it is to be understood that every reference to an escalator could equally be applied to a moving walkway and vice versa. Likewise any reference to escalator steps could equally apply to the pallets of a moving walkway.

As many passengers use an escalator each day, debris, rubbish and other objects may fall on the escalator steps. From there they will be carried to the comb section of the escalator. Certain objects may get stuck in the comb section of the escalator. For example, the object may jam between two teeth of the comb section. Fig. 1 shows an example in which a screw 2 is stuck in the comb section 10 of an escalator. If an object becomes stuck in the comb section 10 it may obstruct movement of the escalator steps 20 into the comb section.

As can be seen in Fig. 1, the screw 2 has become lodged between grooves of a step 20 and prevents the step from moving forward. This interferes with operation of the escalator. Many escalators have a safety switch incorporated into the comb plate, which stops the escalator if the comb plate is forced upwards, for instance due to an object jammed in the comb. However, due to massive weight of consecutive moving steps, the safety switch may not be able to stop the steps in time and serious damage may be caused. For example, one or more steps of the escalator may break as they are pushed forward into the comb section by the step chain, but prevented from entering the comb by the object. Fig. 2 shows an example, in which a step 2 of an escalator 100 has broken due to an object lodged in the escalator comb. While Figs. 1 and 2 show an escalatorjamming objects can cause similar issues for moving walkways. In the context of this disclosure, a dangerous object means an object which is likely to damage an escalator or moving walkway by getting stuck or jamming in the comb section of the escalator or moving walkway.

One aspect of the present disclosure proposes a system for detecting a dangerous object on an escalator or moving walkway, in which the system comprises a video camera positioned adjacent a comb section of an escalator or moving walkway and arranged to monitor the comb section; and a video content analysis module configured to receive a video stream from the video camera, analyze the video stream and generate an alert in response to determining that the video stream includes a dangerous object which may damage the escalator or moving walkway by jamming at the comb section.

Another aspect of the present disclosure proposes a video content analysis module for detecting a dangerous object on an escalator or moving walkway. The video content analysis module may comprise an input to receive a video stream from a video camera that monitors a comb section of an escalator or moving walkway and a video content analyzer. The video content analyzer may be configured to analyze a plurality of images in the video stream to identify objects in the video stream, determine whether any of the identified objects is a dangerous object which may damage the escalator or moving walkway and generate an alert in response to determining that an identified object is a dangerous object.

In one example, the video content analysis module maybe able to detect the dangerous object before it jams in the comb section. In this way the escalator or moving walkway may be stopped before jamming and before any damage occurs.

Fig. 3 is a schematic diagram showing an example of a system according to the present disclosure. The system includes a video camera 30 which is arranged to monitor the comb section 10 of an escalator 100. For example the video camera 30 may be positioned adjacent the comb section 10 and may have field of vision directed at the escalator comb section 10. The video camera generates a video stream comprising a plurality of images which may show the escalator comb section, feet or lower legs of passengers moving past the comb section and any objects dropped on the steps of the escalator or near the escalator comb section. The system further includes a video content analysis module (VCA) 40 which is arranged to receive the video stream from the video camera 30, analyze the video stream and generate an alert in response to determining that the video stream includes a dangerous object which may damage the escalator or moving walkway 100.

Fig. 4 is a schematic diagram showing an example of a video content analysis module 40 according to the present disclosure. The video content analysis module may comprise an interface 42 to receive a video stream and a video content analyzer 44. In one example the interface 42 is a physical interface, such as a cable or cable port which is to connect to a video camera. For instance the interface 42 may be an audio/video (A/V) port, a High-Definition Multimedia Interface (HDMI), Digital Video Interface (DVI), a universal serial bus (USB) port, Video Graphics Array (VGA) interface. Having a physical, rather than wireless, connection between the video camera 30 and video content analysis module 40 may help to ensure reliability, speed of transmission of the video stream and/or high bandwidth. The video content analyzer 44 comprises a memory to store images of the video stream and at least one processor to analyze images of the video stream. The at least one processor may include a general purpose processor, a hardware accelerator and/or specialized graphical processors. The at least one processor may utilize a combination of hardware and/or machine readable instructions executed by the at least one processor to implement artificial intelligence such as a neural network or machine learning algorithm in order to analyzer the images of the video stream.

The video content analyzer 44 may compare features of the identified object against predetermined criteria or rules to determine whether the object is a dangerous object. The video content analyzer 44 may be configured to determine that a screw, nail or other thin elongate object is a dangerous object. The video content analyzer may include artificial intelligence such as a neural network or machine learning algorithm that has been trained to recognize such objects and identify those objects as dangerous. In one example, the video content analyzer is configured to determine that an object having a width of less than 6mm is a dangerous object.

Fig. 5 shows an example method of operation 50 of the video content analysis module 40. At block 52 the interface 42 of the video content analysis module 40 receives a video stream comprising a plurality of images of the escalator comb section 10 from a video camera 30. The video stream may be passed from the interface 42 to the video content analyzer 44. At block 54 the video content analyzer analyzes the video stream to identify objects in the images of the video stream. At block 56 the video content analyzer 44 determines whether an identified object is a dangerous object which is likely to damage the escalator. At block 58, in response to determining that an object is a dangerous object, the video content analyzer 44 generates an alert.

The alert may for example be a visual signal, an audio signal or a message sent to a remote device. In one example the remote device is a server, while in other examples it may be a client device.

Fig. 6 is a schematic diagram showing an example in which a plurality of escalators 100A, 100B, lOOC each have a respective video content analysis module 40A, 40B, 40C which are capable of communicating with a server 200. The video content analysis modules may be coupled to respective video cameras for monitoring comb sections of the respective escalators. When a video content analysis module 40A, 40B, 40C determines that a dangerous object is present at the comb section of an escalator, the video content analysis module may send an alert message to the server 200. The server 200 may include a processor, a memory and a storage device such as a hard disk and a wired or wireless communication interface for receiving an alert from a video content analysis module. The server may execute machine readable instructions stored in the memory to store the received alert in the storage device. The server 200 may display the alert on a display coupled to the server or forward the alert to a client device 300. The client device may for example be a mobile phone, tablet computer, desktop computer or laptop computer etc. The client device may act as a monitoring device which is operated by a user such as a member of staff who has responsibility to monitor the escalators. Upon receiving the alert at their client device, the member of staff may take appropriate remedial action. In other examples, the video content analysis module may send an alert directly to the client device 300.

An example of an alert message is shown in Fig. 7A. The alert message 700 may include an image 740 of the escalator comb section and dangerous object, e.g. the alert message may include one of the images from the plurality of images in the video stream produced by the video camera. The alert message may indicate the location 710 of the detected dangerous object, e.g. the escalator on which the detected dangerous object is located and/or the comb section location, such as at the upper landing or lower landing of the escalator. The alert message may include the landing location, the escalator number, a date and/or time 720 at which the dangerous object was detected. The alert may include further information 730, such as a type of object which has been detected. For example, the type of object may be a generic designation such as“obstacle”, or a more specific designation such as“screw” or“nail” etc.

Fig. 7B shows an example of the message displayed on a computing device, such as a tablet computer, desktop computer or mobile phone etc. The image 740 may be modified to indicate the location of the dangerous object more clearly. For example, the portion of the image including the dangerous object may be magnified 750 and/or indicated by a graphic symbol such as a ring or box surrounding the dangerous object or an arrow pointing to the dangerous object, or displayed in a different color or at a different brightness to the rest of the image. In the example of Fig. 7B, the dangerous object and surrounding area are magnified 750. This may be helpful where the dangerous object is small, e.g. 6mm in width or less, and might otherwise be hard to spot by the human eye. Further, in the example of Fig. 7B the location of the dangerous object within the magnified portion 750 is indicated by a graphical symbol 760 such as a box which may be a bright color such as red or green. This graphical symbol may help a user to spot the dangerous object if it is a similar color to the surrounding environment, e.g. a dangerous object may be metal and a similar color to the comb section and/or escalator steps.

The alerts may be sent to the server 200 using a wired or wireless mode of communication. In one example the video content analysis modules send the alerts wirelessly, e.g. over a wired local area network (WLAN) or a wireless cellular network such as a 3G, 4G or 5G network. By using a wireless mode of communication the system may be deployed economically without laying additional cables or wired communication lines. Further, as the alerts may be relatively rare and may include a single image or small number of images, they may be sent over a wireless network without negatively impacting the bandwidth of the network. Meanwhile, as each video content analysis module is local to the escalator which it is monitoring difficulties of bandwidth and latency in communicating large volumes of video data to a remote server are avoided.

Furthermore, the analysis of the video stream may be carried out in a timely fashion as the video analysis is carried out locally at each escalator. In some examples, each video analysis module be located in close proximity to the video camera and/or have a high bandwidth wired connection to the video camera so as to receive the video stream with little delay and reduce signal

degradation. Time is an important consideration when detecting dangerous objects on an escalator. If an object is left or remains undetected for more than a few seconds while the escalator is running, there is a risk the escalator will be damaged.

In response to the alert, the escalator may be stopped and the dangerous object removed. In this way accidents may be avoided and the service life of the escalator improved. On the other hand, if after inspection, it turns out that the identified object is not a dangerous object and does not pose a significant risk of damaging the escalator, then the alert may be classified as a false alert. Feedback may be sent to the video content analysis module indicating that the object is not a dangerous object. In response to receiving this feedback, the video content analysis module may update the software of a neural network, machine learning algorithm or criteria for determining whether an object is dangerous, so that the accuracy of object classification may be improved and incidence of future false alerts may reduce over time.

Fig. 8 is a schematic diagram showing another example of the present disclosure in which an escalator 100 includes a stopping means 60, as well as a video content analysis module 40 and a video camera 30 for monitoring a comb section 10 of the escalator 100. The stopping means 60 is configured to stop the escalator when activated. For instance, the stopping means may implement this stopping function by turning off the escalator power or stopping the escalator motor. The stopping means may be a smooth stop mechanism which is configured to gradually slow the escalator to stop over a period of a few seconds, rather than immediately stop the escalator. In one example, the stopping means 60 may be activated by an operator remotely, e.g. by sending a stop instruction from a remote device. In this example, the operator may activate the stopping means in response to receiving the alert and determining that the alert is a genuine one, e.g. by reviewing the image in the alert and deciding that the object is a dangerous object which may damage the escalator. In another example, the stopping means 60 may be activated automatically by the video content analysis module. For example the video content analysis module may send an alert in the form of a stopping signal to the escalator system which causes the stopping means to stop the escalator. In one example, the video content analysis module 40 and video camera 30 may be installed at a side of the escalator. As they are installed at the side of the escalator, the video content analysis module 40 is local to the video camera 30. Therefore, the video analysis may be carried out locally. Compared to sending the video stream to a remote server for analysis, carrying out the video analysis locally may avoid difficulties of bandwidth, latency or delay. When a screw, nail or other dangerous object is near the comb section of an escalator, it is desirable to detect the object and take remedial action quickly, so as to reduce the possibility of damage to the escalator. Therefore carrying out the video analysis locally may help to increase the speed of detection and avoid jamming a wireless or other network for connecting the video content analysis module to a remote server.

The video content analysis module 40 and video camera 30 may be provided together as an apparatus 400 for installation in an escalator. In one example, shown in Fig. 9, the video content analysis module 40 and video camera 30 may be provided inside a common housing 410 for easy installation together in the desired location. The housing 410 may include a window or aperture through which the video camera 30 may monitor the comb section of an escalator.

The video content analysis module and the video camera may be installed in a skirt of the escalator near a comb section of the escalator. In one example the video camera is installed in a skirt at the side of the escalator. Fig. 10 shows an example of one end of an escalator 100, including a landing platform 12, comb section 10, skirt 110 and handrail 120. The skirt 110 is the part of the escalator which is to the side of and adjacent the steps 20. The skirt 110 may take the form of a skirt panel and may be below the moving handrail 120. The video camera 30 may be installed behind the skirt panel. The skirt panel may include an opening through which the video camera can view the escalator comb section 10. The video content analysis module may be installed in close proximity to the video camera behind the skirt panel. The skirt panel may further include a light to illuminate the part of the escalator near the landing platform, e.g. for illuminating the escalator comb section 10 and/or landing platform 12 and a nearby step 20. In one example, the video content analysis module and video camera may be integrated together into the same apparatus, for example they may share a common housing. In one example the light may be integrated together with the video content analysis module, the video camera or with the combined video camera and video content analysis module. Providing these components as a single apparatus, may facilitate easy installation and help to reduce wiring and/or reduce degradation of the video signal passing between the video camera and the video content analysis module.

The video camera 30 has a field of view 32, which is the area which can be seen by the video camera. The video camera 30 may be tilted downward, which means that the center line of the field of view of the camera extends at an angle to the horizontal in the downward direction. In this way privacy of passengers on the escalator may be preserved and the field of view may be focused on the area around the comb section 10 of the escalator. In one example an upper line of the field of view is at an angle extending downward from the horizontal. The broader the field of view the more of the escalator comb section can be monitored by the video camera. In one example the vertical field of view, i.e. the extent of the field of view in the vertical direction, is at least 60 degrees. Having a broad vertical field of view may help to reduce blind spots..

If the video camera is placed in a higher position it may see more of the comb section 10, but if too high then it may interfere with privacy of passengers or may not be able to see the comb section clearly. Further, if too high then the video camera may not be able to see portions of the comb section near to the video camera due a blind spot which is out of the field of view. In one example the video camera is positioned at a height of between 5cm and 10cm above the level of the comb section 10.

The video camera 30 may have a position which is horizontally offset from the comb section 10. This may help the video camera to view more of the comb section 10, reduce blind spots and/or help with viewing objects nearer to the side of the escalator. Fig. 11 is a schematic diagram showing an example in which the video camera 30 is positioned at a horizontal distance D behind the comb section 10 (i.e. horizontally offset on the step side of the comb section as shown in Fig. 11). In other examples the video camera 30 may be positioned in front of the comb section (i.e. horizontally offset on the landing plate side of the comb section). When the horizontal offset D is behind the comb section 10, the offset may for example be at least 10cm.

In one example the horizontal offset behind the comb section is between 10cm and 20 cm. When the horizontal offset D is in front of the comb section, the offset may for example be at least 5cm and in one example is between 5cm and 10cm. As shown in Fig. 10, the video camera 30 may have a blind spot 34 including parts of the comb section which cannot be seen by the video camera 30. Therefore, in some examples, there may be a second video camera which may reduce or eliminate the blind spot. Fig. 12 shows an example in which the video camera 30 is located on a first side of the escalator and a second video camera 35 is located at a second side of the escalator. The second video camera 35 has a field of view 36. In some examples, the field of view 36 of the second video camera may extend into the blind spot of the first video camera 30. Thus, when the first video camera and second video camera are on opposite sides of the escalator they may help to mitigate or overcome each other’s blind spots as shown in Fig. 12. In one example, each video camera may view a portion of the comb section so that together between both video cameras the whole comb section is monitored. Having two video cameras may also help to reduce incidence of an object being completely obscured by passengers, as if the view of an object by the first video camera is obscured the second video camera may be able to view the object. In other examples there may be more than two video cameras to monitor a comb section, but in most cases it is envisaged that one or two video cameras should be sufficient. Where there is more than one video camera monitoring a comb section, the video cameras may share a single video content analysis module or each video camera may have its own video content analysis module. For example, the first video camera may be connected to a first video content analysis module at a first side of the escalator and the second video camera may be connected to a second video content analysis module at a second side of the escalator.

While the above description has referred to a video camera to monitor the comb section at a first end of the escalator, there may be another video camera to monitor the comb section at the second end of the escalator. Generally the comb section receiving the steps may be monitored (e.g. the lower landing comb section on an escalator which moves downwards, or the upper landing comb section on an escalator which moves upwards). However, as the direction of an escalator may be reversed, there may be respective video cameras at each end of the escalator. In one example the video camera(s) at the first end of the escalator and second end of the escalator may each be connected to their own respective video content analysis modules. In another example the video cameras may share a common video content analysis module. Providing each video camera with its own video content analysis module may reduce the need for wiring inside the escalator and relieve the processing burden on the video content analysis modules.

The video camera 30 monitors the comb section and produces a video stream comprising a plurality of images. For example each image may be a frame in the video stream. The may be a lot of activity at the comb section of the escalator as passengers move past the comb section and onto the landing platform. Moving passengers may temporarily obscure some or all of the field of view of the video camera. Objects may fall onto an escalator step and then move with the step to the comb section, at which point they may roll or around or lie by the comb section. While some objects, such as screws or nails, may be dangerous in the sense that they may lodge in the comb section and catch a step preventing it from moving into the comb section, other objects such as fluff, or larger objects such as plastic bottles may not pose a danger to the escalator.

Fig. 13 shows an example of an image form the field of view of a video camera positioned near the comb section 10 of an escalator. The comb section 10 includes a plurality of spaced apart teeth 9 which interface with grooves 22 in steps 20 as they move toward the comb section 10.

The comb section may include one or more comb plates 11 each with a plurality of teeth 9 9. The comb plates 11 may be secured to a comb plate (not shown) which is fixed to or forms part of the landing platform 12. The opposite skirt 110 can also be seen in Fig. 13. Two dangerous objects 2A and 2B are shown in Fig. 13. These are highlighted by a graphical box for clarity. Anon- dangerous object 2C can also be seen in Fig. 13. The non-dangerous object may be too large to get stuck in the comb section 10 or may be of a material which is too soft to cause serious obstruction of the escalator.

Identifying objects despite a large volume of passengers and distinguishing dangerous and non- dangerous objects is a complicated task. The video content analysis module may use a machine learning algorithm or neural network to determine whether an object is a dangerous object.

Fig. 14 shows an example structure of a video content analysis module 40 according to an example of the present disclosure. The video content analysis module 40 includes an interface 42 which is to receive a video stream from a video camera and a video content analyzer 44 which is to analyze the video stream. The interface 42 may be a physical interface, while the video content analyzer may include at least one processor and a memory. The at least one processor may include a general purpose processor, a specialized image processor, hardware accelerator and/or other dedicated hardware. The processor may execute hardware logic or instructions stored in the memory to implement machine learning algorithms and/or a neural network. The video content analysis module may also include an image signal processor 43 to process the images of the video stream before they are delivered to the video content analyzer. For example, the image signal process may filter noise, adjust the lighting, sharpness or other qualities of the image to put it in better condition for analysis by the video content analyzer 44.

The video content analysis module 40 may include a storage device 46, such as a hard disk, flash memory or other long term storage device which is capable of storing at least several days of the video stream and may be configured to store the video stream on the storage device 46. In one example, the storage device 46 may have capacity to store at least 7 days of the video stream. In this way, the video may be obtained and reviewed later if needed. The video stream may be fully or partially deleted periodically to make room for new images.

In one example the at least one processor of the video content analyzer may implement an object identifier 44 A to identify objects at the comb section of the escalator and an object classifier 44B to determine whether an identified object is a dangerous object. The at least one processor may also implement an alarm generator 44C to generate an alarm in response to the object classifier determining that an object is a dangerous object. The video content analysis module may include a wireless module 48, such as a wifi, 3G, 4G or 5G interface. The alarm generator 44C may be configured to send an alert to a remote station via the wireless module 48.

The video content analyzer 44 may be trained to disregard background motion and identify substantially stationary objects near the comb section of the escalator or moving walkway. In this context, substantially station objects means objects which stay within the vicinity of the comb section. For example, the object identifier 44A may be trained to treat the motion of passengers as background and focus on unusual features such as objects lying on a step, at the comb section or near the comb section of the escalator. The object identifier 44A may include an obscuration module that is capable of modelling the presence of an object which is temporarily obscured by another object. In this way the object identifier 44A may be able to detect or model the presence of an object which it has previously seen, even if it is temporarily obscured by a passenger walking by.

The object classifier 44B may be configured to determine characteristics of an identified object and classify the object as a dangerous object based on a comparison of the determined characteristics of the object with characteristics of a dangerous object. The determined characteristics may include one or more of the following: a shape of the object, a size of the object, a colour of the object and whether or not the object has a metallic appearance. As near objects may appear larger than distant objects, the object classifier may be calibrated so that is can determine the actual size of objects at different distances from the video camera.

Characteristics of a dangerous object may include at least one of: a width of between 4mm and 6mm, an elongate shape and a metallic appearance.

In one example, the object classifier 44B may be configured to determine that an object is a dangerous object if it has a width of between 4mm and 6mm. In this context a width means a dimension of the object, for example for a screw or nail it may mean a diameter of the screw or nail. In some examples the object classifier may classify any object with an elongate portion with a width between 4mm and 6mm is a dangerous object. In other examples the classifier may classify any object with an elongate portion having a width between 4mm and 6mm and a metallic appearance as a dangerous object

Most escalator and moving walkway combs have a gauge of 6mm, i.e. a gap of 6mm between adjacent teeth. The inventors have found that M4 and M6 screws, which have widths of 4mm and 6mm in their main body, pose a serious risk of getting stuck in the comb section and causing damage. However, screws of less than 4mm are more likely to fall through the gap between the teeth without getting stuck and objects with widths greater than 6mm are less likely to get stuck but may just roll back and forth near the interface between the comb section and the steps without getting stuck.

The video content analysis module may be configured to generate an alert upon detecting presence of a dangerous object at the comb section for at least predetermined period of time. If the predetermined period of time is too short then the incidence of false alerts may increase, but if the predetermined period of time is too long then the risk of a dangerous object getting stuck in the comb section and causing damage to the escalator is increased. Therefore a shorter predetermined period of time may improve the chances of generating an alert and/or stopping the escalator before the object gets jammed in the comb section. In one example, the predetermined period of time may be between 1 second and 4 seconds. In another example, the predetermined period of time may be between 1 and 3 seconds. In another example, the predetermined period of time may be between 2 and 3 seconds. In some examples, the predetermined period of time may be user adjustable. In some examples, the alert may be repeated every predetermined period of time until the object is no longer detected (e.g. until the object is removed).

The video content analyzer may be trained by providing it with sample video streams and images. For example, objects of interest or stationary objects may be highlighted or selected by a user during initial training, so that the video content analyzer may learn to ignore normal motion of passengers getting on or leaving the escalator. In later stages of training the video content analyzer may be provided with images or video streams including dangerous objects and images or video streams without dangerous objects, or with non-dangerous stationary objects, as a training set. At later stages, the video content analyzer may be provided with live video streams and configured to generate alerts which are sent to a user, e.g. via a backend server. The user may review the image and confirm that the object is dangerous or may indicate that the object is not dangerous and that it is a false alert. In this way, in some examples, accuracy of 95% or greater may be achieved in identifying dangerous objects. The training process may be continued while the system is in actual operation, with the operator stopping the escalator and removing dangerous objects when necessary, but also providing feedback to the video content analyzer indicating whether each alert was false or genuine (or the video content analyzer may be configured to determine that the alert was genuine if the escalator was stopped). In this way over days, months and years, the video content analyzer may be continually improved.

Fig. 15 shows an example method 150 of training a video content analysis (VCA) module according to the present disclosure.

At block 152 the video content analysis module sends an alert and an alert is received by a user. The alert indicates presence of a potentially dangerous object at the comb section of an escalator. At block 154 the user provides feedback to the video content analysis module indicating that the potentially dangerous object or is not a dangerous object. For example, the user may send a message or click an icon indicating whether or not the object is dangerous and this may be communicated to the video content analysis module.

At block 156 the video content analysis module 140 is updated based on the feedback. For example artificial intelligence such a machine learning algorithm, neural network or criteria used by the video content analyzer 44 may be updated based on the feedback.

Fig. 16 shows a method 160 of detecting a dangerous object which is likely to damage an escalator. At block 162 the comb section of an escalator is monitored by a video camera. At block 164 a video feed is sent from the video camera to a video content analysis module, such as that described in any of the examples above. At block 166 the video stream is analyzed by the video content analysis module. At block 168 the video content analysis module generates an alert. The alert may be send to a user, e.g. via a remove server. The alert indicates the presence of a dangerous object at the comb section of the escalator or moving walkway. The method may further include stopping the escalator in response to receiving the alert.

The present disclosure has been described above with reference to several illustrative examples. However, it is to be understood that a person skilled in the art may make variations or modifications to the above examples, while still remaining within the scope of the present disclosure. For instance, in any of the above examples references to an escalator or escalator steps may be replaced with references to a moving walkway or pallets of the moving walkway.

Claims

1. A video content analysis module for detecting a dangerous object on an escalator or moving walkway, the video content analysis module comprising:

an input to receive a video stream from a video camera that monitors a comb section of an escalator or moving walkway; and

a video content analyzer configured to:

analyze a plurality of images in the video stream to identify objects in the video stream;

determine whether any of the identified objects is a dangerous object which may damage the escalator or moving walkway by jamming at the comb section;

generate an alert in response to determining that an identified object is a dangerous object.

2. The video content analysis module of claim 1 wherein the video content analyzer is configured to determine that a screw, nail or other thin elongate object is a dangerous object.

3. The video content analysis module of claim 1 wherein the video content analyzer is configured to determine that an object having a width of less than 6mm is a dangerous object.

4. The video content analysis module of any one of the preceding claims, wherein generating an alert includes sending the alert to a remote device.

5. The video content analysis module of claim 4 wherein the alert includes an image from the video stream, wherein the image includes the dangerous object.

6. The video content analysis module of claim 5 wherein the image is modified to highlight the dangerous object.

7. A system for detecting a dangerous object on an escalator or moving walkway, the system comprising:

a video camera positioned adjacent a comb section of an escalator or moving walkway and arranged to monitor the comb section; and

a video content analysis module according to any one of claims 1-7.

8. The system of claim 7 wherein the video camera and the video content analysis module and the video camera are both installed at a side of the escalator or moving walkway.

9. The system of claim 7 or 8 further comprising a stopping means to stop the escalator or moving walkway, the stopping means configured to stop the escalator or moving walkway in response to receiving an alert from the video content analysis module.

10. The system of claim 7 or 8 further comprising a remote device for receiving the alert from the video content analysis module and wherein the escalator or moving walkway comprises a stopping means that is configured to stop the escalator upon receiving an instruction to stop from said remote device or another remote device.

11. The system of any one of claims 7 to 10 wherein the video camera is angled downwards.

12. The system of any one of claims 7 to 11 wherein the video camera has a vertical field of view of at least 60 degrees.

13. The system of any one of claims 7 to 12 wherein the video camera is positioned at between 10cm and 20cm behind of the comb section on a step or pallet side thereof, or between 5cm and 10cm in front of the comb section on a landing plate side thereof.

14. The system of any one of claims 7 to 13 wherein the video camera is located on a first side of the escalator or moving walkway and wherein the system further comprises a second video camera which is located at a second side of the escalator or moving walkway.

15. The video content analysis module of any one of claims 1-6 wherein the video analyzer is configured to use a machine learning algorithm or neural network to determine whether an object is a dangerous object.

16. The video content analysis module of claim 15 wherein the video content analyzer is trained to disregard background motion and identify substantially stationary objects near the comb section of the escalator or moving walkway.

17. The video content analysis module of any one of claims 1-6 or 15-16 wherein the video content analyzer includes an object identifier to identify objects at the comb section of the escalator and an object classifier to determine whether an identified object is a dangerous object.

18. The video content analysis module of claim 17 wherein the object identifier includes an obscuration module that is capable of modelling the presence of an object which is temporarily obscured by another object.

19. The video content analysis module of claim 17 or 18 wherein the object classifier is configured to determine characteristics of an identified object and classify the object as a dangerous object based on a comparison of the determined characteristics of the object with characteristics of a dangerous object.

20. The video content analysis module of claim 19 wherein the characteristics of a dangerous object include at least one of: a width of between 4mm and 6mm, an elongate shape and a metallic appearance.

21. The video content analysis module of any one of claims 1-6 or 15-20 wherein the video analyzer is configured to generate an alert upon detecting presence of a dangerous object at the comb section for at least predetermined period of time, wherein said predetermined period of time is between 1 second and 3 seconds.

22. A method of training a video content analysis module according to any one of claims 1-6 or 15-21, the method comprising: receiving an alert from the video content analysis module, the alert indicating presence of a potentially dangerous object at the comb section of an escalator or moving walkway;

providing feedback to the video content analysis module indicating that the potentially dangerous object or is not a dangerous object.

23. A method of detecting a dangerous object which is likely to damage an escalator or moving walkway, the method comprising:

monitoring a comb section of the escalator or moving walkway with a video camera; sending a video feed from the video camera to a video content analysis module according to any one of claims 1-6 or 15-21;

analyzing the video stream with the video content analysis module; and

receiving an alert from the video content analysis module indicating the presence of a dangerous object at the comb section of the escalator or moving walkway.

24. The method of claim 23 further comprising stopping the escalator in response to receiving the alert.

25. An apparatus comprising a video content analysis module according to any one of claims 1-6 or 15-21 and a video camera for monitoring a receiving comb section of an escalator or moving walkway, wherein the apparatus is configured for installation behind the skirt panel of an escalator or moving walkway proximate a comb section of the escalator or moving walkway.