WO2022183663A1

WO2022183663A1 - Event detection method and apparatus, and electronic device, storage medium and program product

Info

Publication number: WO2022183663A1
Application number: PCT/CN2021/107285
Authority: WO
Inventors: 徐孟莹; 曾星宇; 于志鹏; 梁鼎
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-03-03
Filing date: 2021-07-20
Publication date: 2022-09-09
Also published as: CN113011291A

Abstract

An event detection method and apparatus, and an electronic device, a storage medium and a program product. The method comprises: acquiring a first video stream of an escalator; performing human body detection on the first video stream to determine a first region, in the first video stream, in which a target object is located; determining a relative speed between the target object and the escalator according to a first image position of a target point of the first region in a video frame of the first video stream; and determining an event detection result for the target object according to the relative speed.

Description

Event detection method, apparatus, electronic device, storage medium and program product

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims the priority of the Chinese patent application number 202110236987.7 filed on March 3, 2021, and the application name is "event detection method and device, electronic device and storage medium", the full text of which is incorporated herein by reference Public.

technical field

The present disclosure relates to the technical field of computer vision, and in particular, to an event detection method, apparatus, electronic device, storage medium and program product.

Background technique

With the development of the economy and the continuous improvement of infrastructure construction, the application of escalators in shopping malls, office buildings, public transportation and other scenarios is becoming more and more extensive. While the escalator brings convenience to pedestrians, pedestrians have abnormal actions on the escalator, such as falling, retrograde, walking, running, squatting, etc., which may cause accidents that have attracted more and more attention. If the abnormal movement on the escalator cannot be detected in time, and the escalator is stopped in time by an alarm, it may cause serious damage to the safety of life and property of the pedestrians on the escalator.

SUMMARY OF THE INVENTION

The present disclosure provides an event detection method, apparatus, electronic device, storage medium and program product.

According to an aspect of the present disclosure, an event detection method is provided, including: acquiring a first video stream of an escalator; performing human body detection on the first video stream, and determining the first video stream where a target object is located in the first video stream. an area; according to the first image position of the target point of the first area in the video frame of the first video stream, determine the relative speed between the target object and the escalator; according to the relative speed , and determine the event detection result of the target object.

In a possible implementation manner, determining the distance between the target object and the escalator according to the first image position of the target point of the first area in the video frame of the first video stream The relative speed includes: according to the first image position and the coordinate correspondence between the image coordinate system of the video frame and the plane coordinate system where the escalator is located, determining that the target point is in the plane coordinate system according to the first plane position of the target point in the multiple video frames of the first video stream, determine the first moving speed of the target object; according to the first moving speed and the motor The ratio between the second moving speeds of the escalator determines the relative speed.

In this way, the relative speed between the target object and the escalator can be determined, and then the behavior events of the target object on the escalator can be determined based on the relative speed, so as to determine whether an abnormal event occurs subsequently.

In a possible implementation manner, the determining the first moving speed of the target object according to the first plane position of the target point in multiple video frames of the first video stream includes: according to multiple first The coordinates of the plane position in the vertical direction are used to determine the first moving speed of the target object in the vertical direction by means of least squares straight line fitting.

In this way, since the first moving speed is the moving speed of the target object in the vertical direction, the first moving speed can be kept consistent with the moving speed of the escalator, thereby making the determined relative speed accurate.

In a possible implementation manner, the method further includes: acquiring a calibration image of the escalator, where the collection area of the calibration image is the same as the collection area of the first video stream; the second area in the calibration image; according to the second image positions of the plurality of vertices of the second area in the calibration image, and the second plane positions of the plurality of vertices in the plane coordinate system, The coordinate correspondence is determined.

In this way, the coordinate correspondence between the image coordinate system and the plane coordinate system can be determined, so that the position coordinates of the escalator and the object can be transformed in the subsequent processing, and the convenience and accuracy of the speed calculation can be improved.

In a possible implementation manner, the method further includes: acquiring a second video stream of the escalator, where the collection area of the second video stream is the same as the collection area of the first video stream, and the electric escalator A calibration object is placed on the escalator; target detection is performed on the second video stream to determine a third area where the calibration object is located in the second video stream; The third image position in the video frame of the second video stream, and the coordinate correspondence, determine the third plane position of the calibration point in the plane coordinate system; The third plane position determines the second moving speed of the escalator in the vertical direction.

In this way, the moving speed of the escalator can be pre-calibrated so that the relative speed can be determined in subsequent processing.

In a possible implementation manner, the determining the event detection result of the target object according to the relative speed includes at least one of the following: when the relative speed is less than a first speed threshold, determining the The event detection result is the occurrence of an abnormal event of the first category; when the relative speed is greater than or equal to the first speed threshold and less than or equal to the second speed threshold, it is determined that the event detection result is that no abnormal event has occurred , the second speed threshold is greater than the first speed threshold; when the relative speed is greater than the second speed threshold and less than or equal to the third speed threshold, determine that the event detection result is the occurrence of the second speed threshold category of abnormal events, the third speed threshold is greater than the second speed threshold; when the relative speed is greater than the third speed threshold, it is determined that the event detection result is the occurrence of a third category of abnormal events.

In this way, abnormal events of retrograde, walking and running can be determined, the accuracy of event detection can be improved, and the cost of human detection can be reduced.

In a possible implementation manner, the method further includes: when the event detection result is the occurrence of at least one of the following abnormal events, sending alarm information corresponding to the category of the abnormal event: the first A category of abnormal events, a second category of abnormal events, the third category of abnormal events.

In this way, when an abnormality is determined, an alarm can be timely so that corresponding processing can be performed, thereby reducing the risk of a security accident.

In a possible implementation manner, the abnormal events of the first category include retrograde events, the abnormal events of the second category include walking events, and the abnormal events of the third category include running events.

According to an aspect of the present disclosure, an event detection apparatus is provided, comprising:

The first video stream acquisition part is configured to acquire the first video stream of the escalator; the first area determination part is configured to perform human body detection on the first video stream, and determine the first video stream where the target object is located. an area; a relative speed determination part configured to determine the relative speed between the target object and the escalator according to the first image position of the target point of the first area in the video frame of the first video stream The speed; the event detection part is configured to determine the event detection result of the target object according to the relative speed.

In a possible implementation manner, the relative speed determination part includes: a position determination sub-part, configured according to the first image position, the image coordinate system of the video frame and the plane coordinates where the escalator is located The coordinate correspondence between the coordinate systems is used to determine the first plane position of the target point in the plane coordinate system; the speed determination sub-section is configured to be based on the first video frame of the first video stream. a plane position to determine a first moving speed of the target object; a relative speed determining subsection configured to determine the relative speed according to the ratio between the first moving speed and the second moving speed of the escalator .

In a possible implementation manner, the speed determination sub-section is configured to: determine the vertical direction of the target object by means of least squares straight line fitting according to the coordinates of the multiple first plane positions in the vertical direction the first movement speed on .

In a possible implementation manner, the device further includes: a calibration image acquisition part, configured to acquire a calibration image of the escalator, where the collection area of the calibration image is the same as the collection area of the first video stream; The second area determination part is configured to determine the second area of the escalator in the calibration image; the coordinate relationship determination part is configured to determine the second area in the calibration image according to the plurality of vertices of the second area The image position, and the second plane position of the plurality of vertices in the plane coordinate system determine the coordinate correspondence.

In a possible implementation manner, the apparatus further includes: a second video stream acquiring part, configured to acquire a second video stream of the escalator, the capture area of the second video stream being the same as the first video stream The collection area of the stream is the same, and a calibration object is placed on the escalator; the third area determination part is configured to perform target detection on the second video stream, and determine the third area where the calibration object is located in the second video stream The position determination part is configured to determine the calibration point in the plane coordinates according to the third image position of the calibration point of the third area in the video frame of the second video stream, and the coordinate correspondence a third plane position in the system; a speed determination part configured to determine the second movement of the escalator in the vertical direction according to the third plane position of the calibration point in the plurality of video frames of the second video stream speed.

In a possible implementation manner, the event detection part is further configured to at least one of the following: when the relative speed is less than a first speed threshold, determine that the event detection result is the occurrence of an abnormal event of the first category ; In the case that the relative speed is greater than or equal to the first speed threshold and less than or equal to the second speed threshold, determine that the event detection result is that no abnormal event has occurred, and the second speed threshold is greater than the first speed threshold a speed threshold; when the relative speed is greater than the second speed threshold and less than or equal to the third speed threshold, it is determined that the event detection result is the occurrence of a second category of abnormal events, and the third speed threshold is greater than the second speed threshold; when the relative speed is greater than the third speed threshold, it is determined that the event detection result is the occurrence of an abnormal event of a third category.

In a possible implementation manner, the apparatus further includes: an alarm part configured to send an alarm corresponding to the category of the abnormal event when the event detection result is the occurrence of at least one of the following abnormal events Information: the first category of abnormal events, the second category of abnormal events, the third category of abnormal events.

According to an aspect of the present disclosure, there is provided an electronic device, comprising: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions implementing the above method when executed by a processor.

According to an aspect of the present disclosure, there is provided a computer program product comprising computer readable code, when the computer readable code is executed on a device, a processor in the device executes instructions for implementing the above event detection method.

According to the embodiments of the present disclosure, the area where the object is located in the video stream can be detected; the relative speed between the object and the escalator can be determined according to the image position of the target point in the area; the event detection result can be determined according to the relative speed, which can improve the The accuracy of event detection, thereby reducing the probability of accidents and reducing the cost of human detection.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings required in the embodiments, which are incorporated into the specification and constitute a part of the specification. The drawings illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

1 shows a flowchart of an event detection method according to an embodiment of the present disclosure;

Figure 2a shows a schematic diagram of the moving direction of the escalator in a plane coordinate system when the moving direction is upward;

Fig. 2b shows a schematic diagram in the case where the moving direction of the escalator is downward in a plane coordinate system;

FIG. 2c is a schematic diagram of a process framework of an event detection method provided by an embodiment of the present disclosure;

3 shows a block diagram of an event detection apparatus according to an embodiment of the present disclosure;

FIG. 4 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

FIG. 5 shows a block diagram of another electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures denote elements that have the same or similar functions. While various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, and may mean including from A, B, and C. Any one or more elements selected from the set of B and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are set forth in the following detailed description. It will be understood by those skilled in the art that the present disclosure may be practiced without certain specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present disclosure.

The event detection method according to the embodiment of the present disclosure can be applied to scenes such as shopping malls, office buildings, and public transportation. Based on the deep learning method, by analyzing the video stream of the area where the escalator is located in the scene, it is estimated that the The speed of movement is compared with the speed of the elevator to determine whether pedestrians have abnormal behaviors such as walking, running or retrograde. When an abnormal behavior occurs, locate the area where the abnormal behavior occurs on the escalator and give an alarm in time for corresponding processing, such as stopping the operation of the escalator, thereby reducing the risk of safety accidents.

FIG. 1 shows a flowchart of an event detection method according to an embodiment of the present disclosure. As shown in FIG. 1 , the event detection method includes:

In step S11, obtain the first video stream of the escalator;

In step S12, human body detection is carried out to the first video stream to determine the first area where the target object is located in the first video stream;

In step S13, the relative speed between the target object and the escalator is determined according to the first image position of the target point of the first area in the video frame of the first video stream;

In step S14, the event detection result of the target object is determined according to the relative speed of the target object.

In a possible implementation manner, the event detection method may be performed by an electronic device such as a terminal device or a server, and the terminal device may be one of the following: a mobile phone (Mobile Phone), a tablet computer (Pad), a wireless transceiver Computers, PDAs, desktop computers, personal digital assistants, portable media players, smart speakers, navigation devices, smart watches, smart glasses, smart necklaces and other wearable devices, pedometers, digital TVs, virtual reality (VR) ) terminal equipment, Augmented Reality (AR) terminal equipment, wireless terminals in Industrial Control, wireless terminals in Self Driving, wireless terminals in Remote Medical Surgery, Wireless terminal in Smart Grid, wireless terminal in Transportation Safety, wireless terminal in Smart City, wireless terminal in Smart Home, vehicle in Internet of Vehicles system , in-vehicle equipment, in-vehicle module, etc., the method can be implemented by the processor calling computer-readable instructions stored in the memory. Alternatively, the method may be performed by a server.

For example, at least one image acquisition device can be installed at the location where the escalator to be detected is located, for example, at least one camera facing the escalator can be installed, so as to collect the video stream of the escalator, and monitor the object riding on the escalator in the video stream. (eg pedestrians) for detection. The present disclosure does not limit the installation position of the image acquisition device, the acquisition method of the video stream, and the specific area corresponding to the video stream.

In a possible implementation manner, in step S11, a video stream of the escalator (referred to as the first video stream) may be acquired, for example, a first video stream uploaded by an image capture device may be received; The stream is decoded to obtain a decoded first video stream (also referred to as a picture stream).

In a possible implementation manner, in step S12, human body detection may be performed on the decoded first video stream to determine the human body frame in each video frame of the video stream; and follow the human body frame in each video frame , determine the body frame of a pedestrian (which may be called a target object) belonging to the same identity, that is, determine the first area where the target object is located in the first video stream.

The method of human body detection may be, for example, human body key point recognition, human body contour detection, etc. The method of human body tracking may be, for example, determining objects belonging to the same identity according to the intersection ratio of human body frames in adjacent video frames. Those skilled in the art should understand that human body detection and follow-up may be implemented in any manner in the related art, which is not limited in the present disclosure.

In a possible implementation manner, human body detection and follow-up may be performed on each video frame of the first video stream; the first video stream may also be sampled at a certain time interval, and human body detection and follow-up may be performed on the sampled video frames ; It can also obtain key frames in the first video stream, and perform human detection and follow-up on the key frames. This disclosure does not limit this.

In a possible implementation manner, after the first area is determined, at least one target point may be determined from the first area, so as to calculate the moving speed of the target object. For example, the vertex of the first region, the center point, or the point on the boundary of the region can be used as the target point, and the present disclosure does not limit the selection of the target point.

In a possible implementation manner, in step S13, the moving speed of the target object may be calculated according to the first image positions of the target point in multiple video frames, for example, according to the multiple position coordinates of the target point, by The least squares straight line fitting method is used to calculate the moving speed of the target object; according to the predetermined moving speed of the escalator and the moving speed of the target object, the relative speed between the target object and the escalator can be determined.

In a possible implementation manner, in step S14, the event detection result of the target object may be determined according to the relative speed. For example, multiple speed thresholds may be preset, and according to the relationship between the relative speed and each speed threshold, it is determined that the event detection result is an abnormal event or no abnormal event occurs.

In a possible implementation manner, the object on the escalator may have abnormal behaviors such as retrograde, walking, or running, and accordingly, the abnormal event may include multiple categories. For example, the abnormal events in the first category include retrograde events, the abnormal events in the second category include walking events, and the abnormal events in the third category include running events. The present disclosure does not limit the specific categories of abnormal events.

In a possible implementation manner, if the event detection result is that an abnormal event occurs, alarm information corresponding to the category of the abnormal event may be generated and sent. The alarm information may include a reminder of the abnormal event, and may also include the area where the target object where the abnormal event occurs, so that the relevant personnel can locate it. The present disclosure does not limit the specific content of the alarm information.

In a possible implementation, for example, alarm information can be sent to the control device of the escalator, so that the control device can stop the operation of the escalator; the alarm information can also be sent to a device that can be seen by the relevant personnel responsible for the operation of the escalator (such as terminal equipment and/or screen display equipment, etc.), so that the relevant personnel can stop the operation of the escalator and go to the escalator for rescue, etc. The present disclosure does not limit the content of the alarm information.

The event detection method of the embodiment of the present disclosure will be described below.

As mentioned above, the video stream of the area where the escalator is located can be captured by the camera, and the captured video stream can be transmitted to electronic devices such as a local front-end server or a cloud server. The electronic device can decode the received video stream to obtain the decoded video stream.

In step S12, the human body detection and follow-up can be performed on the decoded video stream through the detection network, the human body frame in each video frame of the video stream is detected, and the human body frame of the pedestrian belonging to the same identity is followed to obtain the human body frame in the video stream. The first area where the target object is located. The detection network may be a convolutional neural network, and the present disclosure does not limit the network structure of the detection network.

After the first area is obtained, the position of the target point in the first area can be determined so as to calculate the moving speed of the target object.

In the image coordinate system of the video frame, there is a perspective transformation effect (that is, the target in the image is near large and far small), resulting in different moving speeds of the escalator everywhere, which is not convenient for subsequent analysis. Therefore, the plane where the escalator is located can be used. Planar coordinate system for easy subsequent processing. In this case, the coordinate correspondence between the image coordinate system of the video frame and the plane coordinate system of the escalator may be predetermined before the event detection.

In a possible implementation manner, the event detection method according to the embodiment of the present disclosure may further include:

acquiring a calibration image of the escalator, where the collection area of the calibration image is the same as the collection area of the first video stream;

determining a second area of the escalator in the calibration image;

The coordinate correspondence is determined according to the second image positions of the plurality of vertices in the calibration image and the second plane positions of the plurality of vertices in the plane coordinate system.

For example, when there are no pedestrians or there are few pedestrians on the escalator, the image of the escalator can be collected by the camera as the calibration image; or a video frame can be selected from the video stream of the escalator as the calibration image. The collection area of the calibration image is the same as the collection area of the first video stream. The present disclosure does not limit the acquisition method of the calibration image.

In a possible implementation manner, an area (which may be referred to as a second area) of the escalator in the calibration image may be determined. The second area can be marked manually; the second area of the escalator in the calibration image can also be determined by performing area detection on the calibration image through the trained area detection network. The region detection network may be, for example, a convolutional neural network, and the present disclosure does not limit the specific network structure and training method of the region detection network.

In a possible implementation manner, the position coordinates of the multiple vertices of the second region in the calibration image (which may be referred to as the second image position) may be determined, for example, the upper left, upper right, lower left, and lower right of the second region may be determined. The position coordinates of a vertex in the image coordinate system.

In a possible implementation, according to the moving direction of the escalator, the position coordinates (x ₁ , y ₁ ), (x ₂ , y of the four vertices p1 , p2 , p3 and p4 in the image coordinate system can be marked ₂ ), (x ₃ , y ₃ ), (x ₄ , y ₄ ), and set the four vertices p1, p2, p3, p4, the second plane positions in the plane coordinate system of the escalator are: (x′ ₁ ,y′ ₁ )=(0,0),(x′ ₂ ,y′ ₂ )=(w,0),(x′ ₃ ,y′ ₃ )=(0,h),(x ′ ₄ ,y′ ₄ )=(w,h), where w and h are the width and height of the elevator in the plane coordinate system, respectively, and a positive real number can be arbitrarily specified.

Fig. 2a shows a schematic diagram in a case where the moving direction of the escalator in a plane coordinate system is upward, and Fig. 2b shows a schematic diagram in a case where the moving direction of the escalator in a plane coordinate system is downward.

As shown in Fig. 2a and Fig. 2b, in a possible implementation, according to the second image position of the vertices p1, p2, p3, p4 in the image coordinate system, and the vertices p1, p2, p3, p4 in the plane coordinate The second plane position in the system can be used to calculate the perspective transformation matrix (called the H matrix) between the image coordinate system and the plane coordinate system, that is, the coordinate correspondence between the image coordinate system and the plane coordinate system. The perspective transformation matrix has 8 undetermined parameters, and the perspective transformation matrix between the image coordinate system and the plane coordinate system can be uniquely determined through the four vertices p1, p2, p3, and p4.

Before the event detection, the moving speed of the escalator can also be pre-calibrated.

acquiring a second video stream of the escalator, where the collection area of the second video stream is the same as the collection area of the first video stream, and a calibration object is placed on the escalator;

Perform target detection on the second video stream, and determine the third area where the calibration object is located in the second video stream;

According to the third image position of the calibration point of the third area in the video frame of the second video stream, and the coordinate correspondence, determine the third plane position of the calibration point in the plane coordinate system ;

The second moving speed of the escalator in the vertical direction is determined according to the third plane position of the calibration point in the plurality of video frames of the second video stream.

For example, when there are no pedestrians or there are few pedestrians, a calibration object that is stationary relative to the escalator can be placed on the escalator, and a video stream of the escalator (which can be referred to as a second video stream) can be captured by a camera. The collection area of the second video stream is the same as the collection area of the first video stream. The calibration object can be, for example, an item or a staff member, which is not limited in the present disclosure.

In a possible implementation, a video stream in which a person or item is stationary relative to the escalator can also be intercepted from the first video stream of the escalator, as the second video stream; and the person or item is used as the second video stream. calibrator. The present disclosure does not limit the acquisition method of the second video stream.

In a possible implementation manner, the trained target detection network may be used to perform target detection on at least two video frames of the second video stream to determine the area of the calibration object in the video frame (which may be referred to as the third area). ). The target detection network may be, for example, a convolutional neural network, and the present disclosure does not limit the specific network structure and training method of the target detection network.

In a possible implementation manner, a calibration point in the third area may be determined, and the calibration point may be, for example, a vertex, a center point, or a point on the boundary of the third area, and the present disclosure does not limit the selection of the calibration point .

In a possible implementation, set the image coordinates (called the third image position) of the calibration point in the i-th video frame and the j-th video frame of the second video stream, respectively (x _i , y _i ) and (x _j , y _j ). According to the coordinate correspondence determined above (ie H matrix), the position coordinates (called the third plane position) of the calibration point in the plane coordinate system can be determined as (x′ _i , y′ _i ) and (x′ _j , _y'j ).

In a possible implementation, assuming that the moving speed of the escalator is constant and constant, then according to the vertical position in the third plane position (x′ _i , y′ _i ) and (x′ _j , y′ _j ) Coordinates, the second moving speed v _y0 of the escalator in the vertical direction is determined by the following formula (1):

v _y0 =(y′ _j -y′ _i )/(ji) (1);

After the first area is obtained in step S12, the position of the target point in the first area can be determined so as to calculate the moving speed of the target object.

Considering that the body of the target object may move, resulting in position changes; and the feet of the target object usually fall on the elevator plane, and the position is more accurate. Therefore, the midpoint of the bottom edge of the first area can be used as the target point to improve Accuracy of movement speed calculations.

In a possible implementation, set the coordinates of the lower left vertex and the lower right vertex of the first region in the image coordinate system to be (x _bl , y _bl ) and (x _br , y _br ) respectively, then the target point is in the image The coordinates in the coordinate system can be expressed as (x _c , y _c )=((x _b1 +x _br )/2, (y _bl +y _br )/2), that is, the first image position.

In a possible implementation, step S13 may include:

Determine the first plane of the target point in the plane coordinate system according to the first image position and the coordinate correspondence between the image coordinate system of the video frame and the plane coordinate system where the escalator is located Location;

determining the first moving speed of the target object according to the first plane position of the target point in the multiple video frames of the first video stream;

The relative speed is determined according to the ratio between the first moving speed and the second moving speed of the escalator.

For example, coordinate transformation can be performed on the first image position (x _c , y _c ) of the target point according to the coordinate correspondence between the image coordinate system of the video frame and the plane coordinate system where the escalator is located (ie, the H matrix). , obtain the first plane position (x′ _c , y′ _c ) of the target point in the plane coordinate system. The first moving speed may be determined according to the first plane position of the target point of the target object in the plurality of video frames.

In a possible implementation manner, the step of determining the first moving speed of the target object according to the first plane position of the target point in multiple video frames of the first video stream may include:

According to the coordinates of a plurality of first plane positions in the vertical direction, the first moving speed of the target object in the vertical direction is determined by means of least squares straight line fitting.

That is to say, coordinate transformation can be performed respectively according to the first image position of the target point in the video frame at the current moment and the multiple video frames (for example, 30 video frames) before the current moment to obtain a plurality of first plane positions; According to the ordinates y′ _c of the multiple first plane positions, the first moving speed _vy of the target object in the vertical direction at the current moment can be estimated by means of least squares straight line fitting. In this way, the accuracy of the velocity calculation can be improved.

It should be understood that the first moving speed may also be estimated in other manners, which is not limited in the present disclosure.

In a possible implementation manner, the relative speed v _r between the target object and the escalator is determined according to the ratio between the first moving speed and the second moving speed of the escalator.

Since both the first moving speed and the second moving speed are relative values, the speed ratio method can improve the accuracy of subsequent determination. It should be understood that the relative speed between the target object and the escalator may also be determined in other manners, which is not limited in the present disclosure.

In a possible implementation manner, in step S14, the event detection result of the target object is determined according to the relative speed, including at least one of the following:

In the case that the relative speed is less than the first speed threshold, determining that the event detection result is the occurrence of an abnormal event of the first category;

When the relative speed is greater than or equal to the first speed threshold and less than or equal to a second speed threshold, it is determined that the event detection result is that no abnormal event has occurred, and the second speed threshold is greater than the first speed threshold speed threshold;

In the case where the relative speed is greater than the second speed threshold and less than or equal to a third speed threshold, determine that the event detection result is the occurrence of a second category of abnormal events, the third speed threshold being greater than the third speed threshold Two speed thresholds;

In the case that the relative speed is greater than the third speed threshold, it is determined that the event detection result is the occurrence of an abnormal event of a third category.

For example, for the abnormal behaviors such as retrograde, walking, or running that may occur to objects on the escalator, it can be set that the abnormal events of the first category include retrograde events, the abnormal events of the second category include walking events, and the third category of abnormal events can be set. The abnormal events include running events, etc. The present disclosure does not limit the specific categories of abnormal events.

In a possible implementation manner, according to multiple categories of abnormal events, multiple speed thresholds may be set respectively, and the event detection result is determined according to the relationship between the relative speed and each speed threshold. For example, a first speed threshold Thresh _back , a second speed threshold Thresh _walk and a third speed threshold Thresh _run may be set. The second speed threshold is greater than the first speed threshold and the third speed threshold is greater than the second speed threshold.

The first speed threshold may be, for example, -0.9, the second speed threshold may be, for example, 0.9, and the third speed threshold may be, for example, 1.8. The present disclosure does not limit the specific value of each speed threshold.

Table 1 shows the correspondence between the relative speed and the determination result.

Table 1

相对速度Relative velocity	判定结果judgement result
v _r＜Thresh _back v _r <Thresh _back	逆行retrograde
Thresh _back≤v _r≤Thresh _walk Thresh _back ≤v _r ≤Thresh _walk	静止still
Thresh _walk＜v _r≤Thresh _run Thresh _walk <v _r ≤Thresh _run	行走walk
Thresh _run＜v _r Thresh _run <v _r	跑动run

In a possible implementation, as shown in Table 1, if the relative speed v _r is less than the first speed threshold Thresh _back , it can be considered that the moving direction of the target object is opposite to that of the escalator, and the reverse speed is relatively large. In this case, it can be determined that the target object has retrograde behavior, and the event detection result is determined to be a retrograde event.

In a possible implementation manner, as shown in Table 1, if the relative speed v _r is greater than or equal to the first speed threshold Thresh _back and less than or equal to the second speed threshold Thresh _walk , it can be considered that the target object is relative to the escalator speed is smaller. In this case, it can be determined that the target object is stationary on the escalator, and it can be determined that the event detection result is that no abnormal event has occurred.

In a possible implementation manner, as shown in Table 1, if the relative speed v _r is greater than the second speed threshold Thresh _walk and less than or equal to the third speed threshold Thresh _run , it can be considered that the target object is moving forward on the escalator walk. In this case, it is determined that the event detection result is the occurrence of a walking event.

In a possible implementation manner, as shown in Table 1, if the relative speed v _r is greater than the third speed threshold Thresh _run , it can be considered that the target object is running forward on the escalator. In this case, it is determined that the event detection result is the occurrence of a running event.

In a possible implementation manner, when the relative velocity is within the corresponding threshold interval and lasts for a certain period of time, it can be determined that an abnormal event has occurred, so as to avoid possible false detections during a single relative velocity determination. The present disclosure does not limit the specific determination method.

In a possible implementation manner, after the event detection result is determined in step S14, the event detection method according to the embodiment of the present disclosure may further include:

When the event detection result is that an abnormal event of at least one of the following occurs, alarm information corresponding to the category of the abnormal event is sent: the abnormal event of the first category, the abnormal event of the second category, the A third category of abnormal events.

That is, if the event detection result is that an abnormal event occurs, alarm information corresponding to the category of the abnormal event, that is, alarm information corresponding to a retrograde event, a walking event, or a running event may be generated and sent.

In a possible implementation manner, alarm information may also be generated and sent when some types of abnormal events occur. For example, only when a retrograde event occurs, the alarm information is sent; or, when a retrograde event or a running event occurs, the alarm information is sent. The present disclosure does not limit the sending conditions of the alarm information.

In a possible implementation manner, the alarm information may include a reminder of an abnormal event, and may also include an area where the target object where the abnormal event occurs, so that relevant personnel can locate. The present disclosure does not limit the specific content of the alarm information.

FIG. 2c is a schematic diagram of a process framework of an event detection method provided by an embodiment of the present disclosure. As shown in FIG. 2c, the process framework includes a pedestrian detection and following module, a walking and running retrograde recognition module, and a squatting recognition module.

After the input video stream is decoded, it is input to the pedestrian detection and follow-up module, and the first area of each pedestrian is obtained (represented as the vertex coordinates of the upper left corner and the lower right corner of each pedestrian detection frame (x ₁ , y ₁ ), (x ₂ , y ₂ ), and the follower ID of the corresponding pedestrian). The pedestrian detection frame can be a rectangular frame. Wherein, pedestrian detection is usually implemented by a deep neural network, and pedestrian following algorithms include algorithms based on deep learning, algorithms based on IoU matching, etc. The embodiments of the present disclosure do not limit which detection and following algorithms are used.

The posture recognition result of the target object can be obtained by performing posture recognition on the image of the first area corresponding to the first area through the trained posture recognition network, that is, the posture recognition result is output through the posture recognition network. For example, the gesture recognition network may be a squatting recognition module, so that the squatting event is output through the squatting recognition module.

In this embodiment of the present disclosure, the walking-running retrograde recognition module can use video streams to analyze pedestrian movement speed, compare it with the elevator speed, and then determine whether the pedestrian has running or retrograde behavior, such as outputting running and retrograde events. The walking and running retrograde recognition module can be used to realize steps S13 and S14: according to the first image position of the target point of the first area in the video frame of the first video stream, determine the target object and the escalator The relative speed between the two; according to the relative speed, the event detection result of the target object is determined.

According to the event detection method of the embodiment of the present disclosure, the area where the object is located in the video stream can be detected, and the transformation of the point in the area between the image coordinate system of the video frame and the elevator plane coordinate system can be realized. The speed of pedestrians is estimated to determine whether there are dangerous actions such as pedestrians retrograde, walking, running, etc., and timely warning can improve the accuracy of event detection and reduce the risk of safety accidents.

According to the event detection method of the embodiment of the present disclosure, event detection can be realized only by using a monocular camera without other sensors (such as infrared sensors, etc.); Additional strict calibration can be adapted to escalator elevator equipment with different speed configurations, which effectively improves the efficiency of event detection.

The event detection method according to the embodiment of the present disclosure can be applied to the field of security monitoring, and can be applied to escalator pedestrian dangerous behavior detection and alarm related products, for example, escalator self-service monitoring systems deployed in large shopping malls, supermarkets, subway stations, office buildings and other application scenarios , to realize automatic detection and alarm of dangerous behaviors such as pedestrians retrograde, walking, running, etc. on the elevator, so as to take measures to slow down the elevator or stop the elevator, and reduce the cost of human monitoring.

It can be understood that the above-mentioned method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle and logic. Those skilled in the art can understand that, in the above method of the specific embodiment, the specific execution order of each step should be determined by its function and possible internal logic.

In addition, the present disclosure also provides an event detection device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any event detection method provided by the present disclosure. For the corresponding technical solutions and descriptions, refer to the corresponding records in the Methods section. ,No longer.

FIG. 3 shows a block diagram of an event detection apparatus according to an embodiment of the present disclosure. As shown in FIG. 3 , the apparatus includes:

The first video stream acquiring part 31 is configured to acquire the first video stream of the escalator;

The first area determination part 32 is configured to perform human body detection on the first video stream, and determine the first area where the target object is located in the first video stream;

a relative speed determination part 33, configured to determine the relative speed between the target object and the escalator according to the first image position of the target point of the first area in the video frame of the first video stream;

The event detection part 34 is configured to determine the event detection result of the target object according to the relative speed.

In a possible implementation manner, the velocity determination sub-section is configured to: determine the vertical direction of the target object by a least squares line fitting method according to the coordinates of a plurality of first plane positions in the vertical direction. The first movement speed in the direction.

In some embodiments, the functions or included parts of the apparatus provided in the embodiments of the present disclosure may be configured to execute the methods described in the above method embodiments, and the specific implementation may refer to the descriptions in the above method embodiments. No longer.

Embodiments of the present disclosure further provide a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

An embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

Embodiments of the present disclosure also provide a computer program product, including computer-readable codes. When the computer-readable codes are run on a device, a processor in the device executes a method for implementing the event detection method provided by any of the above embodiments. instruction.

Embodiments of the present disclosure further provide another computer program product for storing computer-readable instructions, which, when executed, cause a computer to perform operations of the event detection method provided by any of the foregoing embodiments.

The electronic device may be provided as a terminal, server or other form of device.

FIG. 4 shows a block diagram of an electronic device according to an embodiment of the present disclosure. For example, electronic device 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc. terminal.

4, the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814 , and the communication component 816 .

The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to perform all or some of the steps of the methods described above. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

Memory 804 is configured to store various types of data to support operation at electronic device 800 . Examples of such data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. Memory 804 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power supply assembly 806 provides power to various components of electronic device 800 . Power supply components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800 .

Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front-facing camera and/or a rear-facing camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.

Audio component 810 is configured to output and/or input audio signals. For example, audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when electronic device 800 is in operating modes, such as calling mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of electronic device 800 . For example, the sensor assembly 814 can detect the on/off state of the electronic device 800, the relative positioning of the components, such as the display and the keypad of the electronic device 800, the sensor assembly 814 can also detect the electronic device 800 or one of the electronic device 800 Changes in the position of components, presence or absence of user contact with the electronic device 800 , orientation or acceleration/deceleration of the electronic device 800 and changes in the temperature of the electronic device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as wireless network (WiFi), second generation mobile communication technology (2G) or third generation mobile communication technology (3G), or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmed gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation is used to perform the above method.

In an exemplary embodiment, a non-volatile computer-readable storage medium, such as a memory 804 comprising computer program instructions executable by the processor 820 of the electronic device 800 to perform the above method is also provided.

FIG. 5 shows a block diagram of another electronic device according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. 5, electronic device 1900 includes processing component 1922, which further includes one or more processors, and a memory resource represented by memory 1932 for storing instructions executable by processing component 1922, such as applications. An application program stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Additionally, the processing component 1922 is configured to execute instructions to perform the above-described methods.

The electronic device 1900 may also include a power supply assembly 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input output (I/O) interface 1958 . The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as a Microsoft server operating system (Windows Server ^™ ), a graphical user interface based operating system (Mac OS X ^™ ) introduced by Apple, a multi-user multi-process computer operating system (Unix ^™ ), Free and Open Source Unix-like Operating System (Linux ^™ ), Open Source Unix-like Operating System (FreeBSD ^™ ) or the like.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as memory 1932 comprising computer program instructions executable by processing component 1922 of electronic device 1900 to perform the above-described method.

The present disclosure may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present disclosure.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above. Computer-readable storage media, as used herein, are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.

The computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for carrying out operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages. Source or object code, written in any combination, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect). In some embodiments, custom electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be personalized by utilizing state information of computer readable program instructions. Computer readable program instructions are executed to implement various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.

The computer program product can be specifically implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Various embodiments of the present disclosure have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein.

Industrial Applicability

Embodiments of the present disclosure disclose an event detection method, device, electronic device, storage medium and program product, wherein the event detection method includes: acquiring a first video stream of an escalator; performing human body detection on the first video stream , determine the first area where the target object is located in the first video stream; according to the first image position of the target point of the first area in the video frame of the first video stream, determine the relationship between the target object and the target object. relative speed between the escalators; according to the relative speed, determine the event detection result of the target object. Through this method, the accuracy of event detection can be improved, thereby reducing the probability of an accident and reducing the cost of human detection.

Claims

An event detection method comprising:

Get the first video stream of the escalator;

Perform human body detection on the first video stream, and determine the first area where the target object is located in the first video stream;

determining the relative speed between the target object and the escalator according to the first image position of the target point of the first area in the video frame of the first video stream;

According to the relative speed, an event detection result of the target object is determined.
The method according to claim 1, wherein the determination of the relationship between the target object and the escalator is performed according to the first image position of the target point of the first area in the video frame of the first video stream. relative velocities between , including:

Determine the first plane of the target point in the plane coordinate system according to the first image position and the coordinate correspondence between the image coordinate system of the video frame and the plane coordinate system where the escalator is located Location;

determining the first moving speed of the target object according to the first plane position of the target point in the multiple video frames of the first video stream;

The relative speed is determined according to the ratio between the first moving speed and the second moving speed of the escalator.
The method according to claim 2, wherein the determining the first moving speed of the target object according to the first plane position of the target point in multiple video frames of the first video stream comprises:

According to the coordinates of a plurality of first plane positions in the vertical direction, the first moving speed of the target object in the vertical direction is determined by means of least squares straight line fitting.
The method according to claim 2 or 3, wherein the method further comprises:

acquiring a calibration image of the escalator, where the collection area of the calibration image is the same as the collection area of the first video stream;

determining a second area of the escalator in the calibration image;

The coordinate correspondence is determined according to the second image positions of the plurality of vertices in the calibration image and the second plane positions of the plurality of vertices in the plane coordinate system.
The method according to any one of claims 2-4, wherein the method further comprises:

acquiring a second video stream of the escalator, where the collection area of the second video stream is the same as the collection area of the first video stream, and a calibration object is placed on the escalator;

performing target detection on the second video stream to determine the third area where the calibration object is located in the second video stream;

According to the third image position of the calibration point of the third area in the video frame of the second video stream, and the coordinate correspondence, determine the third plane position of the calibration point in the plane coordinate system ;

The second moving speed of the escalator in the vertical direction is determined according to the third plane position of the calibration point in the plurality of video frames of the second video stream.
The method according to any one of claims 1-5, wherein the determining the event detection result of the target object according to the relative speed includes at least one of the following:

In the case that the relative speed is less than the first speed threshold, determining that the event detection result is the occurrence of an abnormal event of the first category;

When the relative speed is greater than or equal to the first speed threshold and less than or equal to a second speed threshold, it is determined that the event detection result is that no abnormal event has occurred, and the second speed threshold is greater than the first speed threshold speed threshold;

In the case where the relative speed is greater than the second speed threshold and less than or equal to a third speed threshold, determine that the event detection result is the occurrence of a second category of abnormal events, the third speed threshold being greater than the third speed threshold Two speed thresholds;

In the case that the relative speed is greater than the third speed threshold, it is determined that the event detection result is the occurrence of an abnormal event of a third category.
The method of claim 6, wherein the method further comprises:

When the event detection result is that an abnormal event of at least one of the following occurs, alarm information corresponding to the category of the abnormal event is sent: the abnormal event of the first category, the abnormal event of the second category, the A third category of abnormal events.
7. The method of claim 6 or 7, wherein the first category of abnormal events includes retrograde events, the second category of abnormal events includes walking events, and the third category of abnormal events includes running events.
An event detection device, comprising:

a first video stream acquiring part, configured to acquire the first video stream of the escalator;

a first area determination part, configured to perform human body detection on the first video stream, and determine the first area where the target object is located in the first video stream;

a relative speed determination part, configured to determine the relative speed between the target object and the escalator according to the first image position of the target point of the first area in the video frame of the first video stream;

The event detection part is configured to determine the event detection result of the target object according to the relative speed.
10. The apparatus of claim 9, wherein the relative velocity determination section comprises:

a position determination sub-section, configured to determine the target point on the plane according to the first image position and the coordinate correspondence between the image coordinate system of the video frame and the plane coordinate system where the escalator is located the position of the first plane in the coordinate system;

a speed determination subsection, configured to determine the first moving speed of the target object according to the first plane position of the target point in the multiple video frames of the first video stream;

A relative speed determination subsection is configured to determine the relative speed according to a ratio between the first moving speed and the second moving speed of the escalator.
The device according to claim 10, wherein the speed determination sub-section is configured to: determine the vertical position of the target object by means of least squares straight line fitting according to the coordinates of the plurality of first plane positions in the vertical direction. The first movement speed in the straight direction.
The apparatus of claim 10 or 11, wherein the apparatus further comprises:

a calibration image acquisition part, configured to acquire a calibration image of the escalator, and the collection area of the calibration image is the same as the collection area of the first video stream;

A second area determination part configured to determine a second area of the escalator in the calibration image;

The coordinate relationship determining part is configured to determine the second image position of the plurality of vertices in the calibration image according to the second image position of the plurality of vertices in the second region and the second plane position of the plurality of vertices in the plane coordinate system The coordinate relationship described above.
The apparatus according to any one of claims 10-12, wherein the apparatus further comprises:

The second video stream acquisition part is configured to acquire a second video stream of the escalator, the collection area of the second video stream is the same as the collection area of the first video stream, and a calibration object is placed on the escalator ;

A third area determining part, configured to perform object detection on the second video stream, and determine a third area where the calibration object is located in the second video stream;

a position determination part, configured to determine that the calibration point is in the plane coordinate system according to the third image position of the calibration point of the third region in the video frame of the second video stream and the coordinate correspondence the third plane position in ;

The speed determination part is configured to determine the second moving speed of the electric escalator in the vertical direction according to the third plane position of the calibration point in the plurality of video frames of the second video stream.
The device according to any one of claims 9-13, wherein the event detection part is further configured as at least one of the following:

In the case that the relative speed is less than the first speed threshold, determining that the event detection result is the occurrence of an abnormal event of the first category;

When the relative speed is greater than or equal to the first speed threshold and less than or equal to a second speed threshold, it is determined that the event detection result is that no abnormal event has occurred, and the second speed threshold is greater than the first speed threshold speed threshold;

In the case where the relative speed is greater than the second speed threshold and less than or equal to a third speed threshold, determine that the event detection result is the occurrence of a second category of abnormal events, the third speed threshold being greater than the third speed threshold Two speed thresholds;

In the case that the relative speed is greater than the third speed threshold, it is determined that the event detection result is the occurrence of an abnormal event of a third category.
The apparatus of claim 14, wherein the apparatus further comprises:

The alarm part is configured to send alarm information corresponding to the category of the abnormal event when the event detection result is an abnormal event of at least one of the following: the abnormal event of the first category, the abnormal event of the second category Anomalies, anomalies of the third category.
15. The apparatus of claim 14 or 15, wherein the first category of abnormal events includes retrograde events, the second category of abnormal events includes walking events, and the third category of abnormal events includes running events.
An electronic device comprising:

processor;

memory configured to store processor executable instructions;

wherein the processor is configured to invoke the memory-stored instructions to perform the method of any one of claims 1-8.
A computer-readable storage medium having computer program instructions stored thereon, the computer program instructions implementing the method of any one of claims 1 to 8 when executed by a processor.
A computer program product comprising computer readable code, when the computer readable code is run on a device, a processor in the device executes instructions for implementing the event detection method of any one of claims 1 to 8.