KR20220125716A - Event detection methods, devices, electronic devices, storage media and program products - Google Patents

Abstract

The present invention relates to an event detection method, apparatus, electronic device, storage medium and program product, the event detection method comprising: acquiring a video stream of an escalator; determining a first region in the video stream where a target object is located by performing human body detection on the video stream; performing pose recognition on a first area image corresponding to the first area to obtain a pose recognition result of the target object, wherein the pose recognition result includes reliability that the target pose is an abnormal pose; and determining an event detection result of the target target according to the pose recognition result of the target target.

Description

Event detection methods, devices, electronic devices, storage media and program products

Cross-referencing of related applications

This application has a Chinese application number 202110236981.X filed on March 3, 2021, the applicant is Shanghai SenseTime Smart Technology Company Limited, and the title of the invention is "Event Detection Method and Apparatus, Electronic Device and Storage Medium" The priority of the application is claimed, and the entire contents of the application are incorporated herein by reference.

The present invention relates to the field of computer vision technology, and more particularly to an event detection method, apparatus, electronic device, storage medium and program product.

With the development of the economy and the continuous improvement of infrastructure construction, the application of escalators in scenarios such as department stores, officetels, and public transportation is also getting wider. While escalators provide convenience to pedestrians, pedestrians are increasingly interested in other accidents that may be caused by abnormal movements on the escalator such as, for example, falling, backing, squatting, and the like. If an abnormal operation in the escalator is not detected and warned in time to stop the operation of the escalator in time, it may seriously damage the life and property safety of pedestrians on the escalator.

The present invention provides an event detection method, apparatus, electronic device, storage medium and program product.

According to one aspect of the present invention, there is provided an event detection method, the event detection method comprising: acquiring a video stream of an escalator; determining a first region in the video stream where a target object is located by performing human body detection on the video stream; performing pose recognition on a first area image corresponding to the first area to obtain a pose recognition result of the target object, wherein the pose recognition result includes reliability that the target pose is an abnormal pose; and determining an event detection result of the target target according to the pose recognition result of the target target.

In a possible implementation form, the determining of the event detection result of the target object according to the pose recognition result of the target object includes: when the reliability of the target object at a current time is greater than or equal to a reliability threshold, the target object and determining that the event detection result is an abnormal event occurrence.

Through this form, it is possible to directly determine that an abnormal event has occurred when the target object is in an abnormal pose, thereby improving the real-time of event detection, thereby providing timely warning and reducing the risk of safety accidents.

In a possible implementation form, the determining of the event detection result of the target object according to the pose recognition result of the target object includes: a reliability of the target object at a current time is greater than or equal to a reliability threshold value, and the reliability is equal to a reliability threshold value and determining that the event detection result of the target object is an abnormal event when the first duration greater than or equal to the threshold is greater than or equal to the activation time.

Through this form, when the target object is in an abnormal pose and persists for a certain period of time, it is determined that an abnormal event has occurred, thereby reducing false detections that may occur during single-frame detection, thereby improving the accuracy of event detection. .

In a possible implementation form, the event detection method further includes transmitting first warning information when the event detection result of the target object is that an abnormal event has occurred.

Through this form, timely warning of abnormal events can be implemented, thereby reducing the risk of safety accidents.

In a possible implementation form, the event detection method may include: changing a detection state of the target object to a detected state when the event detection result of the target object is that an abnormal event has occurred; The detection state of the target object is a detected state, the reliability of the target object at the current time is greater than or equal to a reliability threshold, and a second duration in which the reliability is greater than or equal to the reliability threshold is greater than the cooling time. in this case, determining that the event detection result of the target object is an abnormal event continuation; and transmitting second warning information when the event detection result of the target object is an abnormal event continuation.

Through this form, if the target object is in an abnormal pose and the cooling time is reached, it is determined as an abnormal event continuation, repeatedly warning at the cooling time interval, repeatedly to the monitoring device operated by the escalator or the relevant personnel responsible for the operation of the escalator. By notifying this, it is possible to reduce the risk of safety accidents by reducing the occurrence of cases in which the relevant personnel fail to observe the warning information.

In a possible implementation form, the event detection method includes: a first detection state of the target object is the detected state, a reliability of the target object at a current time is less than a reliability threshold, and a reliability of the target object is less than the reliability threshold. The method further includes determining that the event detection result of the target object is the end of the abnormal event when the 3 duration is greater than the inactivation time.

Through this form, when the target object is not in an abnormal pose and the inactivation time is reached, it is determined as an abnormal event end, thereby completing the entire processing flow of event detection.

In a possible implementation form, the determining of the first region image corresponding to the first region may include: performing expansion on the first region according to a preset expansion coefficient to obtain a second region; and cropping the image of the second region from the video frame of the video stream to obtain the image of the first region.

Through this form, the classifier performance of the pose recognition network can be improved, and the accuracy of subsequent pose recognition can be improved.

In a possible implementation form, the performing pose recognition on the first region image corresponding to the first region includes: performing pose recognition on the first region image corresponding to the first region through a pose recognition network comprising; The event detection method includes: training according to a preset training set to obtain the pose recognition network; in the training set, target poses tagged by sample images include abnormal poses, normal poses, and other poses, and the other poses include: includes poses excluding normal poses and abnormal poses.

Through this form, it is possible to improve the robustness of the pose recognition and improve the accuracy of the pose recognition.

In a possible implementation form, the abnormal event comprises a squat event.

According to one aspect of the present invention, there is provided an event detecting apparatus, the event detecting apparatus comprising: a video stream obtaining part, configured to obtain a video stream of an escalator; a region determining portion configured to perform human body detection on the video stream to determine a first region in the video stream where a target object is located; A pose recognition part configured to obtain a pose recognition result of the target object by performing pose recognition on a first area image corresponding to the first area - Reliability that the pose of the target object is an abnormal pose in the pose recognition result contains - ; and an event detection part, configured to determine an event detection result of the target object according to the pose recognition result of the target object.

In a possible implementation form, the event detection part is further configured to determine that the event detection result of the target object is an abnormal event occurrence, when the reliability of the target object at the current time is greater than or equal to a reliability threshold value.

In a possible implementation form, the event detection part is further configured that the target object has a confidence at a current time greater than or equal to a confidence threshold, and a first duration in which the confidence is greater than or equal to the confidence threshold is greater than an activation time, or In the same case, the event detection result of the target object is configured to determine that an abnormal event occurs.

In a possible implementation form, the event detection device further includes a first warning part configured to transmit first warning information when the event detection result of the target object is that an abnormal event has occurred.

In a possible implementation form, the event detection device further includes a state change part configured to change the detection state of the target object to the detected state when the event detection result of the target object is that an abnormal event has occurred, , the event detection part is further configured to include, when the reliability of the target object at the current time is greater than or equal to a reliability threshold, and a second duration in which the reliability is greater than or equal to the reliability threshold is greater than a cooling time, the target the event detection result of the subject is configured to determine that the abnormal event persists; The event detection apparatus further includes a first warning part configured to transmit second warning information when the event detection result of the target object is an abnormal event continuation.

In a possible implementation form, the event detection part further comprises: the detection state of the target object is the detected state, the reliability of the target object at a current time is less than a reliability threshold, and the reliability is less than the reliability threshold. and when the third duration is greater than the inactivation time, the event detection result of the target object is configured to determine that the abnormal event ends.

In a possible implementation form, the event detection apparatus may include: an expansion part configured to perform expansion on the first area according to a preset expansion coefficient to obtain a second area; and a cropping portion, configured to crop the image of the second region from the video frame of the video stream to obtain the image of the first region.

In a possible implementation form, the pose recognition part is further configured to perform pose recognition on a first area image corresponding to the first area through a pose recognition network; wherein the event detection device includes a training part, configured to obtain the pose recognition network by training according to a preset training set, wherein the target pose tagged by the sample image in the training set includes an abnormal pose, a normal pose and another pose; , the other poses include poses other than normal poses and abnormal poses.

In a possible implementation form, the abnormal event comprises a squat event.

According to one aspect of the present invention, there is provided an electronic device, the electronic device comprising: a processor; and a memory configured to store instructions executable by the processor; Here, the processor is configured to execute the event detection method by calling the instruction stored in the memory.

According to an aspect of the present invention, there is provided a computer-readable storage medium storing computer program instructions, and when the computer program instructions are executed by a processor, the event detection method is implemented.

According to one aspect of the present invention, there is provided a computer program product comprising computer readable code, wherein when the computer readable code is operated in a device, a processor in the device executes instructions for implementing the event detection method .

According to an embodiment of the present invention, detecting an area in a video stream where an object is located; determine the pose of the subject by performing recognition on the image of the region; By determining the event detection result according to the pose of the target, and improving the accuracy of event detection, it is possible to lower the probability of other accidents and reduce the cost of manpower detection.

It should be understood that the above general description and the detailed description below are merely exemplary and interpretative, and are not intended to limit the present invention. Other features and aspects of the present invention become apparent when the following reference drawings are detailed descriptions of exemplary embodiments.

In order to more clearly explain the technical solutions of the embodiments of the present invention, the drawings to be used in the embodiments will be briefly described below, wherein the drawings are incorporated in and constitute a part of the specification, and these drawings represent the present invention. It shows an embodiment conforming to the above, and is intended to explain the technical solution of the present invention together with the specification. It should be understood that the drawings below only show some embodiments of the present invention and should not be regarded as limiting in scope, and those of ordinary skill in the art would appreciate these drawings, even under the premise that creative labor is not imparted. It is possible to obtain other related drawings according to
1 is a flowchart of an event detection method shown according to an embodiment of the present invention.
2A is an exemplary diagram illustrating a change in a detection state of a target object according to an embodiment of the present invention.
2B is an exemplary flow frame diagram of an event detection method provided in an embodiment of the present invention.
3 is a block diagram of an event detection apparatus according to an embodiment of the present invention.
4 is a block diagram of an electronic device according to an embodiment of the present invention.
5 is a block diagram of another electronic device shown according to an embodiment of the present invention.

Various exemplary embodiments, features and aspects of the present invention will be described in detail below with reference to the drawings. In the drawings, the same reference numerals indicate components having the same or similar functions. Although various aspects of the embodiments are illustrated in the drawings, they need not be drawn to scale unless specifically indicated.

As used herein, the word “exemplary” means “used as an example, embodiment, or descriptive”. Any embodiment described herein as “exemplary” is not necessarily to be construed as superior or superior to another embodiment.

As used herein, the term "and/or" merely describes the association of the associated object, and indicates that three relationships may exist, for example, A and/or B, where A exists alone, A and There are three situations in which B exists simultaneously and B exists alone. In addition, the term "at least one" in the text indicates any one in the plural or the combination of any one of at least two in the plural, for example, including at least one of A, B and C is, A, B , C may indicate including any one or a plurality of elements selected from the set.

Furthermore, in order to better illustrate the present invention, numerous specific details are provided in the specific embodiments that follow. A person skilled in the art should understand that the present invention may be practiced without some specific details. In some practical instances, detailed descriptions of methods, means, components, and circuits that are familiar to those skilled in the art are not described in order to highlight the subject matter of the present invention.

According to the event detection method of the embodiment of the present invention, it can be applied to scenarios such as department stores, officetels, public transportation, etc., and based on the method of deep learning, by performing processing and analysis on the video stream of the area where the escalator is located in the scenario , real-time monitoring of abnormal events (e.g., falling, backing, squatting, etc.) occurring on objects (e.g. pedestrians) By stopping, the risk of safety accidents can be lowered.

1 is a flowchart of an event detection method illustrated in accordance with an embodiment of the present invention, and as illustrated in FIG. 1 , the event detection method includes the following steps.

In step S11, the video stream of the escalator is obtained.

In step S12, human body detection is performed on the video stream to determine a first area in which a target object is located in the video stream.

In step S13, pose recognition is performed on the image of the first area corresponding to the first area to obtain a pose recognition result of the target object, and the pose recognition result has a reliability that the pose of the target object is an abnormal pose. Included.

In step S14, an event detection result of the target object is determined according to the pose recognition result of the target object.

In a possible implementation form, the event detection method may be executed by an electronic device such as a terminal device or a server, and the terminal device is a mobile phone, a tablet computer (Pad), a computer having a wireless transmission/reception function, a palmtop computer, and a desktop computer. , personal information terminal, portable media player, smart speaker, navigation device, smart watch, smart glasses, smart necklace, wearable device, pedometer, digital TV, virtual reality (VR) terminal device, augmented reality (AR) Terminal device, wireless terminal in Industrial Control, wireless terminal in Self Driving, wireless terminal in Remote Medical Surgery, wireless terminal in Smart Grid, transportation safety It may be one of a wireless terminal in (Transportation Safety), a wireless terminal in a smart city, a wireless terminal in a smart home, a vehicle in a vehicle Internet system, a vehicle-mounted device, and a vehicle-mounted module. , the event detection method may be implemented through a form in which a processor calls a computer readable instruction stored in a memory. Alternatively, the event detection method may be executed through the server.

For example, at the location where the escalator to be detected is located, for example, by installing at least one image acquisition device such as a camera facing the escalator, the video stream of the escalator is collected, and the object riding on the escalator (e.g., For example, pedestrians) can be detected. The present invention is not limited to the installation location of the image acquisition device, the collection type of the video stream, and the specific area corresponding to the video stream.

In a possible implementation form, in step S11, a video stream of the escalator is obtained, for example, a video stream uploaded by an image collecting device is received; Decoding may be performed on the collected video stream to obtain a decoded video stream (which may also be referred to as an image stream).

In a possible implementation form, in step S12, performing human body detection on the decoded video stream to determine a human body box in each video frame of the video stream; Tracking can be performed on the body box in each video frame to determine the body box of a pedestrian (which may be referred to as a target object) belonging to the same identity, i.e. the first area in the video stream where the target object is located. have.

Here, the form of human body detection may be, for example, human key point recognition, human body contour detection, and the like; The form of body tracking may be to determine objects belonging to the same identity, for example, according to a cross-linking ratio of body boxes in adjacent video frames. Those skilled in the art should understand that human body detection and tracking can be implemented using any one form in the related art, but the present invention is not limited thereto.

In a possible implementation form, it is possible to perform human body detection and tracking for each video frame of the video stream; may sample the video stream at regular time intervals to perform human body detection and tracking on the sampled video frames; It is also possible to obtain a key frame in a video stream, and perform human body detection and tracking on the key frame. The present invention is not limited thereto.

In a possible implementation form, the first region is the upper left (x1, y1) and lower right vertex coordinates (x2, y2) of the anatomical box (which may also be referred to as Region of Interest (ROI)) and the coordinates of the target object. It can be represented by a tracking identifier (Identify, ID), and the present invention is not limited to the display form of the first area.

In a possible implementation form, a first region image corresponding to the first region is cut out from each video frame, and in step S13, pose recognition is performed on the first region image through an already trained pose recognition network, so that the target object of pose recognition results. The pose recognition network may be, for example, a convolutional neural network, and the present invention does not limit the specific network structure and training form of the pose recognition network.

In a possible implementation form, in order to determine whether or not an abnormal event occurs in a subsequent processing, the pose recognition result includes the confidence that the pose of the target object is an abnormal pose. Here, the pose recognition result may include the reliability that the pose of the target object is a normal pose, the reliability that the pose is other than the normal pose and the abnormal pose, and the like, and the present invention does not limit the specific content of the pose recognition result.

In a possible implementation form, the abnormal pose may be, for example, a squat pose, a fall pose, and the like; The normal pose may be, for example, a standing pose; Other poses include a pose or a case in which a pose is not recognized, such as lying down or lying face down. The present invention is not limited to specific categories of various poses.

In a possible implementation form, in step S14, an event detection result of the target object may be determined according to the pose recognition result of the target object. The event detection result may include that an abnormal event has occurred or that an abnormal event has not occurred, for example, when the reliability of the abnormal pose exceeds a threshold value, or the reliability exceeds the threshold value and continues for a certain period of time , the event detection result is determined as the occurrence of an abnormal event. Here, the abnormal event may be determined according to the category of the abnormal pose, such as, for example, a squatting event and a falling event, and the present invention is not limited thereto.

In a possible implementation form, if the event detection result is an abnormal event occurrence, alert information may be generated and transmitted. The warning information may include a notification of an abnormal event in order for a related person to perform positioning, and may include a region in which a target object in which the abnormal event is generated is located. The present invention is not limited thereto.

In a possible implementation form, for example, a warning information can be sent to the control device of the escalator, so that the control device can stop the operation of the escalator, and the relevant personnel can stop the operation of the escalator and go to the escalator to save Thus, the warning information may be transmitted to a device (eg, a terminal device and/or a display device, etc.) that can be viewed by the relevant person responsible for operating the escalator. The present invention does not limit the content of the warning information.

According to an embodiment of the present invention, detecting an area in a video stream where an object is located; determine the pose of the subject by performing recognition on the image of the region; By determining the event detection result according to the pose of the target, and improving the accuracy of event detection, it is possible to lower the probability of other accidents and reduce the cost of manpower detection.

An event detection method according to an embodiment of the present invention will be described below.

As described above, a video stream of an area where the escalator is located may be collected through a camera, and the collected video stream may be transmitted to an electronic device such as a local front server or a cloud server. The electronic device may perform decoding on the received video stream to obtain a decoded video stream.

In step S12, through the detection network, performing human body detection and tracking on the decoded video stream to detect the human body box in each video frame of the video stream, and track the human body box of a pedestrian belonging to the same identity; A first region in which the target object is located in the video stream may be obtained. The detection network may be a convolutional neural network, and the present invention does not limit the network structure of the detection network.

After obtaining the first area, an image of the first area corresponding to the first area may be obtained. The image of the first region may be directly cut out from the video frame of the video stream, and extension processing may be performed on the first region.

In a possible implementation form, according to the event detection method of an embodiment of the present invention, the step of determining a first region image corresponding to the first region includes:

performing expansion on the first area according to a preset extension coefficient to obtain a second area, the second area being the extended first area; and

and cropping the image of the second region from the video frame of the video stream to obtain the image of the first region.

，

)(

，

)라고 설정하면, 공식 (1)은 하기와 같다.As described above, the first region may be represented by the upper left (x1, y1) and lower right vertex coordinates (x2, y2) of the human body box and the tracking ID of the target object. By presetting an extension coefficient, extension may be performed on the first region according to the extension coefficient. The coordinates of the upper-left and lower-right vertices of the extended second region are (

，

)(

，

), the formula (1) is as follows.

In formula (1), W and H represent the image width and height of the video frame, respectively, and e represents the expansion coefficient. The value of the extension coefficient e is, for example, 0.1, and the present invention does not limit the specific value of the extension coefficient e.

According to formula (1), it is possible to implement that the first region extends within the image range of the video frame to obtain the second region. Furthermore, the image of the second region may be cropped out of the corresponding video frame for subsequent processing, thereby obtaining the image of the first region.

Through this form, the classifier performance of the pose recognition network can be improved, and the accuracy of subsequent pose recognition can be improved.

In a possible implementation form, pose recognition may be performed on the first area image through the pose recognition network in step S13 to obtain a pose recognition result of the target object. The pose recognition network may include multiple classifiers to recognize that the pose of the target object is an abnormal pose, a normal pose or another pose. Before deploying the pose recognition network, training may be performed on the pose recognition network.

In a possible implementation form, the event detection method of an embodiment of the present invention comprises:

training according to a preset training set to obtain the pose recognition network, wherein the target poses tagged by the sample image in the training set include abnormal poses, normal poses and other poses, and the other poses include normal poses and abnormal poses. Pose other than − may be included.

For example, to improve the robustness of the classifier of a pose recognition network, a form of open set training can be used. That is, the target pose tagged by the sample image includes other poses in addition to the abnormal pose and the normal pose. For example, if the abnormal pose and the normal pose are a squatting pose and a standing pose, respectively, the other poses may include a lying down pose, a prone pose, etc., for example, a pose such as a half body completely covered, too blurry, etc. is judged It may include cases where it is not; It may also include situations where the detection box covers several people, the main person is not judged, etc. The present invention is not limited to both pose types included in other poses and specific situations in which there is no pose.

In the training process, sample images in the training set are respectively input and processed into a pose recognition network to be trained to obtain a sample recognition result; determine a network loss of the pose recognition network to be trained according to the difference between the sample recognition result and the sample image; reversely adjust the parameters of the pose recognition network to be trained according to the network loss; After several iterations, if the network converges, a trained pose recognition network can be obtained.

Through this form, it is possible to improve the robustness of the pose recognition and improve the accuracy of the pose recognition.

In a possible implementation form, the pose recognition result may include a confidence s that the pose of the target object is an abnormal pose. In step S14, event detection may be performed according to the result of the pose recognition of the target object to determine the event detection result of the target object.

In a possible implementation form, event detection can be implemented by maintaining a state machine at each tracking ID (ie target object), and updating the state of the state machine according to the result of pause recognition of each video frame.

2A is an exemplary diagram illustrating a change in a detection state of a target object according to an embodiment of the present invention. As shown in FIG. 2A , the detection state of the target object may include an idle state (IDLE) 21 , a state under detection (DETECTING) 22 , and a detected state (DETECTED) 23 . In the detection process, a detection state may be changed according to each condition in FIG. 2A , and when a predetermined condition is satisfied, a corresponding event detection result may be output.

In Figure 2a, activation time (Activate seconds, AC), cooling time (Cooldown seconds, CD), deactivate time (Deactivate seconds, DA) and the reliability threshold value T are all preset parameters; Detected Time (DT), Undetected Time (UT), and Trigger Time (TT) are time stamps, for determining whether activation time, deactivation time and cooling time have been reached; These timestamps are dynamically updated during processing; now indicates the timestamp of the current video frame (ie the current time); SET indicates the update of the timestamp, and OUT indicates the output result of the event.

Here, the values of the activation time (AC) and the inactivation time (DA) are, for example, 0 to 3 s, the value of the cooling time is, for example, 1 to 10 s, and the value of the reliability threshold T is, for example, 0.5, The present invention is not limited thereto.

In a possible implementation form, step S14 may include determining that the event detection result of the target object is an abnormal event when the reliability of the target object at the current time is greater than or equal to a reliability threshold value.

For example, if there is no ROI (i.e., the first region) in the video frame of the current time, or if the confidence s that the target object is an abnormal pose is less than the confidence threshold T, i.e., condition 211 (no ROI or s<T) is met. If satisfied, the detection state of the target object is in the idle state 21 .

In a possible implementation form, if the target object is in the idle state 21 and the activation time AC is not set or the activation time AC is set to zero, if the target object's confidence at the current time is the confidence threshold If it is greater than or equal to T, that is, if condition 212 (s>=T and AC=0) is satisfied, the output result is START, that is, the event detection result of the target object is determined as abnormal event occurrence.

in the above case, change the detected state of the target object to the detected state 23, to implement state update; By updating both the trigger time TT and the non-detection time UT to the current time now, it is possible to start calculating the trigger time TT and the non-detection time UT for subsequent determination.

Through this form, it is possible to directly determine that an abnormal event has occurred when the target object is in an abnormal pose, thereby improving the real-time of event detection, thereby providing timely warning and reducing the risk of safety accidents.

In a possible implementation form, step S14 comprises:

When the reliability of the target object at the current time is greater than or equal to the reliability threshold, and the first duration in which the reliability is greater than or equal to the reliability threshold is greater than or equal to the activation time, the event detection result of the target object is abnormal It may include determining that the event occurs. The activation time can be understood as the activation period.

As shown in FIG. 2A , when the target object is in the idle state 21 and the activation time AC is greater than 0, if the target object has a reliability at the current time greater than or equal to the reliability threshold T, that is, If condition 213 (s>=T and AC>0) is satisfied, no result is output, that is, the event detection result indicates that no abnormal event has occurred. In this case, the detection state of the target object may be changed to the detection state 22 , and the detection time DT may be updated to the current time now.

In a possible implementation form, if the target object is in the detecting state 22, if the target object has a confidence at the current time less than the confidence threshold T, or if there is no ROI in the video frame at the current time, i.e. condition 221 ( If there is no ROI or s<T) is satisfied, no result is output, that is, the event detection result is that no abnormal event has occurred. In this case, the detection state is changed to the idle state 21 .

In a possible implementation form, if the target object is in the detecting state 22 , if the target object has a confidence at the current time greater than or equal to a confidence threshold T, and a confidence level greater than or equal to a confidence threshold T, a first duration If the period (now-DT) is less than the activation time (AC), that is, if the condition 222 (s>=T and now-DT<AC) is satisfied, no result is output, that is, the event detection result is an abnormal event. it has not been In this case, the detection state is maintained in the detecting state 22 .

In a possible implementation form, if the detection state of the target object is the detecting state 22, if the target object has a reliability greater than or equal to the reliability threshold T at the current time, and the reliability is greater than or equal to the reliability threshold T 1 If the duration (now-DT) is greater than or equal to the activation time (AC), i.e., condition 223 (s>=T and now-DT>= AC), the output result is START, i.e. the event of the target The detection result is determined as the occurrence of an abnormal event.

in the above case, change the detected state of the target object to the detected state 23, to implement state update; For subsequent determination, both the trigger time TT and the non-detection time UT may be updated to the current time (now).

Through this form, when the target object is in an abnormal pose and persists for a certain period of time, it is determined that an abnormal event has occurred, thereby reducing false detections that may occur during single-frame detection, thereby improving the accuracy of event detection. .

In a possible implementation form, the event detection method according to an embodiment of the present invention further includes the step of transmitting first warning information when an abnormal event has occurred as a result of detecting the event of the target object.

In other words, if the output result is START, that is, if the event detection result of the target object is an abnormal event occurrence, corresponding warning information (referred to as first warning information) may be transmitted. The first warning information may include a notification of occurrence of an abnormal event in order for a related person to perform positioning, and may include a region in which a target object in which the abnormal event has occurred is located. The present invention is not limited thereto.

In a possible implementation form, the first warning information may be sent to the control device of the escalator, for example, so that the control device can stop the operation of the escalator, and cause the relevant personnel to stop the operation of the escalator, go to the escalator and rescue In order to do this, the first warning information may be transmitted to a device (eg, a terminal device and/or a display device, etc.) that can be viewed by the relevant person responsible for operating the escalator. The present invention does not limit the content of the first warning information.

Through this form, timely warning of abnormal events can be implemented, thereby reducing the risk of safety accidents.

In a possible implementation form, an event detection method according to an embodiment of the present invention comprises:

The method further includes changing the detection state of the target object to a detected state when the event detection result of the target object indicates that an abnormal event has occurred.

As described above, if condition 223 (s>=T and now-DT>= AC) is satisfied, the output result is START, that is, the event detection result of the target object is determined as abnormal event occurrence. In this case, the detection state of the target object is changed to the detected state 23 in order to continue the detection in the subsequent processing.

In a possible implementation form, the event detection method comprises:

The detection state of the target object is a detected state, the reliability of the target object at the current time is greater than or equal to a reliability threshold, and a second duration in which the reliability is greater than or equal to the reliability threshold is greater than the cooling time. in this case, determining that the event detection result of the target object is an abnormal event continuation; and

The method further includes transmitting second warning information when the event detection result of the target object is an abnormal event continuation. The cooling time can be understood as the cooling period.

For example, when the detection state of the target object is the detected state 23, if the target object has a reliability greater than or equal to the reliability threshold value T at the current time, and the reliability is greater than or equal to the reliability threshold value T, the first If the duration now-TT is less than or equal to the cooling time CD, it may be considered to be within the cooling time after the previous output, and no result is output.

In a possible implementation form, if the target object has a confidence at the current time greater than or equal to the confidence threshold T, and a second duration (now-TT) for which the confidence is greater than or equal to the confidence threshold T, is greater than the cooling time CD. If it is large, that is, if condition 231 (s>=T; if now-TT>Cd) is satisfied, the output result is ONGING, that is, the event detection result of the target object is determined as abnormal event continuation.

In a possible implementation form, if the target object's confidence at the current time is greater than or equal to the confidence threshold T to restart the dead time calculation, update the dead time (UT) to the current time (now), i.e. The time stamp of the undetected time (UT) can be reset. To restart the cooling time calculation, if the target object has a confidence at the current time greater than or equal to the confidence threshold T, and a second duration (now-TT) for which the confidence is greater than or equal to the confidence threshold T, the cooling time ( CD), the cooling time CD may be updated to the current time now, that is, the time stamp of the cooling time CD may be reset.

In a possible implementation form, when the event detection result of the target object is an abnormal event continuation, corresponding warning information (referred to as second warning information) may be generated and transmitted. The second warning information may include a notification of the continuation of the abnormal event in order for the relevant person to perform positioning, and may include a region in which the target object in which the abnormal event continues is located. The present invention is not limited thereto.

Through this form, if the target object is in an abnormal pose and the cooling time is reached, it is determined as an abnormal event continuation, repeatedly warning at the cooling time interval, repeatedly to the monitoring device operated by the escalator or the relevant personnel responsible for the operation of the escalator. By notifying this, it is possible to reduce the risk of safety accidents by reducing the occurrence of cases in which the relevant personnel fail to observe the warning information.

In a possible implementation form, the event detection method comprises:

When the detection state of the target object is the detected state, the reliability of the target object at the current time is less than a reliability threshold value, and a third duration period in which the reliability is less than the reliability threshold value is greater than the inactivation time, the The method further includes determining that the event detection result of the target object is the end of the abnormal event. The inactivation time may be understood as the inactivation period.

For example, if the detection state of the target object is the detected state 23, if the target object has a reliability at the current time less than the reliability threshold value T, or there is no ROI, and a third continuous If the period (now-UT) is less than or equal to the inactivation time (DA), i.e., if condition 232 ((no ROI or s<T) and now-UT<=DA) is satisfied, then within the inactivation time (DA) after the previous output It can be considered to exist, and no result is output.

In a possible implementation form, if the detection state of the target object is the detected state 23 , if the target object has a confidence at the current time less than the confidence threshold T or no ROI, and a third third whose confidence is less than the confidence threshold If the duration (now-UT) is greater than the inactivation time (DA), i.e., if condition 233 ((no ROI or s<T) and now-UT>DA) is satisfied, then the output result is END, i.e. the event of the target target The detection result is determined as the end of the abnormal event.

In a possible implementation form, when the event detection result is an abnormal event termination, event termination information may be generated and transmitted to notify a monitoring device operated by the escalator or a related person responsible for escalator operation; Event termination information may not be generated. The present invention is not limited thereto.

In a possible implementation form, when the event detection result of the target object is the end of the abnormal event, change the detection state of the target object to the idle state 21 to implement a status update; In order to subsequently determine again, the time stamp of the undetected time UT may be updated to nil.

In a possible implementation form, if the detection state of the target object is in the idle state 21 and there is no ROI of the target object for a period of time, the state machine of the target object is released, or the state machine is set to another target object. can be applied to The present invention is not limited thereto.

Through this form, when the target object is not in an abnormal pose and the inactivation time is reached, it is determined as an abnormal event end, thereby completing the entire processing flow of event detection.

Figure 2b is an exemplary flow frame diagram of the event detection method provided in an embodiment of the present invention, as shown in Figure 2b, the flow frame is a pedestrian detection and tracking module, walking and running retrograde recognition module, squatting recognition module includes

),(

), 및 보행자에 대응되는 추적 ID로 나타냄)을 얻고, 보행자 검출 박스는 직사각형 프레임일 수 있다. 여기서, 보행자 검출은 통상적으로 심층 신경 네트워크에 의해 구현되고, 보행자 추적 알고리즘은 딥러닝에 기반하는 알고리즘, IoU 매칭에 기반하는 알고리즘 등을 포함하며, 본 발명의 실시예는 어떤 검출 및 추적 알고리즘을 사용하는지 한정하지 않는다.The input video stream is decoded and input to the pedestrian detection and tracking module, where each pedestrian's first area (the upper left and lower right vertex coordinates of each pedestrian detection box)

),(

), and a tracking ID corresponding to the pedestrian) is obtained, and the pedestrian detection box may be a rectangular frame. Here, pedestrian detection is typically implemented by a deep neural network, and the pedestrian tracking algorithm includes an algorithm based on deep learning, an algorithm based on IoU matching, etc., and the embodiment of the present invention uses any detection and tracking algorithm do not limit what

By performing pose recognition on the image of the first area corresponding to the first area through the already trained pose recognition network, the pose recognition result of the target object can be obtained, that is, the pose recognition result can be output through the pose recognition network. have. For example, the pose recognition network is a squat recognition module, and may output a squat event through the squat recognition module.

The walking and running retrograde recognition module in the embodiment of the present invention analyzes the pedestrian movement speed using the video stream, and compares it with the escalator speed, for example, walking and running, or the pedestrian walking and running, such as a retrograde event output. It is possible to determine whether a retrograde action exists.

According to the event detection method of the embodiment of the present invention, detection and tracking are performed on the video stream of the area where the escalator is located to determine the human body area of the same object; performing pose recognition on the image of the region to determine a pose of the object; performing event detection according to the subject's pose to determine whether a squatting event has occurred; Finally, by returning an area determined as a squatting event and a corresponding human body box, a warning can be performed, thereby improving the accuracy of event detection and lowering the risk of a safety accident.

According to the event detection method of the embodiment of the present invention, event detection can be implemented by using only a monocular camera without other sensors (eg, an infrared sensor, etc.), thereby improving the efficiency of event detection.

According to the event detection method of the embodiment of the present invention, it can be applied to the security area and can be applied to products related to automatic escalator pedestrian dangerous behavior detection warning, for example, to the escalator self-service system of application scenarios such as large department stores, supermarkets, subway stations, and officetels. Deployed, it can take escalator deceleration or stop action to automatically detect and warn of hazardous behavior, such as event of squatting on escalator, to reduce manpower cost.

It can be understood that each of the method embodiments mentioned in the present invention can be mutually combined with each other to form a combined embodiment, as long as the principles and logic are not violated. No more repeating explanations. A person skilled in the art can understand that in the above method of a specific embodiment, a specific execution order of each step is determined by its function and possible internal logic.

In addition, the present invention further provides an event detection device, an electronic device, a computer-readable storage medium, and a program, wherein the image processing device, the electronic device, the computer-readable storage medium, and the program are all one event provided by the present invention It can be used to implement the detection method, and the corresponding description with reference to the corresponding technical solution and description and the method part will not be repeated any longer.

3 is a block diagram of an event detection device according to an embodiment of the present invention, and as shown in FIG. 3 , the event detection device includes:

a video stream acquiring portion 31, configured to acquire a video stream of the escalator;

a region determining portion (32), configured to perform human body detection on the video stream to determine a first region in the video stream in which a target object is located;

The pose recognition part 33 is configured to perform pose recognition on a first area image corresponding to the first area to obtain a pose recognition result of the target object. - The pose recognition result shows that the target pose is abnormal. Pose-in confidence included - ; and

and an event detection part (34), configured to determine, according to a result of the pose recognition of the target object, an event detection result of the target object.

In a possible implementation form, the event detection part is further configured to determine that the event detection result of the target object is an abnormal event occurrence, when the reliability of the target object at the current time is greater than or equal to a reliability threshold value.

In a possible implementation form, the event detection part is further configured to: when the reliability of the target object at the current time is greater than a confidence threshold, and a first duration in which the confidence is greater than or equal to the confidence threshold is greater than or equal to an activation time , a result of detecting an event of the target object is configured to determine that an abnormal event occurs.

In a possible implementation form, the event detection device further includes a first warning part configured to transmit first warning information when the event detection result of the target object is that an abnormal event has occurred.

In a possible implementation form, the event detection device further includes a state change part configured to change the detection state of the target object to the detected state when the event detection result of the target object is that an abnormal event has occurred, , wherein the event detection part further includes: the detection state of the target object is a detected state, the reliability of the target object at the current time is greater than or equal to the reliability threshold, the reliability is greater than the reliability threshold, or if the same second duration is greater than the cooling time, the event detection result of the target object is configured to be determined as an abnormal event duration; The event detection apparatus further includes a first warning part configured to transmit second warning information when the event detection result of the target object is an abnormal event continuation.

In a possible implementation form, the event detection part further comprises: the detection state of the target object is the detected state, the reliability of the target object at a current time is less than a reliability threshold, and the reliability is less than the reliability threshold. and when the third duration is greater than the inactivation time, the event detection result of the target object is configured to determine that the abnormal event ends.

In a possible implementation form, the event detection apparatus may include: an expansion part configured to perform expansion on the first area according to a preset expansion coefficient to obtain a second area; and a cropping portion, configured to crop the image of the second region from the video frame of the video stream to obtain the image of the first region.

In a possible implementation form, the pose recognition part is further configured to perform pose recognition on a first area image corresponding to the first area through a pose recognition network; wherein the event detection device includes a training part, configured to obtain the pose recognition network by training according to a preset training set, wherein the target pose tagged by the sample image in the training set includes an abnormal pose, a normal pose and another pose; , the other poses include poses other than normal poses and abnormal poses.

In a possible implementation form, the abnormal event comprises a squat event.

In some embodiments, a function possessed by an apparatus provided in an embodiment of the present invention or a module included in it may be used to perform the method described in the above-described method embodiment, and the specific implementation thereof follows the description of the above-described method embodiment. reference, and, for the sake of brevity, are not further repeated herein.

An embodiment of the present invention further provides a computer-readable storage medium storing computer program instructions, and implements the event detection method when the computer program instructions are executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

An embodiment of the present invention further provides an electronic device, the electronic device comprising: a processor; and a memory for storing instructions executable by the processor; Here, the processor is configured to execute the event detection method by calling the instruction stored in the memory.

An embodiment of the present invention further provides a computer program product comprising a computer readable code, wherein when the computer readable code is run in a device, a processor in the device implements the event detection method provided in any of the above embodiments Execute the command to

An embodiment of the present invention further provides another computer program product for storing computer-readable instructions, and when the instruction is executed, causes the computer to execute the operation of the event detection method provided in the one embodiment above.

The electronic device may be provided as a terminal, server, or other type of device.

4 is a block diagram of an electronic device according to an embodiment of the present invention. For example, the electronic device 800 may be a terminal such as a mobile phone, a computer, a digital broadcasting terminal, a message transceiving device, a game console, a tablet device, a medical device, a fitness device, or a personal portable terminal.

Referring to FIG. 4 , the electronic device 800 includes a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , and input/output (I/O) one or more of an interface 812 , a sensor component 814 , and a communication component 816 .

The processing component 802 controls the overall operation of the electronic device 800, such as operations usually related to displays, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 executing instructions to complete all or some steps of the event detection method. Further, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module for easy interaction between multimedia component 808 and processing component 802 .

The memory 804 is configured to store various types of data to support the operation of the electronic device 800 . Examples of such data include instructions, contact data, phone book data, messages, images, videos, and the like of any application program or method for operation on the electronic device 800 . Memory 804 includes static random access memory (SRAM), electrically erasable and programmable read-only memory (EEPROM), erasable and programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM) ), magnetic memory, flash memory, diskette or CD, or any type of volatile or non-volatile storage device, or a combination thereof.

Power component 806 provides power to various components of electronic device 800 . Power component 806 may include a power management system, one or more power sources, and other components related to generating, managing, and distributing power for electronic device 800 .

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive a signal input by a user. The touch panel includes one or a plurality of touch sensors to detect touch, slide, and gestures on the touch panel. The touch sensor not only senses the boundary of a touch or slide action, but also includes a duration associated with the touch or slide action and Pressure can also be detected. In some embodiments, multimedia component 808 includes at least one of one front camera and one rear camera. When the electronic device 800 is in an operation mode such as a photographing mode or a video mode, at least one of the front camera and the rear camera may receive external multimedia data. Each of the front and rear cameras may be a fixed optical lens system or may have focal length and optical zoom capabilities.

The audio component 810 is configured to perform at least one of an audio signal output and a signal input. For example, the audio component 810 includes one microphone (MIC), and when the electronic device 800 is in an operation mode such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. is composed The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, the audio component 810 further includes one speaker for outputting an audio signal.

An input/output (I/O) interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, click wheel, button, or the like. Such buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

The sensor component 814 includes one or a plurality of sensors for providing the electronic device 800 with status evaluation of various aspects. For example, the sensor component 814 may detect an on/off state of the electronic device 800 , a relative position of the component, for example, the component is a monitor and a keypad of the electronic device 800 , and a sensor The component 814 is the electronic device 800 or a change in the position of one component in the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the direction or acceleration/deceleration of the electronic device 800, and the electronic device ( 800) can also be detected. The sensor component 814 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor component 814 may further include an optical sensor for use in imaging applications, such as complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensors. In some embodiments, the sensor component 814 may further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communication in a wired or wireless manner between the electronic device 800 and another device. The electronic device 800 may access a wireless network based on a communication standard such as, for example, Wi-Fi, second-generation mobile communication technology (2G), or third-generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system through a broadcast channel. In one demonstrative embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate near field communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared communication standard (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 includes one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), and programs to perform the event detection method. It may be implemented by a capable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic device.

In an exemplary embodiment, there is further provided a non-volatile computer readable storage medium, such as, for example, a memory 804 comprising computer program instructions, the computer program instructions being provided to the processor 820 of the electronic device 800 . to complete the event detection method by being performed by

5 is a block diagram of another electronic device shown according to an embodiment of the present invention. For example, the electronic device 1900 may be provided as a server. Referring to FIG. 5 , an electronic device 1900 includes a processing component 1922 and further includes one or more processors and memory resources representative of a memory 1932 , such as an application program, for example. to store instructions that may be executed by processing component 1922 . The application program stored in the memory 1932 may include one or more modules corresponding to each instruction set. Further, processing component 1922 is configured to execute instructions to perform the event detection method.

The electronic device 1900 includes a power component 1926 configured to perform power management of the electronic device 1900 , a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and input/output (I/O) ) may further include an interface 1958 . The electronic device 1900 is, for example, a Microsoft server operating system (Windows Server™), a graphical user interface based operating system (Mac OS X™) released by Apple Corporation, a multi-user multi-process computer operating system (Unix™), a free and an operating system stored in the memory 1932 such as an open source code Unix-like operating system (Linux™), an open source code Unix-like operating system (FreeBSD™), or a similar operating system.

In an exemplary embodiment, there is further provided a non-volatile computer readable storage medium, such as, for example, a memory 1932 comprising computer program instructions, the computer program instructions comprising: a processing component 1922 of an electronic device 1900 to complete the event detection method by being performed by

The present invention may be at least one of a system, a method and a computer program product. The computer program product may include a computer-readable storage medium, in which the computer-readable program instructions for causing a processor to implement each aspect of the present invention are loaded.

A computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory). ), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital video disk (DVD), memory stick, floppy disk, encoded devices and any suitable combination thereof. As used herein, a computer-readable storage medium includes radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, pulses passing through fiber optic cables), or electrical signals transmitted over wires. It is not interpreted as the same instantaneous signal itself.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing/processing device, or stored on an external computer or external storage via a network such as at least one of the Internet, a local area network, a wide area network, and a wireless network. It can be downloaded to your device. The network may include at least one of a copper transmission cable, a fiber optic transmission, a wireless transmission, a router, a firewall, a switchboard, a gateway computer, and an edge server. A network adapter card or network interface of each computing/processing device receives computer readable program instructions from a network and transmits the computer readable program instructions for storage in a computer readable storage medium in each computing/processing device .

The computer program instructions for carrying out the operations of the present invention may include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or any of one or more programming languages. It may be source code or object code written in combination, and the programming languages include object-oriented programming languages such as Smalltalk, C++, and the like, and general programming languages such as "C" languages or similar programming languages. The computer readable program instructions may be performed entirely on the user's computer, partially on the user's computer, as a standalone software package, in part on the user's computer and in part on a remote computer, or completely on the remote computer or server. can be performed. In situations involving remote computers, the remote computer is connected to your computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g. using an Internet service provider). can be connected via the Internet). In some embodiments, an electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) can be personalized using state information in computer readable program instructions, the electronic circuit may execute computer readable program instructions, thereby implementing various aspects of the present invention.

Here, various aspects of the present invention have been described with reference to at least one of a flowchart and a block diagram of a method, an apparatus (system), and a computer program product according to an embodiment of the present invention. It should be understood that each block of at least one of the flowcharts and block diagrams and combinations of blocks of at least one of the flowcharts and block diagrams may all be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device, thereby causing the machine to cause such instructions to be executed by the processor of the computer or other programmable data processing device. and an apparatus implementing the functions/operations specified in one or a plurality of blocks in at least one of a flowchart and a block diagram is created. These computer readable program instructions may be stored in a computer readable storage medium, which instructions may cause at least one of a computer, a programmable data processing apparatus, and other devices to be operated in a specific way, thereby causing the computer readable instructions stored thereon. The capable medium includes an article of manufacture comprising instructions for implementing the functions/acts specified in one or a plurality of blocks in at least one of a flowchart and a block diagram.

The computer readable program instructions may be loaded into a computer, other programmable data processing device, or other device such that a series of operational steps is performed on the computer, other programmable data processing device, or other device to create a computer-implemented process. By causing it to be executed, the instructions executed on the computer, other programmable data processing device, or other device implement the functions/actions specified in one or a plurality of blocks in at least one of the flowcharts and block diagrams.

The flow diagrams and block diagrams in the drawings illustrate implementable system architectures, functions, and operations of systems, methods, and computer program products in accordance with a plurality of embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or portion of an instruction, wherein the module, program segment or portion of the instruction is an executable instruction for implementing one or a plurality of specified logical functions. includes In some alternative implementations, the functions indicated in the blocks may occur in a different order than indicated in the figures. For example, two blocks shown in succession may actually be performed simultaneously, and sometimes may be performed in the reverse order according to the related function, which is determined by the related function. It should also be noted that each block in at least one of the block diagrams and flowcharts, and combinations of blocks in at least one of the block diagrams and flowcharts, may be implemented by a dedicated hardware-based system for executing specified functions or operations, or , can be implemented by a combination of dedicated hardware and computer instructions.

The computer program product may be specifically implemented in the form of hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium, and in another optional embodiment, the computer program product is a software product, for example a Software Development Kit (SDK) or the like. is specifically embodied as

Each embodiment of the present invention has been described above, and the description is illustrative, not exhaustive, and is not limited to each disclosed embodiment. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of each described embodiment. The choice of terminology used herein best interprets the principle of each embodiment, practical application, or improvement of technology in the market, or that others skilled in the art can understand each embodiment disclosed herein. The purpose.

An embodiment of the present invention discloses an event detection method, apparatus, electronic device, storage medium and program product, wherein the event detection method includes: acquiring a video stream of an escalator; determining a first region in the video stream where a target object is located by performing human body detection on the video stream; performing pose recognition on a first area image corresponding to the first area to obtain a pose recognition result of the target object, wherein the pose recognition result includes reliability that the target pose is an abnormal pose; and determining an event detection result of the target target according to the pose recognition result of the target target. Through the event detection method, by improving the accuracy of event detection, it is possible to lower the probability of occurrence of other accidents and reduce the cost of manpower detection.

Claims (21)

An event detection method comprising:
acquiring a video stream of the escalator;
determining a first region in the video stream where a target object is located by performing human body detection on the video stream;
performing pose recognition on a first area image corresponding to the first area to obtain a pose recognition result of the target object, wherein the pose recognition result includes reliability that the target pose is an abnormal pose; and
and determining an event detection result of the target object according to the pose recognition result of the target object. According to claim 1,
The step of determining the event detection result of the target object according to the pose recognition result of the target object,
and determining that the event detection result of the target object is an abnormal event when the reliability of the target object at the current time is greater than or equal to a reliability threshold. According to claim 1,
The step of determining the event detection result of the target object according to the pose recognition result of the target object,
When the reliability of the target object at the current time is greater than or equal to the reliability threshold, and the first duration in which the reliability is greater than or equal to the reliability threshold is greater than or equal to the activation time, the event detection result of the target object is abnormal An event detection method comprising determining that an event has occurred. 4. The method according to any one of claims 1 to 3,
The event detection method is
The method of claim 1, further comprising: transmitting first warning information when an abnormal event has occurred as a result of detecting the event of the target target. 5. The method according to any one of claims 1 to 4,
The event detection method is
changing the detection state of the target object to a detected state when the event detection result of the target object is that an abnormal event has occurred;
The detection state of the target object is a detected state, the reliability of the target object at the current time is greater than or equal to a reliability threshold, and a second duration in which the reliability is greater than or equal to the reliability threshold is greater than the cooling time. in this case, determining that the event detection result of the target object is an abnormal event continuation; and
The method of claim 1, further comprising: transmitting second warning information when the event detection result of the target object is an abnormal event continuation. 6. The method of claim 5,
The event detection method is
When the detection state of the target object is the detected state, the reliability of the target object at the current time is less than a reliability threshold value, and a third duration period in which the reliability is less than the reliability threshold value is greater than the inactivation time, the The event detection method according to claim 1, further comprising determining that the event detection result of the target object is the end of the abnormal event. 7. The method according to any one of claims 1 to 6,
The step of determining a first area image corresponding to the first area includes:
performing expansion on the first area according to a preset extension coefficient to obtain a second area;
and cropping the image of the second region from the video frame of the video stream to obtain the image of the first region. 8. The method according to any one of claims 1 to 7,
The performing pose recognition on the first area image corresponding to the first area includes performing pose recognition on the first area image corresponding to the first area through a pose recognition network;
The event detection method includes: training according to a preset training set to obtain the pose recognition network; in the training set, target poses tagged by sample images include abnormal poses, normal poses, and other poses, and the other poses include: includes poses other than normal poses and abnormal poses. 7. The method according to any one of claims 2 to 6,
The abnormal event is an event detection method, characterized in that it comprises a squatting event. An event detection device comprising:
a video stream acquiring portion, configured to acquire a video stream of the escalator;
a region determining portion configured to perform human body detection on the video stream to determine a first region in the video stream where a target object is located;
A pose recognition part configured to obtain a pose recognition result of the target object by performing pose recognition on a first area image corresponding to the first area - Reliability that the pose of the target object is an abnormal pose in the pose recognition result contains - ; and
and an event detection part configured to determine an event detection result of the target object according to a pose recognition result of the target object. 11. The method of claim 10,
The event detection unit is further configured to determine that the event detection result of the target object is an abnormal event occurrence when the reliability of the target object at the current time is greater than or equal to a reliability threshold value. 11. The method of claim 10,
The event detection part is further configured to: when the target object has a reliability at the current time greater than or equal to a reliability threshold, and a first duration in which the reliability is greater than or equal to the reliability threshold is greater than or equal to an activation time, the target object The event detection device, characterized in that the event detection result of the target is configured to determine that an abnormal event occurs. 13. The method according to any one of claims 10 to 12,
The event detecting device further comprises a first warning part configured to transmit first warning information when the event detection result of the target is that an abnormal event has occurred. 14. The method according to any one of claims 10 to 13,
the event detection apparatus further includes a state change part configured to change the detection state of the target object to the detected state when the event detection result of the target object is that an abnormal event has occurred;
The event detection part may further include: a second detection state of the target object is a detected state, the reliability of the target object at the current time is greater than or equal to a reliability threshold, and the reliability is greater than or equal to the reliability threshold. if the duration is greater than the cooling time, the event detection result of the target object is configured to determine an abnormal event duration;
The event detection device further comprises a first warning part configured to transmit second warning information when the event detection result of the target object is an abnormal event continuation. 15. The method of claim 14,
The event detection part may further include a third duration in which the detection state of the target object is the detected state, the target object has a reliability at a current time less than a reliability threshold value, and the reliability is less than the reliability threshold value. If the inactivation time is greater than the inactivation time, the event detection result of the target object is an event detection device, characterized in that configured to determine the end of the abnormal event. 16. The method according to any one of claims 10 to 15,
The event detection device,
an expansion part configured to perform expansion on the first area according to a preset expansion coefficient to obtain a second area; and
and a cropping portion configured to crop the image of the second region from the video frame of the video stream to obtain the image of the first region. 17. The method according to any one of claims 10 to 16,
the pose recognition part is further configured to perform pose recognition on a first area image corresponding to the first area through a pose recognition network;
wherein the event detection device includes a training part, configured to obtain the pose recognition network by training according to a preset training set, wherein the target pose tagged by the sample image in the training set includes an abnormal pose, a normal pose and another pose; , The other poses include poses other than normal poses and abnormal poses. 16. The method according to any one of claims 11 to 15,
The abnormal event is an event detection device, characterized in that it includes a squatting event. As an electronic device,
processor; and
the processor includes a memory configured to store executable instructions;
The electronic device, characterized in that the processor is configured to execute the event detection method according to any one of claims 1 to 9 by calling an instruction stored in the memory. A computer readable storage medium comprising:
A computer readable storage medium having stored thereon computer program instructions, and when the computer program instructions are executed by a processor, the event detection method according to any one of claims 1 to 9 is implemented. A computer program product comprising:
Computer readable code is included, characterized in that when the computer readable code is operated in a device, a processor in the device executes instructions for implementing the method of detecting an event according to any one of claims 1 to 9. A computer program product made with

Images