CN114283362A - Elevator passenger abnormal behavior detection method, system, terminal device and storage medium - Google Patents
Elevator passenger abnormal behavior detection method, system, terminal device and storage medium Download PDFInfo
- Publication number
- CN114283362A CN114283362A CN202111575522.0A CN202111575522A CN114283362A CN 114283362 A CN114283362 A CN 114283362A CN 202111575522 A CN202111575522 A CN 202111575522A CN 114283362 A CN114283362 A CN 114283362A
- Authority
- CN
- China
- Prior art keywords
- key point
- passenger
- behavior
- elevator
- human body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Indicating And Signalling Devices For Elevators (AREA)
- Maintenance And Inspection Apparatuses For Elevators (AREA)
Abstract
The invention discloses a method, a system, terminal equipment and a storage medium for detecting abnormal behaviors of elevator passengers, wherein the method comprises the following steps: acquiring a video frame image in an elevator; converting the video frame image into an HSV color space image; carrying out ROI (region of interest) region segmentation on a human body in the HSV color space image to obtain human body key points; identifying key points of a human body to obtain key point connection data and key point coordinate data of corresponding passengers; and classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal behavior based on a classification and identification result. The invention improves the calculation efficiency, simultaneously eliminates the identification error caused by environmental factors, improves the identification accuracy of abnormal behaviors in the elevator and meets the detection requirement.
Description
Technical Field
The invention relates to the technical field of elevator safety, in particular to an elevator passenger abnormal behavior detection method, system, terminal equipment and storage medium based on AI and image processing.
Background
In recent years, with the increase of high-rise buildings, people rely on elevators more and more strongly. However, the elevator car is narrow in space, closed and opaque, and it is often difficult for the outside to find man-made abnormal behaviors such as robbery, fighting, invasion, door opening and the like in the elevator car in time. Therefore, the elevator passenger behavior monitoring device is very important, accidents such as elevator falling and the like are possibly caused by passenger dangerous behaviors such as jumping and door opening, the life and property safety of passengers is threatened, timely and effective measures are taken, and the loss of life and property can be greatly avoided or saved.
Along with the development of artificial intelligence technology and computer vision field technology, replace the manpower with the computer and detect the abnormal situation and popularize gradually, consequently, through the unusual action of action detection in time discernment passenger and send corresponding alarm, can effectual reduction elevator accident, the safety of better guarantee passenger trip to have important research and commercial value.
At present, the most direct method for detecting elevator passenger dangerous behaviors is to perform image processing on elevator monitoring videos, but most of the prior art only directly utilizes the deep learning technology to perform passenger behavior identification, and images are not preprocessed, so that the influence of environmental factors on identification precision is very large, the accuracy of abnormal behavior identification is reduced, meanwhile, the prior art cannot consider the calculation efficiency, and the detection requirement cannot be met.
Disclosure of Invention
The invention mainly aims to provide a method, a system, a terminal device and a storage medium for detecting abnormal behaviors of passengers in an elevator, aiming at improving the identification accuracy and the calculation efficiency of the abnormal behaviors in the elevator and meeting the detection requirement.
In order to achieve the above object, an embodiment of the present invention provides an elevator passenger abnormal behavior detection method, including the steps of:
acquiring a video frame image in an elevator;
converting the video frame image into an HSV color space image;
carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image to obtain human body key points;
identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers;
and classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal behavior based on a classification and identification result.
Optionally, the classifying and identifying the passenger behavior based on the key point connection data and the key point coordinate data, and the determining whether the passenger behavior is an abnormal behavior based on the classification and identification result includes:
selecting key point connection data and key point coordinate data of continuous N frames of images;
inputting the selected key point connection data and key point coordinate data of the N frames of images as a group of data into a passenger behavior recognition model established in advance, and carrying out classification recognition on the behaviors of passengers;
sliding M frames backwards according to the time sequence of the video frame images, reselecting key point connection data and key point coordinate data of the N frames of images, inputting the data into the passenger behavior recognition model, carrying out classification recognition on the passenger behaviors, and so on, and when the passenger behaviors are classified into the set abnormal behaviors for T times, judging the passenger behaviors as the abnormal behaviors; wherein N, M, T is a positive integer.
Optionally, N is a positive integer in the interval [30, 60], M is a positive integer in the interval [1, 5], and T is greater than or equal to 3.
Optionally, the classifying and identifying the passenger behavior based on the key point connection data and the key point coordinate data, and the determining whether the passenger behavior is an abnormal behavior based on the classification and identification result includes:
classifying and identifying the behavior of the passenger based on the key point connection data, the key point coordinate data and a passenger behavior identification model established in advance through a cloud server, and judging whether the behavior of the passenger is abnormal based on a classification identification result.
Optionally, the method further comprises:
and when the passenger behavior is detected to be abnormal behavior, the cloud server informs the property management terminal to take countermeasures.
Optionally, the step of converting the video frame image into an HSV color space image includes:
converting, by an edge computing device, the video frame image into an HSV color space image;
the step of carrying out ROI region segmentation on the human body in the HSV color space image to obtain the key points of the human body comprises the following steps:
carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image through the edge computing device to obtain human body key points;
the step of identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers comprises the following steps:
and identifying each segmented human key point by the edge computing device by using a convolutional neural network to obtain key point connection data and key point coordinate data of the corresponding passenger, and transmitting the key point connection data and the key point coordinate data to the cloud server.
Optionally, the step of acquiring video frame images in the elevator comprises:
receiving an access instruction which is triggered by a user through a property management terminal and used for accessing the state of a camera in an elevator through the cloud server;
sending the access instruction to the edge computing device through the cloud server;
and acquiring data in the elevator through a camera corresponding to the edge computing device to obtain a video frame image in the elevator.
Optionally, the key point is a joint point of the passenger or a geometric center point of the trunk of the body, and the key point connection data is a set of corresponding joint connection relationships or trunk connection relationships, and is stored in a topological structure.
Optionally, the passenger behavior recognition model is formed by connecting an LSTM network in series with a plurality of SVM classifiers connected in parallel, and the number of the SVM classifiers is the same as the number of preset types of passenger abnormal behaviors.
Optionally, the step of classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and determining whether the behavior of the passenger is an abnormal behavior based on a classification identification result further includes:
and establishing the passenger behavior recognition model based on a plurality of frames of images collected in advance.
In addition, the embodiment of the present invention further provides an elevator passenger abnormal behavior detection system, where the system includes:
the image acquisition module is used for acquiring a video frame image in the elevator;
the image conversion module is used for converting the video frame image into an HSV color space image;
the image segmentation module is used for carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image to obtain human body key points;
the identification module is used for identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers;
and the behavior judgment module is used for classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data and judging whether the behavior of the passenger is abnormal based on a classification and identification result.
In addition, the embodiment of the present invention further provides an elevator passenger abnormal behavior detection system, where the system includes:
the edge computing device is used for acquiring a video frame image in the elevator, converting the video frame image into an HSV color space image, performing ROI (region of interest) region segmentation on a human body in the HSV color space image to obtain a human body key point, and identifying the human body key point to obtain key point connection data and key point coordinate data of a corresponding passenger;
and the cloud server is used for classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal based on a classification and identification result.
In addition, the embodiment of the present invention further provides a terminal device, where the terminal device includes a memory, a processor, and a computer program stored on the memory and operable on the processor, and when the computer program is executed by the processor, the method for detecting abnormal behavior of elevator passengers as described above is implemented.
Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the elevator passenger abnormal behavior detection method as described above.
Furthermore, an embodiment of the present invention further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for detecting abnormal behavior of elevator passengers as described above is implemented.
According to the elevator passenger abnormal behavior detection method, the elevator passenger abnormal behavior detection system, the terminal device and the storage medium, the video frame image in the elevator is obtained; converting the video frame image into an HSV color space image; carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image to obtain human body key points; identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers; and classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal behavior based on a classification and identification result. Therefore, the monitoring video is preprocessed through image color space conversion and human joint key point identification, the calculated amount of a cloud server is reduced, the calculation efficiency is improved, meanwhile, the identification error caused by environmental factors is eliminated, and the identification accuracy of abnormal behaviors in the elevator is improved; in addition, passenger behaviors can be accurately classified in a time period, misjudgment caused by single-frame images or recognition errors can be effectively avoided, the recognition accuracy of abnormal behaviors in the elevator is further improved, and the detection requirement is met.
Drawings
FIG. 1 is a schematic diagram of a hardware environment of an elevator passenger abnormal behavior detection system according to the present invention;
fig. 2 is a schematic diagram of the architecture of an elevator passenger abnormal behavior detection system according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of an exemplary embodiment of an elevator passenger abnormal behavior detection method of the present invention;
fig. 4 is a schematic flow chart of another exemplary embodiment of the elevator passenger abnormal behavior detection method of the present invention;
fig. 5 is a functional block diagram of an elevator passenger abnormal behavior detection system according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The main solution of the embodiment of the invention is as follows: obtaining a video frame image in an elevator; converting the video frame image into an HSV color space image; carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image to obtain human body key points; identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers; and classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal behavior based on a classification and identification result. Therefore, the monitoring video is preprocessed through image color space conversion and human joint key point identification, the calculated amount of a cloud server is reduced, the calculation efficiency is improved, meanwhile, the identification error caused by environmental factors is eliminated, and the identification accuracy of abnormal behaviors in the elevator is improved; in addition, passenger behaviors can be accurately classified in a time period, misjudgment caused by single-frame images or recognition errors can be effectively avoided, the recognition accuracy of abnormal behaviors in the elevator is further improved, and the detection requirement is met.
The technical terms related to the embodiment of the invention are as follows:
HSV (Hue, Saturation), a color space created by a.r. smith in 1978 based on the intuitive nature of color, also known as the hexagonal cone Model (Hexcone Model). The parameters of the colors in this model are: hue (H), saturation (S), lightness (V).
The embodiment of the invention considers that in the existing related scheme, the identification and analysis of the abnormal behaviors of passengers are mostly aimed at, but the pertinence is too wide, the adopted image identification technology is only a simple image identification technology, the requirement can not be met in the aspect of accuracy, and the calculation efficiency is not considered; although various network models can be adopted to learn the camera video, the change of scenes and the lack of data sets inevitably cause the problem of identification accuracy. Therefore, the existing related scheme is only to simply perform image processing on the monitoring image, but lacks of preprocessing on the motion of the person, so that the identification accuracy is low, and the calculation efficiency cannot be considered.
Therefore, the embodiment of the invention provides a solution, which can improve the identification accuracy and the calculation efficiency of abnormal behaviors in the elevator and meet the detection requirement.
Specifically, referring to fig. 1, fig. 1 is a schematic diagram of a hardware environment of an elevator passenger abnormal behavior detection system according to the present invention. The elevator passenger abnormal behavior detection system can be borne on the terminal equipment in a hardware or software mode. The terminal device can be an intelligent mobile terminal such as a mobile phone and a tablet personal computer, and can also be a network device such as a server.
In this embodiment, the elevator passenger abnormal behavior detection system at least includes an output module 110, a processor 120, a memory 130, and a communication module 140.
The memory 130 stores therein an operating system and an elevator passenger abnormal behavior detection program; the output module 110 may be a display screen, a speaker, etc. The communication module 140 may include a WIFI module, a mobile communication module, a bluetooth module, and the like, and communicates with an external device or a server through the communication module 140.
As an embodiment, the elevator passenger abnormal behavior detection program in the memory 130 implements the following steps when executed by the processor:
acquiring a video frame image in an elevator;
converting the video frame image into an HSV color space image;
carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image to obtain human body key points;
identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers;
and classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal behavior based on a classification and identification result.
Further, the elevator passenger abnormal behavior detection program in the memory 130 when executed by the processor further implements the steps of:
selecting key point connection data and key point coordinate data of continuous N frames of images;
inputting the selected key point connection data and key point coordinate data of the N frames of images as a group of data into a passenger behavior recognition model established in advance, and carrying out classification recognition on the behaviors of passengers;
sliding M frames backwards according to the time sequence of the video frame images, reselecting key point connection data and key point coordinate data of the N frames of images, inputting the data into the passenger behavior recognition model, carrying out classification recognition on the passenger behaviors, and so on, and when the passenger behaviors are classified into the set abnormal behaviors for T times, judging the passenger behaviors as the abnormal behaviors; wherein N, M, T is a positive integer.
Further, the elevator passenger abnormal behavior detection program in the memory 130 when executed by the processor further implements the steps of:
classifying and identifying the behavior of the passenger based on the key point connection data, the key point coordinate data and a passenger behavior identification model established in advance through a cloud server, and judging whether the behavior of the passenger is abnormal based on a classification identification result.
Further, the elevator passenger abnormal behavior detection program in the memory 130 when executed by the processor further implements the steps of:
and when the passenger behavior is detected to be abnormal behavior, the cloud server informs the property management terminal to take countermeasures.
Further, the elevator passenger abnormal behavior detection program in the memory 130 when executed by the processor further implements the steps of:
converting, by an edge computing device, the video frame image into an HSV color space image;
carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image through the edge computing device to obtain human body key points;
and identifying each segmented human key point by the edge computing device by using a convolutional neural network to obtain key point connection data and key point coordinate data of the corresponding passenger, and transmitting the key point connection data and the key point coordinate data to the cloud server.
Further, the elevator passenger abnormal behavior detection program in the memory 130 when executed by the processor further implements the steps of:
receiving an access instruction which is triggered by a user through a property management terminal and used for accessing the state of a camera in an elevator through the cloud server;
sending the access instruction to the edge computing device through the cloud server;
and acquiring data in the elevator through a camera corresponding to the edge computing device to obtain a video frame image in the elevator.
Further, the elevator passenger abnormal behavior detection program in the memory 130 when executed by the processor further implements the steps of:
and establishing the passenger behavior recognition model based on a plurality of frames of images collected in advance.
According to the scheme, the video frame image in the elevator is obtained; converting the video frame image into an HSV color space image; carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image to obtain human body key points; identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers; and classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal behavior based on a classification and identification result. Therefore, the monitoring video is preprocessed through image color space conversion and human joint key point identification, the calculated amount of a cloud server is reduced, the calculation efficiency is improved, and meanwhile, the identification error caused by environmental factors is eliminated; a passenger behavior recognition model is established for multi-frame images, passenger behaviors are accurately classified in a time period, and misjudgment caused by single-frame images or recognition errors can be effectively avoided. Meanwhile, the computing efficiency is improved through a mode of combining the edge computing nodes and the cloud server.
Referring to fig. 2, fig. 2 is a schematic diagram of an architecture of an elevator passenger abnormal behavior detection system according to an embodiment of the present invention.
As shown in fig. 2, the elevator passenger abnormal behavior detection system mainly includes: edge computing device 3 and cloud server 1, wherein:
the edge computing device 3 is used for acquiring a video frame image in the elevator through a camera, converting the video frame image into an HSV color space image, performing ROI (region of interest) region segmentation on a human body in the HSV color space image to obtain human body key points, and identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers;
and the cloud server 1 is used for classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal based on a classification and identification result.
Specifically, as shown in fig. 2, as one embodiment, the elevator passenger abnormal behavior detection system may include: the system comprises a cloud server 1, a cloud server database 2, universal edge computing devices 3 and 7, local databases 4 and 8, elevator cameras 5 and 6 and a property management terminal 9. Wherein:
the cloud server 1 is respectively connected with a cloud database 2, an edge computing device 3 and an edge computing device 7; the edge computing device 3 is respectively connected with a local database 4, an elevator camera 5 and an elevator camera 6; the edge computing device 7 is respectively connected with a local database 8 and a property management terminal 9.
When a user accesses the states of the elevator cameras 5 and 6 through the network remote access cloud server of the property management terminal 9, the cloud server 1 broadcasts an access instruction to the edge computing device 3 and the edge computing device 7, and the edge computing device 3 and the edge computing device 7 match the access instruction, wherein the purpose of matching is to determine that the property management terminal 9 connected with the edge computing device 7 needs to access the states of the elevator cameras 5 and 6.
After receiving the access instruction, the peripheral interface or the network module of the edge computing device 3 performs data acquisition through the elevator camera 5 and the elevator camera 6 corresponding to the edge computing device 3. The elevator camera 5 and the elevator camera 6 transmit data acquired in real time to an external interface or a network module of the edge computing device 3 connected with the elevator camera, and the external interface or the network module uploads the received data to a memory of the corresponding edge computing device 3; then, a computing unit of the edge computing device 3 calls an image color space conversion and human body joint key point identification algorithm to preprocess data, a computing result is uploaded to a cloud server 1 through a network module of the edge computing device 3, the cloud server 1 carries out classification identification on passenger behaviors through a passenger behavior identification model, the results are stored in a cloud database 2, and meanwhile, the computing result is stored in a local database 4 through an external interface of the edge computing device 3.
Therefore, the monitoring video is preprocessed through image color space conversion and human joint key point identification, the calculated amount of a cloud server is reduced, the calculation efficiency is improved, meanwhile, the identification error caused by environmental factors is eliminated, and the identification accuracy of abnormal behaviors in the elevator is improved; in addition, passenger behaviors can be accurately classified in a time period, misjudgment caused by single-frame images or recognition errors can be effectively avoided, the recognition accuracy of abnormal behaviors in the elevator is further improved, and the detection requirement is met.
It should be noted that the number of the elevator cameras, the edge calculating device 7, the corresponding database, and the like may be set according to actual situations, and this embodiment is not particularly limited thereto.
Based on the above system architecture but not limited to the above architecture, embodiments of the method of the present invention are presented.
Referring to fig. 3, fig. 3 is a schematic flow chart of an exemplary embodiment of the elevator passenger abnormal behavior detection method of the present invention. The elevator passenger abnormal behavior detection method comprises the following steps:
step S101, acquiring a video frame image in an elevator;
the execution main body of the method of the embodiment can be an elevator passenger abnormal behavior detection device, or can be a terminal device, a cloud server or an elevator passenger abnormal behavior detection system, the embodiment is exemplified by the elevator passenger abnormal behavior detection system, and the elevator passenger abnormal behavior detection system can comprise an edge computing device and a cloud server.
Specifically, to monitor passenger behavior within an elevator, first, video frame images within the elevator are acquired.
As an implementation mode, a video frame image in the elevator can be acquired through a camera arranged in the elevator, so that the behavior of passengers in the elevator can be acquired based on the video frame image, and whether the behavior of the passengers is abnormal behavior such as dangerous behavior or not can be judged.
As another embodiment, the user may actively initiate the acquisition of image data of passengers in the elevator through the property management terminal.
Specifically, a user triggers an access instruction for accessing the state of the camera in the elevator through the property management terminal, and receives the access instruction for accessing the state of the camera in the elevator, which is triggered by the user through the property management terminal, through the cloud server; sending the access instruction to the edge computing device through the cloud server; and acquiring data in the elevator through a camera corresponding to the edge computing device to obtain a video frame image in the elevator.
More specifically, when a user accesses the state of the elevator camera through the network remote access cloud server of the property management terminal, the cloud server broadcasts an access instruction to the edge computing device, and after a peripheral interface or a network module of the edge computing device receives the access instruction, data acquisition is performed through the elevator camera corresponding to the edge computing device. The elevator camera transmits the data collected in real time to a peripheral interface or a network module of the edge computing device connected with the elevator camera, and the peripheral interface or the network module uploads the received data to a local database of the corresponding edge computing device.
The video frame image in the elevator collected by the camera is an RGB image.
Step S102, converting the video frame image into an HSV color space image;
wherein, as an embodiment, the video frame image is converted into an HSV color space image by an edge computing device;
as described above, the video frame image in the elevator collected by the camera of the edge computing device is an RGB image. In this embodiment, the edge computing device converts the single frame RGB image in the local database into the HSV color space image by using the image color space conversion method, so that the monitoring video is preprocessed by using the image color space conversion at the edge, the calculation amount of the cloud server is reduced, the calculation efficiency is improved, and meanwhile, the recognition error caused by environmental factors is eliminated, for example, the influence of shadows on the model detection effect due to insufficient illumination is eliminated, which is helpful for improving the accuracy of abnormal behavior recognition.
Step S103, carrying out ROI region segmentation on the human body in the HSV color space image to obtain human body key points;
step S104, identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers;
specifically, as an embodiment, the edge computing device may perform ROI region segmentation on the human body in the HSV color space image to obtain human body key points;
and then, identifying each segmented human key point by using the convolutional neural network through the edge computing device to obtain key point connection data and key point coordinate data of the corresponding passenger, and transmitting the key point connection data and the key point coordinate data to the cloud server.
The key points are the joint points of the passengers or the geometric central points of the body trunk, and the key point connection data are corresponding joint connection relations or a set of body trunk connection relations, which can be stored by adopting a topological structure.
Therefore, the monitoring video is preprocessed by adopting image color space conversion and human joint key point identification at the edge end, the calculated amount of a cloud server is reduced, the calculation efficiency is improved, meanwhile, the identification error caused by environmental factors is eliminated, and the image identification accuracy is improved.
And S105, classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal based on a classification and identification result.
Specifically, as an implementation manner, when judging whether the behavior of a passenger in an elevator is an abnormal behavior, the cloud server may classify and identify the behavior of the passenger based on the key point connection data, the key point coordinate data and a passenger behavior identification model created in advance, and judge whether the behavior of the passenger is the abnormal behavior based on a classification and identification result.
The passenger behavior recognition model is formed by connecting an LSTM network in series with a plurality of SVM classifiers connected in parallel, and the number of the SVM classifiers is the same as the number of types of preset passenger abnormal behaviors.
In a specific implementation, the present embodiment may establish the passenger behavior recognition model based on a plurality of frames of images collected in advance.
As a specific implementation mode, when judging whether the behavior of passengers in the elevator is abnormal, firstly, selecting key point connection data and key point coordinate data of continuous N frames of images;
secondly, inputting the key point connection data and the key point coordinate data of the selected N frames of images as a group of data into a pre-established passenger behavior recognition model, and classifying and recognizing the behaviors of the passengers;
sliding M frames backwards according to the time sequence of the video frame images, reselecting key point connection data and key point coordinate data of the N frames of images, inputting the data into the passenger behavior recognition model, carrying out classification recognition on the passenger behaviors, and so on, and when the passenger behaviors are classified into the set abnormal behaviors for T times, judging the passenger behaviors as the abnormal behaviors; wherein N, M, T is a positive integer.
Specifically, as an embodiment, the value of N, M, T may be set as follows: n is a positive integer in the interval [30, 60], M is a positive integer in the interval [1, 5], and T is greater than or equal to 3.
Further, when the passenger behavior is detected to be abnormal behavior, the cloud server informs the property management terminal to take countermeasures.
The following describes the scheme of the embodiment in detail with reference to fig. 4 by taking the detection of the dangerous behavior of the elevator passenger as an example:
as shown in fig. 4, the flow of detecting the dangerous behavior of the elevator passenger in this embodiment includes:
and 8, aiming at the judged dangerous behaviors of the passengers, the cloud server timely informs the property to take countermeasures through the property management terminal.
Wherein, N is a positive integer in the interval of [30, 60], M is a positive integer in the interval of [1, 5 ].
According to the elevator passenger dangerous behavior detection method based on AI and image processing, the monitored video is preprocessed by adopting image color space conversion and human joint key point identification at the edge end, so that the calculated amount of a cloud server is reduced, the calculation efficiency is improved, and meanwhile, the identification error caused by environmental factors is eliminated; a passenger behavior recognition model is established for multi-frame images, passenger behaviors are accurately classified in a time period, and misjudgment caused by single-frame images or recognition errors can be effectively avoided.
In addition, as shown in fig. 5, an embodiment of the present invention further provides an elevator passenger abnormal behavior detection system, where the system includes:
the image acquisition module 10 is used for acquiring video frame images in the elevator;
an image conversion module 20, configured to convert the video frame image into an HSV color space image;
the image segmentation module 30 is configured to perform ROI region segmentation on the human body in the HSV color space image to obtain human body key points;
the identification module 40 is used for identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers;
and a behavior judging module 50, configured to perform classification and identification on the behavior of the passenger based on the key point connection data and the key point coordinate data, and judge whether the behavior of the passenger is an abnormal behavior based on a classification and identification result.
The principle and implementation process for detecting abnormal behaviors of passengers in an elevator are realized in this embodiment, please refer to the above embodiments, and are not described herein again.
In addition, the embodiment of the present invention further provides a terminal device, where the terminal device includes a memory, a processor, and a computer program stored on the memory and operable on the processor, and when the computer program is executed by the processor, the method for detecting abnormal behavior of elevator passengers according to the above embodiment is implemented.
Because the elevator passenger abnormal behavior detection program is executed by the processor, all technical solutions of all the embodiments are adopted, so that at least all the beneficial effects brought by all the technical solutions of all the embodiments are achieved, and detailed description is omitted.
Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the method for detecting abnormal behavior of an elevator passenger according to the above embodiment.
Furthermore, an embodiment of the present invention further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for detecting abnormal behavior of elevator passengers according to the above embodiment is implemented.
Because the elevator passenger abnormal behavior detection program is executed by the processor, all technical solutions of all the embodiments are adopted, so that at least all the beneficial effects brought by all the technical solutions of all the embodiments are achieved, and detailed description is omitted.
Compared with the prior art, the elevator passenger abnormal behavior detection method, the elevator passenger abnormal behavior detection system, the elevator passenger abnormal behavior detection terminal device and the elevator passenger abnormal behavior detection storage medium provided by the embodiment of the invention acquire the video frame image in the elevator; converting the video frame image into an HSV color space image; carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image to obtain human body key points; identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers; and classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal behavior based on a classification and identification result. Therefore, the monitoring video is preprocessed through image color space conversion and human joint key point identification, the calculated amount of a cloud server is reduced, the calculation efficiency is improved, meanwhile, the identification error caused by environmental factors is eliminated, and the identification accuracy of abnormal behaviors in the elevator is improved; in addition, passenger behaviors can be accurately classified in a time period, misjudgment caused by single-frame images or recognition errors can be effectively avoided, the recognition accuracy of abnormal behaviors in the elevator is further improved, and the detection requirement is met.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, a controlled terminal, or a network device) to execute the method of each embodiment of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (14)
1. An elevator passenger abnormal behavior detection method, characterized in that the method comprises the steps of:
acquiring a video frame image in an elevator;
converting the video frame image into an HSV color space image;
carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image to obtain human body key points;
identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers;
and classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal behavior based on a classification and identification result.
2. The method according to claim 1, wherein the step of classifying and identifying the behavior of the passenger based on the key point connection data and key point coordinate data, and the step of determining whether the behavior of the passenger is abnormal based on the classification and identification result comprises:
selecting key point connection data and key point coordinate data of continuous N frames of images;
inputting the selected key point connection data and key point coordinate data of the N frames of images as a group of data into a passenger behavior recognition model established in advance, and carrying out classification recognition on the behaviors of passengers;
sliding M frames backwards according to the time sequence of the video frame images, reselecting key point connection data and key point coordinate data of the N frames of images, inputting the data into the passenger behavior recognition model, carrying out classification recognition on the passenger behaviors, and so on, and when the passenger behaviors are classified into the set abnormal behaviors for T times, judging the passenger behaviors as the abnormal behaviors; wherein N, M, T is a positive integer.
3. The method according to claim 2, wherein N is a positive integer in the interval [30, 60], M is a positive integer in the interval [1, 5], and T is greater than or equal to 3.
4. The method according to claim 1, wherein the step of classifying and identifying the behavior of the passenger based on the key point connection data and key point coordinate data, and the step of determining whether the behavior of the passenger is abnormal based on the classification and identification result comprises:
classifying and identifying the behavior of the passenger based on the key point connection data, the key point coordinate data and a passenger behavior identification model established in advance through a cloud server, and judging whether the behavior of the passenger is abnormal based on a classification identification result.
5. The method of claim 4, further comprising:
and when the passenger behavior is detected to be abnormal behavior, the cloud server informs the property management terminal to take countermeasures.
6. The method of claim 4,
the step of converting the video frame image into an HSV color space image comprises:
converting, by an edge computing device, the video frame image into an HSV color space image;
the step of carrying out ROI region segmentation on the human body in the HSV color space image to obtain the key points of the human body comprises the following steps:
carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image through the edge computing device to obtain human body key points;
the step of identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers comprises the following steps:
and identifying each segmented human key point by the edge computing device by using a convolutional neural network to obtain key point connection data and key point coordinate data of the corresponding passenger, and transmitting the key point connection data and the key point coordinate data to the cloud server.
7. The method of claim 6, wherein the step of obtaining video frame images within the elevator comprises:
receiving an access instruction which is triggered by a user through a property management terminal and used for accessing the state of a camera in an elevator through the cloud server;
sending the access instruction to the edge computing device through the cloud server;
and acquiring data in the elevator through a camera corresponding to the edge computing device to obtain a video frame image in the elevator.
8. The method according to any one of claims 1-7, wherein the key point is a geometric center point of the occupant's joint or torso, and the key point connection data is a set of corresponding joint connections or torso connections, stored in a topological structure.
9. The method according to any one of claims 2-7, wherein the passenger behavior recognition model is formed by connecting an LSTM network in series with a plurality of SVM classifiers connected in parallel, and the number of SVM classifiers is the same as the number of types of the preset passenger abnormal behaviors.
10. The method according to any one of claims 2 to 7, wherein the step of performing classification recognition on the behavior of the passenger based on the key point connection data and key point coordinate data and determining whether the behavior of the passenger is abnormal based on the classification recognition result further comprises:
and establishing the passenger behavior recognition model based on a plurality of frames of images collected in advance.
11. An elevator passenger abnormal behavior detection system, characterized in that the system comprises:
the image acquisition module is used for acquiring a video frame image in the elevator;
the image conversion module is used for converting the video frame image into an HSV color space image;
the image segmentation module is used for carrying out ROI (region of interest) region segmentation on the human body in the HSV color space image to obtain human body key points;
the identification module is used for identifying the human body key points to obtain key point connection data and key point coordinate data of corresponding passengers;
and the behavior judgment module is used for classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data and judging whether the behavior of the passenger is abnormal based on a classification and identification result.
12. An elevator passenger abnormal behavior detection system, characterized in that the system comprises:
the edge computing device is used for acquiring a video frame image in the elevator, converting the video frame image into an HSV color space image, performing ROI (region of interest) region segmentation on a human body in the HSV color space image to obtain a human body key point, and identifying the human body key point to obtain key point connection data and key point coordinate data of a corresponding passenger;
and the cloud server is used for classifying and identifying the behavior of the passenger based on the key point connection data and the key point coordinate data, and judging whether the behavior of the passenger is abnormal based on a classification and identification result.
13. Terminal device, characterized in that the terminal device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, implements an elevator passenger abnormal behavior detection method according to any of claims 1-10.
14. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the elevator passenger abnormal behavior detection method according to any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111575522.0A CN114283362A (en) | 2021-12-21 | 2021-12-21 | Elevator passenger abnormal behavior detection method, system, terminal device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111575522.0A CN114283362A (en) | 2021-12-21 | 2021-12-21 | Elevator passenger abnormal behavior detection method, system, terminal device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114283362A true CN114283362A (en) | 2022-04-05 |
Family
ID=80874169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111575522.0A Pending CN114283362A (en) | 2021-12-21 | 2021-12-21 | Elevator passenger abnormal behavior detection method, system, terminal device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114283362A (en) |
-
2021
- 2021-12-21 CN CN202111575522.0A patent/CN114283362A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108446630B (en) | Intelligent monitoring method for airport runway, application server and computer storage medium | |
| JP6905850B2 (en) | Image processing system, imaging device, learning model creation method, information processing device | |
| CN110781914B (en) | Equipment fault monitoring and processing method, device, equipment and storage medium | |
| CN106372576A (en) | Deep learning-based intelligent indoor intrusion detection method and system | |
| CN110490171B (en) | Dangerous posture recognition method and device, computer equipment and storage medium | |
| CN111523559B (en) | Abnormal behavior detection method based on multi-feature fusion | |
| CN111047824B (en) | Indoor child nursing linkage control early warning method and system | |
| CN112184773A (en) | Helmet wearing detection method and system based on deep learning | |
| CN109544870A (en) | Alarm decision method and intelligent monitor system for intelligent monitor system | |
| CN111191507A (en) | Safety early warning analysis method and system for smart community | |
| CN116664359A (en) | Intelligent fire early warning decision system and method based on multi-sensor fusion | |
| CN114092877A (en) | Garbage can unattended system design method based on machine vision | |
| CN115620471A (en) | Image identification security system based on big data screening | |
| CN113955594B (en) | Elevator control method and device, computer equipment and storage medium | |
| CN114332925A (en) | Method, system and device for detecting pets in elevator and computer readable storage medium | |
| CN112528825A (en) | Station passenger recruitment service method based on image recognition | |
| CN114283362A (en) | Elevator passenger abnormal behavior detection method, system, terminal device and storage medium | |
| CN114758286B (en) | Intelligent edge safety monitoring method and device based on work ticket event | |
| CN115953815A (en) | Monitoring method and device for infrastructure site | |
| CN115180522A (en) | Safety monitoring method and system for hoisting device construction site | |
| CN114387665A (en) | Unmanned staircase identification system ascends a height based on portable cloth accuse ball | |
| CN114359839A (en) | Method and system for identifying entrance of electric vehicle into elevator | |
| CN107749942A (en) | Suspension image pickup method, mobile terminal and computer-readable recording medium | |
| CN113177453A (en) | Intelligent monitoring system for intelligent community based on image recognition | |
| CN116311181B (en) | Method and system for rapidly detecting abnormal driving |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |