CN114550044A

CN114550044A - Elevator car abnormal event monitoring and early warning system based on target detection network

Info

Publication number: CN114550044A
Application number: CN202210162438.4A
Authority: CN
Inventors: 路文; 龙钢; 李玎; 朱振杰; 朱志强
Original assignee: Xidian University
Current assignee: Xidian University
Priority date: 2022-02-22
Filing date: 2022-02-22
Publication date: 2022-05-27

Abstract

The invention discloses an elevator car abnormal event monitoring and early warning system based on a target detection network, which mainly solves the problems of network delay, incapability of monitoring in real time and low computing power in the prior art. It includes: elevator car, switch and server install control panel, camera and speaker in this elevator car, the camera is connected gradually with switch and server, and the camera is connected respectively with control panel and speaker. Be equipped with artificial intelligence chip and different functional module in the camera, carry out the target detection through the camera to the video of shooing, when detecting the electric motor car, the speaker sends alarm signal, and control panel control elevator car stops to move. Meanwhile, the camera encodes and pushes the video code stream with the detection result, and the server receives, decodes and displays the video code stream. The invention can avoid network time delay, send out alarm signal in real time, control elevator operation, improve computing power and computing speed, and can be used for accurate monitoring and identification of the electric vehicle.

Description

Elevator car abnormal event monitoring and early warning system based on target detection network

Technical Field

The invention belongs to the technical field of target identification, and particularly relates to an elevator car abnormal event monitoring and early warning system which can be used for accurately monitoring and identifying an electric vehicle, early warning an electric vehicle intrusion event in a boundary area in real time and protecting the safety of an elevator.

Background

With the vigorous development of the economy in China, whether in large cities or small towns, high-rise buildings and mansions stand, the elevator walks into the lives of people along with the high-rise buildings and the small towns, and the safety problem of the elevator also becomes a focus. Because people's elevator safe use consciousness is weak, the phenomenon that the electric motor car freely gets in and out of the elevator car is gradually increased. The electric motor car is taken on the elevator and placed in a corridor to be charged, so that the load of the elevator can be increased, and potential safety hazards also exist. Many policies are set by relevant departments and units for prohibiting the electric vehicle from driving into the elevator, but the effects are little. The electric vehicle can be prevented from entering the elevator car effectively by the elevator forbidden monitoring and early warning system aiming at the problem that the electric vehicle enters the elevator and goes upstairs for charging. The system introduces an artificial intelligence and automation solution, accurately monitors and analyzes the electric vehicle, realizes analysis and identification of the electric vehicle, and warns the electric vehicle intrusion event in the perimeter area in real time. The installation of the electric vehicle entrance-forbidding elevator identification and control system is beneficial to promoting the safe operation of the urban elevators and is also beneficial to the further development of urban communities. And further, the elevator monitoring equipment is clear and qualitatively understood, and an exclusive intelligent security monitoring system for cities and industries is established.

The patent document with the application number of 202011311209.1 discloses an elevator electric vehicle recognition and early warning method based on AI and HMM. Firstly, preprocessing an original image of an elevator monitoring video; inputting the preprocessed images into a trained AI-based electric vehicle identification network model for identification to obtain identification results of video frames, judging whether each frame has an electric vehicle, and forming an electric vehicle identification sequence on the electric vehicle identification results of multiple frames of videos; and finally, inputting the electric vehicle recognition sequence results of multiple frames into a trained elevator state recognition network model based on the HMM, and outputting the elevator state to realize elevator early warning. According to the method, as the video data are transmitted to the server for processing, the problem of network delay exists, and an alarm signal cannot be sent out in real time.

Shanxi Hongyuan special equipment research institute limited company in the patent document with application number 202010479416.1 discloses a method and a system for detecting the abnormity of an elevator car aiming at the recognition of an electric car, so as to detect the event that the electric car breaks into the elevator car. Firstly, constructing a special training data set for the electric vehicle; then, through iterative optimization training, the accuracy of the electric vehicle identification model is improved; and finally, deploying the electric vehicle identification model on a raspberry group, connecting the raspberry group with a camera at the top of the elevator car through a wired network, and solving the problem of network delay depending on the cloud by adopting an edge calculation mode. Compared with a special neural network processor, the raspberry pi adopted by the method has limited resources and low computational power, so that the final detection precision and the real-time performance cannot achieve ideal effects.

Disclosure of Invention

The invention aims to provide an elevator cage abnormal event monitoring and early warning system based on a target detection network aiming at the defects of the prior art, so as to avoid network time delay, send out an alarm signal in real time, control the operation of an elevator and improve the computational power and the computational speed.

In order to achieve the above object, the elevator car abnormal event monitoring and early warning system based on the target detection network of the present invention comprises: elevator car, switch and server install control panel, camera and speaker in this elevator car, and the camera connects gradually its characterized in that with switch and server:

be equipped with artificial intelligence chip and following functional module in the camera:

a target detection module: the target detection is carried out on the video frame;

the video code stream coding and pushing module: the video server is used for coding the video code stream and pushing the coded video code stream to the server through the switch;

an alarm module: the alarm device is used for sending an alarm signal through a loudspeaker when the electric vehicle is detected;

a control module: the elevator car control system is used for feeding back a signal whether the electric car is detected or not to the control board so as to control the operation of the elevator car;

the server is internally provided with the following functional modules:

the video code stream acquiring and decoding module: the device is used for acquiring the video code stream transmitted by the camera and decoding the acquired video code stream;

a visualization module: used for displaying the result of the target detection and the elevator control state.

Further, a neural network processor NPU is arranged in the artificial intelligence chip and used for calculating the target detection network model.

Further, the object detection module includes:

a preprocessing submodule: the system comprises a sensor, a gamma correction unit, a data processing unit and a data processing unit, wherein the data processing unit is used for carrying out interpolation on video frames acquired by the sensor to obtain low-resolution image data, and then carrying out brightness and color correction on the low-resolution image data by adopting gamma correction to obtain processed image data;

training a submodule: the system is used for training a single-stage multi-label target detection network;

a quantization submodule: the network model is used for carrying out quantitative processing on the trained network model;

deploying an implementation submodule: the intelligent network model is used for deploying the quantized network model on an artificial intelligence chip in the camera, and carrying out reasoning calculation on processed image data through a neural network processor of the artificial intelligence chip so as to realize detection on the electric vehicle target.

Further, the training submodule trains the single-stage multi-label target detection network, 33725 training data of various targets in the elevator collected in a real elevator scene are input into the single-stage multi-label target detection network, errors between actual output values and expected output values are sequentially transmitted to previous layers from the last output layer through a back propagation algorithm, and an adaptive moment estimation Adam optimizer is used for continuously adjusting and optimizing connection weights and biases of the layers to reduce the errors until the errors tend to be stable, so that a trained network model is obtained.

Further, the quantization submodule performs quantization processing on the trained network model, namely, the weight of the network model is converted from a 32-bit floating point type to an 8-bit integer type, so that the complexity of the model is reduced, the storage space of the model is reduced, and the inference calculation of the network model is accelerated under the condition of ensuring the detection precision

Furthermore, the video code stream coding and pushing module codes the video code stream, namely, the boundary frame identified by the target detection module is firstly drawn into the original video code stream, and then the video code stream with the operation result is coded into the H264 format video code stream through the coding command.

Further, the video code stream coding and pushing module pushes the coded video code stream to the server through the switch, which is realized through a real-time streaming protocol RTSP.

Further, the camera is respectively connected with the loudspeaker and the control panel, when the target detection module in the camera detects the electric vehicle, alarm signals for forbidding the elevator are respectively sent out through the loudspeaker, the elevator car is controlled to stop running through the control panel, and the loudspeaker stops giving an alarm until the electric vehicle cannot be detected, and the elevator car resumes running.

Compared with the prior art, the invention has the following advantages:

first, the invention has real-time property and accelerates the reasoning calculation of the network model because the target detection module is arranged in the camera and the camera is arranged in the elevator car, and the electric vehicle can be detected in real time at the edge end, so as to send out an alarm signal and control the operation of the elevator car without considering the problem of network delay.

Secondly, because the artificial intelligence chip is arranged in the camera and the artificial intelligence chip is provided with the special neural network processor, the computing power and the throughput rate are improved compared with other processors.

Thirdly, the visual module is arranged in the server, so that the target detection result can be displayed on the server, the control state of the elevator can be displayed, and monitoring personnel can conveniently and directly obtain the condition in the elevator car.

Drawings

FIG. 1 is a system block diagram of the present invention;

FIG. 2 is a schematic diagram of the internal structure of a camera in the system of the present invention;

FIG. 3 is a schematic diagram of the operation of the system of the present invention for monitoring and warning abnormal events of an elevator car;

FIG. 4 is a diagram of a visualization interface in the system of the present invention.

Detailed Description

The following describes the real-time embodiment and effects of the present invention in further detail with reference to the accompanying drawings:

referring to fig. 1, this example is based on elevator car abnormal event monitoring early warning system of target detection network, including elevator car 1, control panel 2, camera 3, speaker 4, switch 5 and server 6, wherein, control panel 2 is elevator car 1's self-contained part, and camera 3 and speaker 4 are installed in the different positions of elevator car 1, and this camera 3 passes through the signal line to be connected with control panel 2 and speaker 4 respectively, and camera 3 passes through the net twine and is connected with switch 5, and switch 5 and server 6 pass through the net twine and connect.

An artificial intelligence chip 31 and four functional modules, namely a target detection module 32, a video code stream coding and pushing module 33, an alarm module 34 and a control module 35, are arranged in the camera 3, as shown in fig. 2. Wherein:

the artificial intelligence chip 31 is internally provided with a neural network processor NPU for reasoning and calculating a target detection network model.

The target detection module 32 includes a preprocessing submodule 321, a training submodule 322, a quantization submodule 323, and a deployment implementation submodule 324, and the four submodules function as follows:

the preprocessing submodule 321 is configured to perform interpolation on the video frame acquired by the sensor to obtain low-resolution image data, and then perform gamma correction on the low-resolution image data to obtain processed image data;

the training submodule 322 is used for training the single-stage multi-label target detection network, namely 33725 training data of various targets in the elevator collected in a real elevator scene are input into the single-stage multi-label target detection network, errors between actual output values and expected output values are sequentially transmitted to previous layers from the last output layer through a back propagation algorithm, and an adaptive moment estimation Adam optimizer is used for continuously adjusting and optimizing connection weights and offsets of the layers to reduce the errors until the errors tend to be stable, so that a trained network model is obtained;

the quantization submodule 323 is used for performing quantization processing on the trained network model, converting the weight of the network model from a 32-bit floating point type to an 8-bit integer type, so as to reduce the complexity of the model, reduce the storage space of the model, accelerate the inference calculation of the network model, realize the lightweight of the target detection model and ensure that the target detection model meets the real-time requirement of engineering application under the condition of ensuring the detection precision;

the deployment implementation submodule 324 is configured to deploy the quantized network model on the artificial intelligence chip 31 inside the camera, and perform inference calculation on the processed image data through the neural network processor of the artificial intelligence chip 31 to implement detection on the electric vehicle target.

The video stream coding and pushing module 33 is configured to code and push a video stream, that is, draw a bounding box identified by the target detection module 32 into an original video stream, code the video stream with an operation result into an H264 format video stream through a coding command, and finally push the coded video stream to the server 6 through the switch 5 by using the existing real-time streaming protocol RTSP.

The alarm module 34 is configured to send an alarm signal, that is, when the target detection module 32 in the camera 3 detects the electric vehicle, the camera 3 sends a signal to the speaker 4, and after receiving the signal, the speaker 4 sends an alarm signal to prohibit entering the elevator, and until the electric vehicle cannot be detected by the camera 3, the speaker 4 stops giving an alarm.

The control module 35 is used for controlling the operation of the elevator car 1, namely when the target detection module 32 in the camera 3 detects the electric vehicle, the camera 3 sends a signal to the control panel 2, and after the control panel 2 receives the signal, the elevator car 1 is controlled to stop operating until the electric vehicle can not be detected by the camera 3, and the elevator car 1 resumes operating.

The server 6 is internally provided with two functional modules, namely a video code stream acquisition and decoding module 61 and a visualization module 62. Wherein:

the video code stream acquiring and decoding module 61 is configured to acquire a video code stream transmitted by a camera and decode the acquired video code stream, that is, first acquire the video code stream transmitted by the camera through a real-time streaming protocol RTSP, and then decode the video code stream into an H264 format through a video decoding command.

The visualization module 62 is configured to display the result of the target detection and the elevator control status, and includes a video frame display with a target detection bounding box, a current recognition status display, and an elevator control status display, as shown in fig. 4, where:

the current identification state includes: no target, target confirmation, target entering, target exiting and target stationary;

the elevator control state includes: no alarm, sending alarm, failure of alarm, in-process alarm, alarm release and released alarm.

Referring to fig. 3, the working principle of monitoring and warning the abnormal event of the elevator car by using the system of the embodiment is as follows:

firstly, the camera carries out real-time electric vehicle detection on the shot video in the elevator car, if the camera detects the electric vehicle, the camera sends signals to the loudspeaker and the control panel respectively, after the loudspeaker and the control panel receive the signals, the loudspeaker and the control panel send alarm signals for prohibiting the elevator and control the elevator car to stop running respectively, and the elevator car resumes running until the camera cannot detect the electric vehicle, the loudspeaker stops giving an alarm, so that the electric vehicle is monitored at the edge end in real time, the alarm signals are sent, and the elevator is controlled to run. Meanwhile, the video code stream with the detection result is coded and then transmitted to the server through the switch, the video code stream is received and decoded at the server end, and finally the video picture with the target detection boundary frame, the current identification state and the elevator control state are displayed at the background, so that monitoring personnel can conveniently and directly obtain the situation in the elevator car, and the accident situation can be timely processed.

The effects of the present invention can be further illustrated by the following actual measurement results:

the system of the present example was installed and tested in one elevator in a residential area. Because the frequency of the electric vehicle entering the elevator car is relatively low, in order to make the test result stable and reliable, the example system continuously runs for 172 hours, and various data in the running time period of the example system are counted to obtain the data shown in the following table.

TABLE 1 example System operation results data statistics Table

Processing video frame numbers	3715200 frame
		Detecting the number of target frames	1073 frame
Number of times of triggering alarm	9 times of
		False detection rate	0.23％

As can be seen from the data in Table 1, in the operation time period of the example system, 3715200 frames of video frames are processed together, 1073 frames of target frames are detected, and 9 times of alarm triggering are performed, which shows that the system can successfully monitor the electric vehicle and trigger the alarm.

Generally, when the electric vehicle in the elevator car is detected, objects with characteristics similar to those of the electric vehicle can be easily detected into the electric vehicle, such as a bicycle, a barreled object and the like, and the false detection rate of the system of the embodiment is only 0.23%, which shows that the system of the embodiment can well distinguish the electric vehicle from the objects with characteristics similar to those of the electric vehicle.

Claims

1. The utility model provides an elevator car abnormal events monitoring and early warning system based on target detection network, includes elevator car (1), switch (5) and server (6), installs control panel (2), camera (3) and speaker (4) in this elevator car, and camera (3) and switch (5) and server (6) connect gradually its characterized in that:

be equipped with artificial intelligence chip (31) and following functional module in camera (3):

target detection module (32): the target detection is carried out on the video frame;

video code stream coding and pushing module (33): the video coding and decoding device is used for coding a video code stream and pushing the coded video code stream to a server (6) through a switch (5);

alarm module (34): the alarm device is used for sending an alarm signal through a loudspeaker (4) when the electric vehicle is detected;

a control module (35): the system is used for feeding back whether an electric vehicle signal is detected to the control panel (2) so as to control the operation of the elevator car (1);

the server (6) is internally provided with the following functional modules:

video code stream acquisition and decoding module (61): the device is used for acquiring the video code stream transmitted by the camera and decoding the acquired video code stream;

visualization module (62): and the elevator control system is used for displaying the result of target detection and the elevator control state.

2. The system according to claim 1, wherein the artificial intelligence chip (31) is built in with a neural network processor NPU for computation of a target detection network model.

3. The system as set forth in claim 1, wherein the object detection module (32) includes:

a pre-processing submodule (321): the system comprises a sensor, a gamma correction unit, a data processing unit and a data processing unit, wherein the data processing unit is used for carrying out interpolation on video frames acquired by the sensor to obtain low-resolution image data, and then carrying out brightness and color correction on the low-resolution image data by adopting gamma correction to obtain processed image data;

training submodule (322): the system is used for training the single-stage multi-label target detection network;

quantization submodule (323): the network model is used for carrying out quantitative processing on the trained network model;

deployment implementation submodule (324): the intelligent detection system is used for deploying the quantized network model on an artificial intelligence chip (31) in the camera, and carrying out reasoning calculation on the processed image data through a neural network processor of the artificial intelligence chip (31) so as to realize detection on the electric vehicle target.

4. The system of claim 3, wherein the training sub-module (322) trains the single-stage multi-label target detection network by inputting 33725 targets in the elevator collected from a real elevator scene into the single-stage multi-label target detection network, sequentially propagating errors between actual output values and expected output values from the last output layer to the previous layers through a back propagation algorithm, and continuously adjusting and optimizing connection weights and offsets of the layers by using an adaptive moment estimation Adam optimizer to reduce the errors until the errors tend to be stable, so as to obtain a trained network model.

5. The system according to claim 3, wherein the quantization submodule (323) quantizes the trained network model by converting the weight of the network model from 32-bit floating point type to 8-bit integer type, so as to reduce the complexity of the model, reduce the storage space of the model and accelerate the inference calculation of the network model while ensuring the detection precision.

6. The system according to claim 1, wherein the video stream coding and pushing module (33) codes the video stream by drawing a bounding box identified by the target detection module (32) into the original video stream, and then coding the video stream with the operation result into the H264 format video stream through a coding command.

7. The system according to claim 1, wherein the video stream coding and pushing module (33) pushes the coded video stream to the server (6) through the switch (5) by means of a real-time streaming protocol RTSP.

8. The system according to claim 1, wherein the camera (3) is respectively connected with the loudspeaker (4) and the control panel (2), when the target detection module (32) in the camera detects the electric vehicle, an alarm signal for forbidding the elevator is respectively sent out through the loudspeaker (4), the elevator car (1) is controlled to stop running through the control panel (2), until the electric vehicle is not detected, the loudspeaker (4) stops alarming, and the elevator car (1) resumes running.

9. The system according to claim 1, wherein the video stream acquiring and decoding module (61) decodes the acquired video stream by acquiring the video stream transmitted by the camera through the real-time streaming protocol RTSP and then decoding the video stream into the H264 format through the video decoding command.

10. The system of claim 1, wherein the visualization module (62) displays results of object detection and elevator control status, including a video screen display with an object detection bounding box, a current recognition status display, and an elevator control status display;

the elevator control state includes: no alarm, sending alarm, failure of alarm, alarm in process, and alarm release and neutralization alarm release.