CN116534688A - Elevator trapped person detection method based on video analysis - Google Patents
Elevator trapped person detection method based on video analysis Download PDFInfo
- Publication number
- CN116534688A CN116534688A CN202310435812.8A CN202310435812A CN116534688A CN 116534688 A CN116534688 A CN 116534688A CN 202310435812 A CN202310435812 A CN 202310435812A CN 116534688 A CN116534688 A CN 116534688A
- Authority
- CN
- China
- Prior art keywords
- elevator
- frame
- passenger
- passengers
- trapped
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 26
- 238000004458 analytical method Methods 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 21
- 238000012544 monitoring process Methods 0.000 claims abstract description 16
- 230000002159 abnormal effect Effects 0.000 claims abstract description 9
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims abstract description 4
- 230000005856 abnormality Effects 0.000 claims description 6
- 230000003068 static effect Effects 0.000 claims description 6
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 3
- 230000003044 adaptive effect Effects 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
- B66B5/0006—Monitoring devices or performance analysers
- B66B5/0012—Devices monitoring the users of the elevator system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B13/00—Doors, gates, or other apparatus controlling access to, or exit from, cages or lift well landings
- B66B13/02—Door or gate operation
- B66B13/14—Control systems or devices
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B3/00—Applications of devices for indicating or signalling operating conditions of elevators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
- B66B5/0006—Monitoring devices or performance analysers
- B66B5/0018—Devices monitoring the operating condition of the elevator system
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B50/00—Energy efficient technologies in elevators, escalators and moving walkways, e.g. energy saving or recuperation technologies
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Indicating And Signalling Devices For Elevators (AREA)
Abstract
The invention belongs to the technical field of elevator monitoring. The elevator trapped person detection method based on the video analysis is strong in real-time performance and adaptive to multiple scenes, so that the workload of manual monitoring is reduced. The technical proposal is as follows: a method for detecting elevator trapped people based on video analysis; the method comprises the following steps: step 1: manually marking passengers, elevator doors and floor readings of each frame of image in an elevator car monitoring video, and training by using a deep convolutional neural network by taking the passenger, the elevator doors and the floor readings as data sets to obtain a multitask detection model LiftDet for passenger target tracking, elevator door state judgment and floor reading identification; step 2: aiming at a video to be detected, a detection set Ro is obtained by using a multitasking detection model LiftDet; step 3: and selecting a part of the elevator car monitoring video, which is not subjected to abnormal people trapping, and evaluating the standard limit frame number of normal stay of passengers.
Description
Technical Field
The invention belongs to the technical field of elevator monitoring, and particularly relates to an elevator personnel trapping abnormality detection method based on video analysis.
Background
The elevator is transportation equipment with higher use frequency, and failure in maintenance, part aging, irregular use and the like can cause failure, and the elevator is one of common phenomena caused by elevator operation failure. The elevator is an abnormal phenomenon that the elevator suddenly stops running in normal operation due to the disconnection of a safety loop or a door lock loop, the failure or misoperation of a control system, the power failure of a power supply system, the human being and the like, and passengers are detained in the elevator.
In order to accurately detect the abnormal elevator trapping situation and timely take emergency measures, chinese patent application (application number 202111412911.1) discloses an elevator trapping detection system, which adopts an infrared sensor combined with a sliding rail and a gravity sensor signal of a car, sets a people carrying time threshold to judge whether an elevator is trapped, and uses a motor push rod to automatically trigger an emergency button. Chinese patent (application No. 20201041436. X) discloses a computer vision-based elevator trapping detection method, which detects elevator trapping by judging whether an elevator is closed and whether the closing time exceeds a set threshold value, and uses a convolutional neural network to identify information such as the number of people in the elevator and feed back to a monitoring center. Ma Zhenbao (video image-based intelligent alarm system for elevator car trapping [ J ]. Chinese elevator, 2022) provides a system for realizing intelligent alarm for elevator car trapping by using video images, wherein the system obtains the motion state of a car door by adopting an edge detection technology according to the constituent elements of elevator trapping, recognizes floor information by adopting an OCR recognition technology, performs background modeling by adopting the edge detection technology, and judges whether the elevator carries passengers or not by background subtraction, thereby realizing intelligent detection alarm. However, the method adopts a manually designed feature extraction and identification method, and the method needs to manually adjust the super parameters of the algorithm in different scenes.
In summary, the shortcomings of the prior art scheme are: (1) In elevator trapped person detection based on sensor signals, the signals of the sensors are accurate but not rich, and the robustness is poor in complex scenes. (2) In the elevator trapped person detection method based on vision, the manual design feature extraction and the pattern recognition are mainly used, and the deep learning algorithm is only used for auxiliary judgment of information acquisition of the number of elevator people and the like.
Disclosure of Invention
The invention aims to overcome the defects of the background technology, and provides an elevator trapped person detection method based on video analysis, which has strong real-time performance and is adaptive to multiple scenes, so as to reduce the workload of manual monitoring.
The technical scheme provided by the invention is as follows:
a method for detecting elevator trapped people based on video analysis; the method comprises the following steps:
step 1: manually marking passengers, elevator doors and floor readings of each frame of image in an elevator car monitoring video, and training by using a deep convolutional neural network by taking the passenger, the elevator doors and the floor readings as data sets to obtain a multitask detection model LiftDet for passenger target tracking, elevator door state judgment and floor reading identification;
step 2: aiming at the video to be detected, a detection set R is obtained by using a multitasking detection model LiftDet o ；
R o ＝{(m ij ,c ij ,d i ,l i )|i＝1,2,…,K,j＝1,2,3,…,M i }；
Wherein: m is m ij Representing the coordinates of the center point of the BoundingBox of the jth passenger in the ith frame and the width and height of the seat;
c ij an ID number indicating the jth passenger in the ith frame;
d i representing the status of the elevator door in the i-th frame, the specific status being determined by formula (1);
l i indicating the floor indication of the elevator in the ith frame;
k represents the number of video frames;
M i representing the total number of passengers in the i-th frame;
step 3: selecting a part of the elevator car monitoring video, which is not subjected to abnormal people trapping, and evaluating the standard limit frame number of normal stay of passengers; the method comprises the following steps: the frame numbers of passengers with ID number c entering and exiting the elevator are respectively marked asAnd->Calculating the number of stay frames of the passenger according to (2)>Maximum number of stay frames according to (3)>Updating; calculating a standard limit frame number t of normal stay according to the formula (4);
wherein c represents the ID number of the passenger,indicating that the passenger enters an elevator keyframe, +.>Indicating that the passenger left the elevator keyframe, +.>The number of passengers staying in the elevator is represented by xi, the update coefficient of the maximum staying frame number is represented by L, the maximum floor number of the elevator is represented by t, and the standard limiting frame number of passengers staying in the elevator is represented by t;
step 4: respectively selecting continuous frame sequences of elevator floor number static unchanged and elevator door closed from current frame kAnd a sequence of consecutive frames with elevator door closed +.>Record->For the static frame number of elevator floor->Closing the elevator door for a number of frames; if the current frame k meets the formula (5), recording k as a judging key frame of the man-trapped abnormality;
wherein M is k Representing the number of passengers in the elevator under the kth frame of image;
step 5: selecting a subsequent continuous frame ZP= { i|i=k, k+1, …, k+t } by taking the judging key frame k of the trapped abnormality as a starting point;
calculating an elevator people trapping anomaly coefficient p from the (6) and (7) i ；
Wherein t represents the standard limit frame number of the passenger stop, d i Indicating the door closed status of the i-th frame, l i Floor indication indicating the i-th frame;
step 6: calculating an elevator people trapping abnormal threshold value alpha according to the formula (8); if p i >Alpha, judging that the current elevator has a problem of people trapping;
wherein M is i Indicating the number of passengers in the ith frame, W i Representing the number of passenger ID matches between the i-th frame and the i-1 th frame;
step 7: the elevator people trapping condition is evaluated, specifically: calculating the number n of passengers currently trapped in the elevator according to the formula (9); calculating the trapped time t of the elevator in real time according to the formula (10);
wherein r is s Representing the frame rate of video, f i A frame number representing the current frame;
compared with the prior art, the invention has the beneficial effects that: the elevator car monitoring system and the elevator car monitoring method based on the video frame by frame analysis track passengers, detect the closing state of the elevator door and identify floor readings in real time, accurately judge whether the elevator has the problem of trapping people and evaluate the number of the trapped people and the time length of the trapped people, have the characteristics of strong real-time performance and self-adaption multiple scenes, and can reduce the workload of manual monitoring, thereby providing references for the rescue of maintenance personnel and monitoring centers.
Drawings
Fig. 1 is an image of the result of a multi-task test in an elevator surveillance video.
Fig. 2 is an image of a process of detecting a trapped person in an elevator surveillance video.
Fig. 3 is an image of the result of the calculation of the length of time trapped in the elevator surveillance video.
Detailed Description
The invention will be described in detail with reference to examples.
The invention discloses an elevator trapped person detection method based on video analysis, which specifically comprises the following steps:
step 1: manually marking passengers, elevator doors and floor readings of each frame in an elevator car monitoring video, and training by using a deep convolutional neural network by taking the passenger, the elevator doors and the floor readings as data sets to obtain a multitask detection model LiftDet for passenger target tracking, elevator door state judgment and floor reading identification;
step 2: aiming at the video to be detected, a detection set R is obtained by using a multitasking detection model LiftDet o ；
R o ＝{(m ij ,c ij ,d i ,l i )|i＝1,2,…,K,j＝1,2,3,…,M i -a }; wherein m is ij Representing the coordinates of the center point of the BoundingBox of the jth passenger in the ith frame and the width and height, c ij An ID number, d, indicating the jth passenger in the ith frame i Representing the status of the elevator door in the i-th frame, the specific status being determined by the formula (1), l i Indicating the floor indication of the elevator in the ith frame, K indicating the number of video frames, M i Representing the total number of passengers in the i-th frame; in this embodiment, as shown in fig. 1, the result of the multitasking detection model is that the ID number of the passenger is 27, the floor indication number is 9, and the elevator door is in a closed state;
step 3: selecting a part of the elevator car monitoring video, which is not in abnormal condition, and evaluating the standard limit frame number of normal stay of passengers, wherein the method specifically comprises the following steps: the frame number of the passengers with ID number c entering and exiting the elevator is recorded asAnd->Calculating the number of the passenger's stay frames according to (2) and (3)>And +.>Updating; calculating a standard limit frame number f of normal stay according to the formula (4); in the present embodiment, the standard limit frame number is 1776;
wherein c represents the ID number of the passenger,indicating that the passenger enters an elevator keyframe, +.>Indicating that the passenger left the elevator keyframe, +.>The number of passengers staying in the elevator is represented by xi, the update coefficient of the maximum staying frame number is represented by L, the maximum floor number of the elevator is represented by t, and the standard limiting frame number of passengers staying in the elevator is represented by t;
step 4: respectively selecting continuous frame sequences of elevator floor number static unchanged and elevator door closed from current frame kAnd->Record->For the static frame number of elevator floor->Closing the elevator door for a number of frames; if the current frame k meets the formula (5), recording k as a judging key frame of the man-trapped abnormality; as shown in fig. 2, in the present embodiment, the number of elevator floor stationary frames is 1777, and the number of elevator door closing frames is 2116;
wherein M is k Representing the number of passengers in the elevator at the kth frame;
step 5: selecting a subsequent continuous frame zp= { i|i=k, k+1, …, k+t } with the key frame k as a starting point; calculating an elevator people trapping anomaly coefficient p from the (6) and (7) i The method comprises the steps of carrying out a first treatment on the surface of the In this embodiment, the anomaly coefficient of elevator trapped is 0.973;
wherein t represents the standard limit frame number of the passenger stop, d i Indicating the door closed status of the i-th frame, l i Floor indication indicating the i-th frame;
step 6: calculating an elevator people trapping anomaly threshold value alpha (i.e. confidence) from formula (8); if p i >Alpha, judging that the current elevator has a problem of people trapping; in this embodiment, the confidence α is 0.5;
wherein M is i Indicating the number of passengers in the ith frame, W i Representing the number of passenger ID matches between the i-th frame and the i-1 th frame;
step 7: the elevator people trapping condition is evaluated, specifically: calculating the number n of passengers currently trapped in the elevator according to the formula (9); calculating the trapped time t of the elevator in real time according to the formula (10); in this embodiment, the number of passengers is 1, and the trapping time is 142s, as shown in fig. 3;
wherein r is s Representing the frame rate of video, f i Representing the frame number of the current frame.
Claims (7)
1. A method for detecting elevator trapped people based on video analysis; the method comprises the following steps:
step 1: manually marking passengers, elevator doors and floor readings of each frame of image in an elevator car monitoring video, and training by using a deep convolutional neural network by taking the passenger, the elevator doors and the floor readings as data sets to obtain a multitask detection model LiftDet for passenger target tracking, elevator door state judgment and floor reading identification;
step 2: aiming at the video to be detected, a detection set R is obtained by using a multitasking detection model LiftDet o ；
Step 3: selecting a part of the elevator car monitoring video, which is not subjected to abnormal people trapping, and evaluating the standard limit frame number of normal stay of passengers;
step 4: respectively selecting continuous frame sequences of elevator floor number static unchanged and elevator door closed from current frame kAnd a continuous frame sequence MP with elevator doors closed;
step 5: selecting a subsequent continuous frame ZP= { i|i=k, k+1, …, k+t } by taking the judging key frame k of the trapped abnormality as a starting point; calculating an elevator trapped anomaly coefficient p i ；
Step 6: calculating an elevator people trapping abnormal threshold alpha; if p i >Alpha, judging that the current elevator has a problem of people trapping;
step 7: the elevator is evaluated for the condition of getting trapped.
2. The method for detecting elevator trapping based on video analysis according to claim 1, wherein: the detection set R in the step 2 o ＝{(m ij ,c ij ,d i ,l i )|i＝1,2,…,K,j＝1,2,3,…,M i }；
Wherein: m is m ij Representing the coordinates of the center point of the BoundingBox of the jth passenger in the ith frame and the width and height of the seat;
c ij an ID number indicating the jth passenger in the ith frame;
d i representing the status of the elevator door in the i-th frame, the specific status being determined by formula (1);
l i indicating the floor indication of the elevator in the ith frame;
k represents the number of video frames;
M i representing the total number of passengers in the i-th frame;
。
3. the elevator trapped person detection method based on video analysis according to claim 2, wherein: the specific method for evaluating in the step 3 is as follows:
the frame numbers of passengers with ID numbers c entering and exiting the elevator are respectively marked as f s c Andcalculating the number of stay frames of the passenger according to (2)>Maximum number of stay frames according to (3)>Updating; calculating a standard limit frame number t for normal stay of the passenger according to the formula (4);
wherein c represents the ID number of the passenger, f s c Indicating that the passenger is entering an elevator keyframe,indicating that the passenger left the elevator keyframe, +.>The number of passengers staying in the elevator is represented by xi, the update coefficient of the maximum number of passengers staying in the elevator is represented by L, the maximum floor number of the elevator is represented by t, and the standard limit number of passengers staying normally is represented by t.
4. The method for detecting elevator trapping based on video analysis according to claim 3, wherein: the expression of the continuous frame sequence MP with the closed elevator door in the step 4 is as follows;
for the static frame number of elevator floor->Closing the elevator door for a number of frames;
if the current frame k meets the formula (5), recording k as a judging key frame of the man-trapped abnormality;
wherein M is k Indicating the number of passengers in the elevator for the kth frame of image.
5. The method for detecting elevator trapping based on video analysis according to claim 4, wherein: the elevator trapped anomaly coefficient p in the step 5 i Calculated from formulas (6) and (7):
wherein t represents the standard limit frame number of the passenger stop, d i Indicating the door closed status of the i-th frame, l i The floor indication representing the i-th frame.
6. The method for detecting elevator trapping based on video analysis according to claim 5, wherein: the elevator people trapping abnormal threshold alpha in the step 6 is calculated by the formula (8):
wherein M is i Indicating the number of passengers in the ith frame, W i Indicating the number of passenger ID matches between the i-th frame and the i-1 frame.
7. The method for detecting elevator trapping based on video analysis according to claim 6, wherein: the method for evaluating the elevator trapped condition in the step 7 is as follows:
calculating the number n of passengers currently trapped in the elevator according to the formula (9); calculating the trapped time t of the elevator in real time according to the formula (10);
wherein r is s Representing the frame rate of video, f i Representing the frame number of the current frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310435812.8A CN116534688A (en) | 2023-04-12 | 2023-04-12 | Elevator trapped person detection method based on video analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310435812.8A CN116534688A (en) | 2023-04-12 | 2023-04-12 | Elevator trapped person detection method based on video analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116534688A true CN116534688A (en) | 2023-08-04 |
Family
ID=87453432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310435812.8A Pending CN116534688A (en) | 2023-04-12 | 2023-04-12 | Elevator trapped person detection method based on video analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116534688A (en) |
-
2023
- 2023-04-12 CN CN202310435812.8A patent/CN116534688A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109522793B (en) | Method for detecting and identifying abnormal behaviors of multiple persons based on machine vision | |
| CN109359536B (en) | Passenger behavior monitoring method based on machine vision | |
| CN106981163B (en) | A kind of personnel's invasion abnormal event alarming method | |
| CN110991348B (en) | Face micro-expression detection method based on optical flow gradient amplitude characteristics | |
| CN113158752A (en) | Intelligent safety management and control system for electric power staff approach operation | |
| CN202257856U (en) | Driver fatigue-driving monitoring device | |
| CN112573316B (en) | Elevator trapping detection method based on computer vision | |
| CN109919066B (en) | Method and device for detecting density abnormality of passengers in rail transit carriage | |
| CN110790101A (en) | Elevator trapping false alarm identification method based on big data analysis | |
| CN107416630A (en) | The detection method and system of the improper closing of elevator | |
| CN110826406A (en) | Child high-altitude protection method based on deep learning model | |
| CN114926422B (en) | Method and system for detecting passenger flow of getting on and off vehicles | |
| CN114436087B (en) | Deep learning-based elevator passenger door-pulling detection method and system | |
| CN115273146A (en) | Escalator passenger posture abnormity detection method based on improved SSD model | |
| CN113044694B (en) | System and method for counting number of persons in building elevator based on deep neural network | |
| CN110633681A (en) | Bed-leaving detection method based on video | |
| CN112127718B (en) | Method for detecting running state of sliding door lock of platform door system | |
| CN116534688A (en) | Elevator trapped person detection method based on video analysis | |
| CN113031610A (en) | Automatic prison night inspection robot system and control method thereof | |
| CN110065867A (en) | The method and system of elevator Comfort Evaluation based on audio-video | |
| CN114057051B (en) | Reminding method and system for elevator calling in car | |
| CN113111701B (en) | Charging station electric ground lock state monitoring method and system based on intelligent identification | |
| CN112794174B (en) | Real-time video judgment elevator door opening and closing abnormity scheme based on big data | |
| CN111932585B (en) | Intelligent elevator room trailing behavior identification method based on big data | |
| CN109739144A (en) | Intelligent drive device for gate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |