CN117228475A - Elevator control method, device, electronic equipment and storage medium - Google Patents
Elevator control method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117228475A CN117228475A CN202311394282.3A CN202311394282A CN117228475A CN 117228475 A CN117228475 A CN 117228475A CN 202311394282 A CN202311394282 A CN 202311394282A CN 117228475 A CN117228475 A CN 117228475A
- Authority
- CN
- China
- Prior art keywords
- elevator
- elevator door
- video frame
- calibration
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000010586 diagram Methods 0.000 claims abstract description 31
- 238000012545 processing Methods 0.000 claims abstract description 30
- 230000009471 action Effects 0.000 claims abstract description 17
- 238000004458 analytical method Methods 0.000 claims abstract description 16
- 230000008569 process Effects 0.000 claims description 15
- 230000004927 fusion Effects 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 11
- 230000011218 segmentation Effects 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 description 26
- 230000001276 controlling effect Effects 0.000 description 18
- 230000000875 corresponding effect Effects 0.000 description 9
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000009434 installation Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 239000013256 coordination polymer Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 239000003973 paint Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B50/00—Energy efficient technologies in elevators, escalators and moving walkways, e.g. energy saving or recuperation technologies
Landscapes
- Indicating And Signalling Devices For Elevators (AREA)
Abstract
The application provides an elevator control method, an elevator control device, electronic equipment and a storage medium, which belong to the technical field of intelligent equipment, wherein the method comprises the following steps: acquiring a video frame shot by a camera in an elevator for an elevator door; inputting the video frame into an elevator door identification model for processing to obtain an elevator door characteristic diagram of the elevator door position in the video frame, inputting the video frame into a motion analysis model, and identifying the number of people in the elevator; inputting the elevator door characteristic diagram into an elevator door state classifier to obtain the opening and closing state of the elevator door; controlling the elevator to execute an action instruction associated with the switch status and/or the number of people.
Description
Technical Field
The application belongs to the technical field of intelligent equipment, and particularly relates to an elevator control method, an elevator control device, electronic equipment and a storage medium.
Background
With the development of computer technology, more and more electronic devices use intelligent systems, in which an elevator is required to recognize the state of an elevator door through an image acquired by a camera in order to provide various services to a user based on the operation state of the elevator.
But the scene in the elevator car is complex and various, and is particularly important for the visual classifier, and when the field angle, the direction and the installation height of the car camera are greatly different from the training data of the visual classifier, the classification precision of the whole image can be negatively affected. Therefore, the area where the elevator is located needs to be set, and the accuracy of the elevator door state classifier can be further improved. However, when the system is deployed and implemented, the traditional method needs to rely on an installer to access a camera through an interface, after a preview video stream is obtained, the area where the landing door is located is set, the process is complicated, and the set error can bring adverse effects to a landing door identification algorithm; in addition, in the use process of the camera, the elevator door position which is manually set can be displaced due to the fact that the elevator camera is touched to displace under the conditions of daily maintenance, large furniture in-out and the like due to some unexpected factors, and finally, the elevator door state identifier and other elevator functions are disabled.
Disclosure of Invention
The application provides an elevator control method, an elevator control device, electronic equipment and a storage medium.
Some embodiments of the application provide an elevator control method, the method comprising:
acquiring a video frame shot by a camera in an elevator for an elevator door;
inputting the video frame into an elevator door identification model for processing to obtain an elevator door characteristic diagram of the elevator door position in the video frame, inputting the video frame into a motion analysis model, and identifying the number of people in the elevator;
inputting the elevator door characteristic diagram into an elevator door state classifier to obtain the opening and closing state of the elevator door;
controlling the elevator to execute an action instruction associated with the switch status and/or the number of people.
Optionally, the elevator door recognition model is obtained through training of the following steps:
acquiring a sample video frame obtained by shooting an elevator door by a camera in an elevator;
constructing a semantic segmentation network to obtain an elevator door recognition model, wherein the elevator door recognition model at least comprises: an activation layer, a lightweight network and a fusion layer;
acquiring spatial position information of the elevator door in the sample video frame by using the activation layer;
encoding semantic information in the sample video frames using the lightweight network;
the spatial position information and the coded sample video frame are fused by utilizing the fusion layer, and a calibration feature map of an area where an elevator door is located in the sample video frame is obtained;
and when the trained elevator door identification model meets the convergence requirement, taking the position of the calibration feature map as the elevator door calibration position.
Optionally, when the trained elevator door recognition model meets the convergence requirement, the step of taking the position of the calibration feature map as the elevator door calibration position includes:
when the number of the obtained calibration feature images reaches a frame number threshold, integrating a plurality of the calibration feature images and then carrying out average calculation to obtain a target calibration feature image;
and taking the position corresponding to the target calibration characteristic diagram as the calibration position of the elevator door.
Optionally, the step of taking the position corresponding to the target calibration feature map as the calibration position of the elevator door includes:
performing image binarization processing on the target calibration feature map by using a preset pixel threshold value;
and taking the position of the minimum circumscribed matrix of the maximum connected domain in the target calibration characteristic diagram after binarization processing as the calibration position of the elevator door.
Optionally, the acquiring the sample video frame obtained by shooting the elevator door by the camera in the elevator includes:
performing motion detection on video frames shot by a camera through a motion analysis model;
and when no moving object exists in the video frames, sampling video frames from the video frames.
Optionally, before the step of acquiring the video frame captured by the camera for the elevator door in the elevator, the method further comprises:
when the current scene meets scene conditions, carrying out position calibration on the elevator door by using the elevator door identification model, wherein the scene conditions at least comprise:
starting a camera in the elevator;
the current time point is a night time point;
the time period from the last execution of the elevator door position calibration process reaches a preset time interval.
Optionally, the step of controlling the elevator to execute an action instruction associated with the switch status and/or the number of people comprises:
when the time length of the switch state being the closing state exceeds a time length threshold value and the number of people is not 0, controlling the elevator to send out alarm information for prompting that the elevator is trapped; or,
the step of controlling the elevator to execute an action instruction associated with the switch status and/or the number of people comprises:
when the switch state is switched from the on state to the off state and the number of people is not 0, controlling projection equipment in the elevator to play a push video; or,
the step of controlling the elevator to execute an action instruction associated with the switch status and/or the number of people comprises:
and when the switch state is switched from the closed state to the open state, controlling the projection equipment in the elevator to be closed.
Some embodiments of the application provide an elevator control apparatus, the apparatus comprising:
the acquisition module is used for acquiring video frames shot by the camera in the elevator for the elevator door;
the identification module is used for inputting the video frame into an elevator door identification model for processing to obtain an elevator door characteristic diagram of the position of an elevator door in the video frame, inputting the video frame into a motion analysis model and identifying the number of people in the elevator;
inputting the elevator door characteristic diagram into an elevator door state classifier to obtain the opening and closing state of the elevator door;
and the control module is used for controlling the elevator to execute action instructions related to the switch state and/or the personnel number.
Optionally, the apparatus further comprises: training module for:
acquiring a sample video frame obtained by shooting an elevator door by a camera in an elevator;
constructing a semantic segmentation network to obtain an elevator door recognition model, wherein the elevator door recognition model at least comprises: an activation layer, a lightweight network and a fusion layer;
acquiring spatial position information of the elevator door in the sample video frame by using the activation layer;
encoding semantic information in the sample video frames using the lightweight network;
the spatial position information and the coded sample video frame are fused by utilizing the fusion layer, and a calibration feature map of an area where an elevator door is located in the sample video frame is obtained;
and when the trained elevator door identification model meets the convergence requirement, taking the position of the calibration feature map as the elevator door calibration position.
Optionally, the training module is further configured to:
when the number of the obtained calibration feature images reaches a frame number threshold, integrating a plurality of the calibration feature images and then carrying out average calculation to obtain a target calibration feature image;
and taking the position corresponding to the target calibration characteristic diagram as the calibration position of the elevator door.
Optionally, the training module is further configured to:
performing image binarization processing on the target calibration feature map by using a preset pixel threshold value;
and taking the position of the minimum circumscribed matrix of the maximum connected domain in the target calibration characteristic diagram after binarization processing as the calibration position of the elevator door.
Optionally, the training module is further configured to:
performing motion detection on video frames shot by a camera through a motion analysis model;
and when no moving object exists in the video frames, sampling video frames from the video frames.
Optionally, the training module is further configured to:
when the current scene meets scene conditions, carrying out position calibration on the elevator door by using the elevator door identification model, wherein the scene conditions at least comprise:
starting a camera in the elevator;
the current time point is a night time point;
the time period from the last execution of the elevator door position calibration process reaches a preset time interval.
Optionally, the control module is further configured to:
when the time length of the switch state being the closing state exceeds a time length threshold value and the number of people is not 0, controlling the elevator to send out alarm information for prompting that the elevator is trapped; or,
when the switch state is switched from the on state to the off state and the number of people is not 0, controlling projection equipment in the elevator to play a push video; or,
and when the switch state is switched from the closed state to the open state, controlling the projection equipment in the elevator to be closed.
Some embodiments of the application provide a computing processing device comprising:
a memory having computer readable code stored therein;
one or more processors, the computer-readable code, when executed by the one or more processors, performs the elevator control method as described above.
Some embodiments of the application provide a computer program comprising computer readable code which, when run on a computing processing device, causes the computing processing device to perform an elevator control method as described above.
Some embodiments of the application provide a non-transitory computer readable medium in which an elevator control method as described above is stored.
According to the elevator control method, the elevator control device, the electronic equipment and the storage medium, the elevator door identification model is utilized to identify the video frame shot by the camera aiming at the elevator door to dynamically calibrate the position of the elevator door, so that the elevator door characteristic diagram is intercepted from the video frame, the elevator door state identification model is used for identifying the opening and closing state of the elevator door, the elevator is controlled to execute corresponding action instructions based on the opening and closing state and the number of people in the elevator, the sensor in the elevator is not needed, the elevator door position can be dynamically calibrated only by the camera, the elevator door area can be set without manually calibrating the position of the elevator door, the elevator door state classifier is assisted, the position of the elevator door can be more accurately obtained, and the accuracy of the state identification system and the usability of deployment and implementation are improved.
The foregoing description is only an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present application more readily apparent.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 schematically illustrates a flow chart of an elevator control method provided by some embodiments of the present application;
fig. 2 schematically illustrates a flowchart of a training method for an elevator door recognition model according to some embodiments of the present application;
fig. 3 schematically illustrates one of the scene diagrams of an elevator control method provided by some embodiments of the present application;
fig. 4 schematically illustrates a second exemplary scenario of an elevator control method provided by some embodiments of the present application;
fig. 5 schematically illustrates a structural diagram of an elevator control apparatus provided by some embodiments of the present application;
FIG. 6 schematically illustrates a block diagram of a computing processing device for performing methods according to some embodiments of the application;
fig. 7 schematically illustrates a memory unit for holding or carrying program code for implementing methods according to some embodiments of the application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Before describing in detail the aspects of embodiments of the present application, the terminology used in connection with the present application will be described:
convolutional neural networks (Convolutional Neural Networks, CNN), which are a type of feed forward neural networks (feedforwardshaving a deep structure) that involve convolutional calculations, are one of the representative algorithms of deep learning.
Semantic segmentation: semantic segmentation is a fundamental task in computer vision, where we need to divide visual input into different semantic interpretable categories, "semantic interpretability," i.e., classification categories, are meaningful in the real world. For example, we may need to distinguish all pixels belonging to an automobile in an image and paint those pixels blue.
BiSeNet: a bi-directional segmentation network for real-time semantic segmentation.
Fig. 1 schematically shows a flow chart of an elevator control method provided by the application, the method comprising:
step 101, obtaining a video frame shot by a camera in the elevator for an elevator door.
It should be noted that, the camera is installed inside the elevator, the installation angle of the camera can be determined according to the elevator model, and the camera can be applicable to the embodiment of the application as long as people inside the elevator and the elevator door can be shot.
In the embodiment of the application, the system can continuously detect the opening and closing states of the elevator door and the personnel conditions in the elevator by detecting the video frames shot by the elevator door according to the target time interval through the camera. The system can detect the opening and closing state of the elevator door automatically when a person enters the elevator, and can detect the opening and closing state of the elevator door according to a preset period.
And 102, inputting the video frame into an elevator door identification model for processing to obtain an elevator door characteristic diagram of the elevator door position in the video frame, inputting the video frame into a motion analysis model, and identifying the number of people in the elevator.
It should be noted that the elevator door identification model is a model for determining the position of the elevator door from the video frame, and may be calibrated by using the coordinate position and the width height. Illustratively, the elevator door recognition model is used for determining the position of the elevator door in the video frame by adopting a mode of combining angular point detection with direct detection, and the mode has the advantages of reduced required calculation force and storage space and suitability for a low-performance platform with low calculation force and storage space; the elevator door recognition model can also adopt a machine learning model based on an image detection technology of a convolutional neural network, and the convolutional neural network adopts a Yolo algorithm, an SSD algorithm and the like.
In the embodiment of the application, the system inputs the video frame shot for the elevator door into the elevator door identification model, the elevator door identification model identifies and calibrates the image area of the elevator door in the video frame and outputs the elevator door characteristic diagram of the image area, and as the elevator door identification model is adopted to calibrate the position of the elevator door, even if the shooting angle of the camera is changed due to human error or accidental collision, the position of the elevator door can be dynamically corrected through the elevator door identification model, thereby avoiding the occurrence of the situation of error calibration of the elevator door position caused by the change of the shooting angle of the camera. And also need not to revise the shooting angle of camera through the manual work, only need the camera of installation aim at the lift-cabin door can to can carry out dynamic calibration to the lift-cabin door position through deploying lift-cabin door identification model simply, need not the manual work and marks the lift-cabin door position, alleviateed the manpower input under the lift-cabin door discernment scene.
Furthermore, a motion analysis model can be deployed in the system, and the motion analysis model is used for detecting a moving object in the elevator, so that the number of people in the elevator can be detected through human body characteristics, and a subsequent system can send a control instruction to the elevator according to the opening and closing state of the elevator door and the conditions of the people in the elevator.
And step 103, inputting the elevator door characteristic diagram into an elevator door state classifier to obtain the opening and closing states of the elevator door.
It should be noted that the elevator door state classifier may be a network model using two-class or multi-class image classification.
In the embodiment of the application, the system inputs the elevator door characteristic diagram intercepted by the elevator door recognition model from the video frame to the elevator door state classifier, and the elevator door state classifier classifies the elevator door characteristic diagram according to the preset switch state. The switch state may be two states of an on state and an off state, or may be three states of a full on state, a half on state and an off state, which is of course only described here as an example, and the setting mode of the switch state may be set according to actual requirements, which is not limited herein.
Step 104, controlling the elevator to execute an action instruction associated with the switch state and/or the number of people.
In the embodiment of the application, after the system acquires the current opening and closing state of the elevator door and the number of people in the elevator, the system inquires the action instructions related to the opening and closing state and the number of people according to the preset elevator action rules, controls the elevator to execute corresponding actions through the action instructions, for example, after the elevator is opened and people enter, can play advertisement videos through a display in the elevator, or can control the elevator to send out alarm information that people are trapped when the elevator door is in the closed state for a long time, or can control the elevator to automatically express voice information and the like for welcome the elevator after the elevator door is opened, and the association relation between different opening and closing states, the number of people and the action instructions can be automatically set based on the actual demand application scene and the elevator configuration.
According to the embodiment of the application, the position of the elevator door is dynamically calibrated by utilizing the elevator door identification model to identify the video frame shot by the camera aiming at the elevator door, so that the elevator door characteristic diagram is intercepted from the video frame, the elevator door state identification model is used for identifying the opening and closing state of the elevator door, the elevator is controlled to execute corresponding action instructions based on the opening and closing state and the number of people in the elevator, the elevator door position can be dynamically calibrated only by the camera without depending on a sensor in the elevator, the elevator door area can be set without manually calibrating the position of the elevator door, the elevator door state classifier is assisted, the position of the elevator door can be more accurately obtained, the accuracy of a state identification system is improved, and the usability of deployment and implementation is improved.
Optionally, referring to fig. 2, the elevator door recognition model is trained by:
step 201, obtaining a sample video frame obtained by shooting an elevator door by a camera in an elevator.
In the embodiment of the application, the sample video frame can be obtained by extracting the video frame from the video shot by the camera for the elevator door in an offline state, and the training process can also be completed in the offline state, and the elevator door state recognition system is reset after the model training is completed.
Step 202, constructing a semantic segmentation network to obtain an elevator door recognition model, wherein the elevator door recognition model at least comprises: an activation layer, a lightweight network and a fusion layer.
And 203, acquiring the spatial position information of the elevator door in the sample video frame by using the activation layer.
Step 204, encoding semantic information in the sample video frames using the lightweight network.
And 205, fusing the spatial position information and the encoded sample video frame by using the fusion layer to obtain a calibration feature map of the area where the elevator door is located in the sample video frame.
And 206, when the trained elevator door identification model meets the convergence requirement, taking the position of the calibration feature map as the elevator door calibration position.
In the embodiment of the application, a BiSeNet-based structure can be adopted to build a semantic segmentation network, wherein the BiSeNet framework mainly uses a space branch SP, and a packet can comprise a repeated three-layer convolution +Bn +Relu activation layer with 2 compensation for outputting space position information; the BiSeNet architecture also uses a content branch CP, and is characterized in that a lightweight network is used for encoding semantic information, a larger receptive field is obtained through global averaging and pooling, and finally, the global pooled result is fused with a feature map after the lightweight network encoding in an up-sampling mode. Wherein the lightweight network can adopt Xreception as a backbone network; the BiSeNet architecture also contains a fusion layer that combines the features of the spatial branch SP and the content branch CP using an FFM architecture, followed by upsampling to recover resolution.
The output result of the trained elevator door recognition model is a feature map of the area where the elevator door is located in the video frame, the position of each coordinate point in the feature map is a floating point value used for identifying the confidence that the point belongs to an elevator door pixel, the coordinate point higher than the confidence threshold can be judged to be the area where the elevator door is located, and the coordinate point lower than the confidence threshold can be judged to be a background area beyond the elevator door position.
Furthermore, after the elevator door recognition model is subjected to offline training, the elevator door recognition model can be used by being online to an elevator door state recognition system, and particularly, when the elevator door state recognition system works, sample video frames shot for the elevator door can be intercepted offline to update and train the elevator door recognition model, after the update of the elevator door recognition model is completed, the updated elevator door recognition model can be utilized to replace an old elevator door recognition model, so that the influence of the update and training process of the elevator door recognition model on the normal use process of the elevator door recognition system is reduced.
Optionally, the step 206 includes:
step 2061, when the number of the obtained calibration feature images reaches a frame number threshold, integrating a plurality of the calibration feature images and then calculating an average value to obtain a target calibration feature image;
and step 2062, taking the position corresponding to the target calibration characteristic diagram as the elevator door calibration position.
In the embodiment of the application, the convergence condition for judging the end of training for the elevator door recognition model can be judged by presetting a frame number threshold value, and the model convergence can be judged when the number of the calibration feature images obtained by training reaches the integer threshold value, so that the training is ended.
Further, after the elevator door recognition model training is finished, part or all of the obtained calibration feature images can be integrated and then averaged, and the final target calibration feature image can be obtained.
According to the embodiment of the application, the interference on the recognition result caused by shielding of the elevator door in the video frame is reduced by adopting a multi-frame fusion mode after the training of the elevator door recognition model is finished, and the robustness of the elevator door recognition model is improved.
Optionally, the step 2062 includes:
step 20621, performing image binarization processing on the target calibration feature map by using a preset pixel threshold;
and 20622, taking the position of the minimum circumscribed matrix of the maximum connected domain in the target calibration feature map after the binarization processing as the calibration position of the elevator door.
In the embodiment of the application, the preset pixel value threshold is a pixel value for dividing the pixel value difference between the position of the elevator door in the feature map and the position of the background map, and the target calibration feature can be subjected to image binarization processing by using the preset pixel value threshold, wherein the area of the elevator door is marked by 1, and the area of the background is marked by 0. Then, the cavities and noise in the target calibration feature map can be removed by adopting morphological operation, and then, the minimum missed matrix of the maximum connected domain in the target calibration feature map is calculated to be used as the area where the elevator door is positioned, so that the calibration process of the elevator door area is completed. The area where the elevator door is located can be expanded according to a certain proportion on the basis of the minimum external matrix, so that the identified elevator door calibration position can be ensured to contain the complete elevator door as much as possible.
Optionally, the step 101 includes:
step 1011, performing motion detection on a video frame shot by a camera through a motion analysis model;
step 1012, extracting a sample video frame from the video frame when there is no moving object in the video frame.
In some embodiments of the present application, referring to fig. 3 and 4, considering that a moving object in a video frame may block an image of an area where an elevator door is located, the recognition result of an elevator door recognition model is affected, so when the video frame is extracted from a video shot by a camera, the video is detected by a motion analysis model, and when no object is detected to move, the system can trigger a step of extracting the video frame and inputting the video frame into the elevator door recognition model.
According to the embodiment of the application, the video is detected through the motion analysis model before the video frame is extracted, and the video frame is extracted for the elevator door recognition model to be used after the non-running object is detected, so that the influence of the moving object on the elevator door position detection accuracy in the video frame is reduced, and the elevator door recognition accuracy is improved.
Optionally, before the step 101, the method further includes: when the current scene meets scene conditions, carrying out position calibration on the elevator door by using the elevator door identification model, wherein the scene conditions at least comprise:
starting a camera in the elevator;
the current time point is a night time point;
the time period from the last execution of the elevator door position calibration process reaches a preset time interval.
In some embodiments of the present application, in order to enable the elevator door recognition model to dynamically calibrate the elevator door position, some scene conditions may be set, and when the conditions are satisfied, the system automatically triggers the training process of the elevator door recognition model, and because the position of the elevator door is recognized in the training process, the calibration of the area where the elevator door is located can be completed through training.
The automatic calibration process of the area where the elevator door is located can be carried out under the scene condition that the camera needs to be started and initialized each time, for example, under the conditions of initial installation, reinstallation after maintenance, power failure restarting and the like, the elevator door identification model can be automatically trained and updated, so that the change process of shooting angles of the camera during restarting is applicable, and the area where the elevator door is located is recalibrated.
The other condition is that because the elevator in the daytime has more people in and out, the elevator door recognition model can be automatically trained and updated when the people in and out at night are fewer, so that the area where the elevator door is located is recalibrated, the negative influence of the personnel shielding the elevator door on the elevator door recognition model training effect can be reduced, and the recognition accuracy of the elevator door recognition model is improved.
Moreover, the system can update and train the elevator door identification model automatically according to the preset time interval by setting the preset time interval, and recalibrate the area of the elevator door, so that the identification effect of the elevator door identification model can be kept consistent with the shooting angle of the camera.
According to the embodiment of the application, the elevator door identification model is automatically updated and trained by setting the various scene condition control systems, so that the accuracy of identification of the elevator door identification model is ensured.
Optionally, the step 104 includes: when the time length of the switch state being the closing state exceeds a time length threshold value and the number of people is not 0, controlling the elevator to send out alarm information for prompting that the elevator is trapped; or when the switch state is switched from the on state to the off state and the number of people is not 0, controlling projection equipment in the elevator to play a push video; or when the switch state is switched from the off state to the on state, the projection equipment in the elevator is controlled to be turned off.
In the embodiment of the application, the situation that the elevator is closed for a long time under the condition that people exist in the elevator is considered, and the elevator is trapped due to the elevator fault, so that the system automatically controls the elevator to send out alarm information that people exist when the opening and closing state of the elevator door is in the closed state and the number of people is not 0, namely, the situation that people exist, and prompts the outside personnel to timely implement rescue measures.
Further, under the condition that information such as advertisements, public welfare shortages, event notices and the like need to be played through a display in the elevator, no one in the elevator can cause resource waste, and a user can be frightened to the user by watching screen flickering and sounding when the elevator is opened, so that the system can control the display to play the push video after the elevator door is opened and someone enters, the playing power of the display is saved, and the comfort of watching the push video by the user is improved.
Further, considering that the resource waste is caused by continuously displaying the video when no person is in the elevator, the system can display that the video is started to be played, if the elevator door is detected to be switched from the closed state to the open state, the user in the elevator needs to go out of the elevator, and the display can be controlled to stop playing the video to save the playing electricity for displaying the video.
Fig. 5 schematically shows a schematic structural view of an elevator control device 30 provided by the present application, the device comprising:
the acquisition module 301 is configured to acquire a video frame captured by a camera in an elevator for an elevator door;
the identification module 302 is configured to input the video frame to an elevator door identification model for processing, obtain an elevator door feature map of a position of an elevator door in the video frame, input the video frame to a motion analysis model, and identify the number of people in the elevator;
inputting the elevator door characteristic diagram into an elevator door state classifier to obtain the opening and closing state of the elevator door;
a control module 303 for controlling the elevator to execute an action instruction associated with the switch status and/or the number of people.
Optionally, the apparatus further comprises: training module for:
acquiring a sample video frame obtained by shooting an elevator door by a camera in an elevator;
constructing a semantic segmentation network to obtain an elevator door recognition model, wherein the elevator door recognition model at least comprises: an activation layer, a lightweight network and a fusion layer;
acquiring spatial position information of the elevator door in the sample video frame by using the activation layer;
encoding semantic information in the sample video frames using the lightweight network;
the spatial position information and the coded sample video frame are fused by utilizing the fusion layer, and a calibration feature map of an area where an elevator door is located in the sample video frame is obtained;
and when the trained elevator door identification model meets the convergence requirement, taking the position of the calibration feature map as the elevator door calibration position.
Optionally, the training module is further configured to:
when the number of the obtained calibration feature images reaches a frame number threshold, integrating a plurality of the calibration feature images and then carrying out average calculation to obtain a target calibration feature image;
and taking the position corresponding to the target calibration characteristic diagram as the calibration position of the elevator door.
Optionally, the training module is further configured to:
performing image binarization processing on the target calibration feature map by using a preset pixel threshold value;
and taking the position of the minimum circumscribed matrix of the maximum connected domain in the target calibration characteristic diagram after binarization processing as the calibration position of the elevator door.
Optionally, the training module is further configured to:
performing motion detection on video frames shot by a camera through a motion analysis model;
and when no moving object exists in the video frames, sampling video frames from the video frames.
Optionally, the training module is further configured to:
when the current scene meets scene conditions, carrying out position calibration on the elevator door by using the elevator door identification model, wherein the scene conditions at least comprise:
starting a camera in the elevator;
the current time point is a night time point;
the time period from the last execution of the elevator door position calibration process reaches a preset time interval.
Optionally, the control module 303 is further configured to:
when the time length of the switch state being the closing state exceeds a time length threshold value and the number of people is not 0, controlling the elevator to send out alarm information for prompting that the elevator is trapped; or,
when the switch state is switched from the on state to the off state and the number of people is not 0, controlling projection equipment in the elevator to play a push video; or,
and when the switch state is switched from the closed state to the open state, controlling the projection equipment in the elevator to be closed.
According to the embodiment of the application, the position of the elevator door is dynamically calibrated by utilizing the elevator door identification model to identify the video frame shot by the camera aiming at the elevator door, so that the elevator door characteristic diagram is intercepted from the video frame, the elevator door state identification model is used for identifying the opening and closing state of the elevator door, the elevator is controlled to execute corresponding action instructions based on the opening and closing state and the number of people in the elevator, the elevator door position can be dynamically calibrated only by the camera without depending on a sensor in the elevator, the elevator door area can be set without manually calibrating the position of the elevator door, the elevator door state classifier is assisted, the position of the elevator door can be more accurately obtained, the accuracy of a state identification system is improved, and the usability of deployment and implementation is improved.
Various component embodiments of the application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in a computing processing device according to embodiments of the present application may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present application can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present application may be stored on a non-transitory computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
For example, FIG. 6 illustrates a computing processing device in which methods according to the present application may be implemented. The computing processing device conventionally includes a processor 410 and a computer program product in the form of a memory 420 or a non-transitory computer readable medium. The memory 420 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Memory 420 has storage space 430 for program code 431 for performing any of the method steps described above. For example, the memory space 430 for the program code may include individual program code 431 for implementing the various steps in the above method, respectively. The program code can be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. Such a computer program product is typically a portable or fixed storage unit as described with reference to fig. 7. The storage unit may have memory segments, memory spaces, etc. arranged similarly to the memory 420 in the computing processing device of fig. 6. The program code may be compressed, for example, in a suitable form. Typically, the storage unit comprises computer readable code 431', i.e. code that can be read by a processor, such as 410, for example, which when run by a computing processing device causes the computing processing device to perform the steps in the method described above.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
Reference herein to "one embodiment," "an embodiment," or "one or more embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Furthermore, it is noted that the word examples "in one embodiment" herein do not necessarily all refer to the same embodiment.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. An elevator control method, characterized in that the method comprises:
acquiring a video frame shot by a camera in an elevator for an elevator door;
inputting the video frame into an elevator door identification model for processing to obtain an elevator door characteristic diagram of the elevator door position in the video frame, inputting the video frame into a motion analysis model, and identifying the number of people in the elevator;
inputting the elevator door characteristic diagram into an elevator door state classifier to obtain the opening and closing state of the elevator door;
controlling the elevator to execute an action instruction associated with the switch status and/or the number of people.
2. The method of claim 1, wherein the elevator door identification model is trained by:
acquiring a sample video frame obtained by shooting an elevator door by a camera in an elevator;
constructing a semantic segmentation network to obtain an elevator door recognition model, wherein the elevator door recognition model at least comprises: an activation layer, a lightweight network and a fusion layer;
acquiring spatial position information of the elevator door in the sample video frame by using the activation layer;
encoding semantic information in the sample video frames using the lightweight network;
the spatial position information and the coded sample video frame are fused by utilizing the fusion layer, and a calibration feature map of an area where an elevator door is located in the sample video frame is obtained;
and when the trained elevator door identification model meets the convergence requirement, taking the position of the calibration feature map as the elevator door calibration position.
3. The method according to claim 2, wherein the step of taking the position of the calibration feature map as the calibration position of the elevator door when the trained elevator door identification model meets the convergence requirement comprises:
when the number of the obtained calibration feature images reaches a frame number threshold, integrating a plurality of the calibration feature images and then carrying out average calculation to obtain a target calibration feature image;
and taking the position corresponding to the target calibration characteristic diagram as the calibration position of the elevator door.
4. A method according to claim 3, wherein the step of taking the position corresponding to the target calibration feature map as the elevator door calibration position comprises:
performing image binarization processing on the target calibration feature map by using a preset pixel threshold value;
and taking the position of the minimum circumscribed matrix of the maximum connected domain in the target calibration characteristic diagram after binarization processing as the calibration position of the elevator door.
5. The method according to claim 2, wherein the acquiring the sample video frames taken by the camera of the elevator for the elevator door comprises:
performing motion detection on video frames shot by a camera through a motion analysis model;
and when no moving object exists in the video frames, sampling video frames from the video frames.
6. The method of claim 1, wherein prior to the step of capturing video frames captured by a camera of an elevator for an elevator door, the method further comprises:
when the current scene meets scene conditions, carrying out position calibration on the elevator door by using the elevator door identification model, wherein the scene conditions at least comprise:
starting a camera in the elevator;
the current time point is a night time point;
the time period from the last execution of the elevator door position calibration process reaches a preset time interval.
7. The method according to claim 1, characterized in that the step of controlling the elevator to execute an action instruction associated with the switch status and/or the number of people comprises:
when the time length of the switch state being the closing state exceeds a time length threshold value and the number of people is not 0, controlling the elevator to send out alarm information for prompting that the elevator is trapped; or,
the step of controlling the elevator to execute an action instruction associated with the switch status and/or the number of people comprises:
when the switch state is switched from the on state to the off state and the number of people is not 0, controlling projection equipment in the elevator to play a push video; or,
the step of controlling the elevator to execute an action instruction associated with the switch status and/or the number of people comprises:
and when the switch state is switched from the closed state to the open state, controlling the projection equipment in the elevator to be closed.
8. An elevator control apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring video frames shot by the camera in the elevator for the elevator door;
the identification module is used for inputting the video frame into an elevator door identification model for processing to obtain an elevator door characteristic diagram of the position of an elevator door in the video frame, inputting the video frame into a motion analysis model and identifying the number of people in the elevator;
inputting the elevator door characteristic diagram into an elevator door state classifier to obtain the opening and closing state of the elevator door;
and the control module is used for controlling the elevator to execute action instructions related to the switch state and/or the personnel number.
9. A computing processing device, comprising:
a memory having computer readable code stored therein;
one or more processors, the computing processing device performing the elevator control method of any of claims 1-7 when the computer readable code is executed by the one or more processors.
10. A non-transitory computer readable medium, characterized in that a computer program of the elevator control method according to any one of claims 1-7 is stored therein.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311394282.3A CN117228475A (en) | 2023-10-25 | 2023-10-25 | Elevator control method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311394282.3A CN117228475A (en) | 2023-10-25 | 2023-10-25 | Elevator control method, device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117228475A true CN117228475A (en) | 2023-12-15 |
Family
ID=89082645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311394282.3A Pending CN117228475A (en) | 2023-10-25 | 2023-10-25 | Elevator control method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117228475A (en) |
-
2023
- 2023-10-25 CN CN202311394282.3A patent/CN117228475A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10255517B2 (en) | Information processing apparatus and information processing method | |
| CN107679471B (en) | Indoor personnel air post detection method based on video monitoring platform | |
| CN110032966B (en) | Human body proximity detection method for intelligent service, intelligent service method and device | |
| CN109740590B (en) | ROI accurate extraction method and system based on target tracking assistance | |
| CN111325769B (en) | Target object detection method and device | |
| US20100290710A1 (en) | System and method for motion detection in a surveillance video | |
| CN109409315B (en) | Method and system for detecting remnants in panel area of ATM (automatic Teller machine) | |
| CN109657581A (en) | Urban track traffic gate passing control method based on binocular camera behavioral value | |
| KR20130104286A (en) | Method for processing image | |
| CN111523559B (en) | Abnormal behavior detection method based on multi-feature fusion | |
| JP7185419B2 (en) | Method and device for classifying objects for vehicles | |
| KR102391853B1 (en) | System and Method for Processing Image Informaion | |
| EP2447912B1 (en) | Method and device for the detection of change in illumination for vision systems | |
| CN112487877A (en) | Monitoring method, system, device and medium for standard operation of kitchen waste | |
| Jeong et al. | Probabilistic method to determine human subjects for low-resolution thermal imaging sensor | |
| CN110969173B (en) | Target classification method and device | |
| US20220366570A1 (en) | Object tracking device and object tracking method | |
| US10735660B2 (en) | Method and device for object identification | |
| CN117228475A (en) | Elevator control method, device, electronic equipment and storage medium | |
| CN109034171B (en) | Method and device for detecting unlicensed vehicles in video stream | |
| CN116357198A (en) | Vehicle window control method and system and vehicle | |
| CN113496501B (en) | Method and system for detecting invader in dynamic scene based on video prediction | |
| CN114359839A (en) | Method and system for identifying entrance of electric vehicle into elevator | |
| JP5241687B2 (en) | Object detection apparatus and object detection program | |
| JP6944201B2 (en) | Electronic circuits, hardware components, edge processing systems, edge computing systems, identification methods, identification programs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |