CN116071682A - Elevator door opening and closing detection method and system based on neural network - Google Patents
Elevator door opening and closing detection method and system based on neural network Download PDFInfo
- Publication number
- CN116071682A CN116071682A CN202310093099.3A CN202310093099A CN116071682A CN 116071682 A CN116071682 A CN 116071682A CN 202310093099 A CN202310093099 A CN 202310093099A CN 116071682 A CN116071682 A CN 116071682A
- Authority
- CN
- China
- Prior art keywords
- elevator
- door opening
- elevator door
- closing detection
- door
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Indicating And Signalling Devices For Elevators (AREA)
Abstract
The invention discloses an elevator door opening and closing detection method based on a neural network, which comprises the following steps: step one, acquiring images of elevator door areas, wherein the opening degree of an elevator door of each image is used as a label of the image; establishing a door opening and closing detection network model based on a neural network; step three, obtaining a video frame detection result, and fusing multiple video frame detection resultsAnd (5) combining. The invention also discloses an elevator door opening and closing detection system based on the neural network. The invention uses the opening degree of the elevator door as an image tag, so as to avoid the ambiguity problem caused by classifying the elevator switch state; elevator door opening degree P integrating multiple video frames for accumulation k Calculating to obtain the final opening degree P ave The elevator door switching state is not dependent on a moment, the problem that the detection result is low in following degree due to the fact that the elevator is abnormally blocked to switch and then to recover to the former switching state is avoided, the required calculation force is low, and the elevator door switching state can be rapidly deployed and operated on an embedded platform.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an elevator door opening and closing detection method and system based on a neural network.
Background
The detection method for the elevator door opening and closing in the prior art mainly comprises the following steps: the first is to directly perform a two-classification of the image of the whole elevator door area, which has the disadvantages that: 1. only the two states of fully open and close of the elevator door are included, and the method ignores the states that the elevator door also includes the states of partial opening, so that the classification has ambiguity for the classification algorithm. 2. In addition, due to the difference of the acquisition equipment and the processing algorithm, the proportion of the elevator door area in the image is different, and if no further processing is performed, the generalization learned by the neural network is poor. 3. The method is high in labor cost for batch detection, and passengers in the elevator can shade the markers in the riding process, so that the detection result is influenced. The second is to classify video sequences based on a transducer or RNN network, but intelligent algorithms such as a neural network are complex, have large calculation amount, require large storage resources and calculation resources, and cannot be provided for a common embedded platform.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides the elevator door opening and closing detection method and system based on the neural network, which have simple structure and reasonable design, and the problem of ambiguity caused by classifying the elevator door opening and closing states is avoided by using the elevator door opening degree as an image tag; elevator door opening degree P accumulated by fusing a plurality of video frames k Calculating to obtain the final opening degree P ave So that the elevatorThe switch state of the door is not dependent on a moment, so that the problem that the detection result is low in following degree due to the fact that the elevator is abnormally blocked by the switch and then is recovered before the elevator is recovered is avoided, the overall required calculation force is low, and the elevator can be rapidly deployed and operated on an embedded platform.
In order to solve the technical problems, the invention adopts the following technical scheme: the elevator door opening and closing detection method based on the neural network is characterized by comprising the following steps of: the method comprises the following steps:
step one, acquiring images of elevator door areas, wherein the opening degree of an elevator door of each image is used as a label of the image, the images form an image set, and the images in the image set are divided into a training set and a verification set;
step two, establishing a switch door detection network model based on a neural network: establishing a door opening and closing detection network model based on a neural network, performing back propagation training on the door opening and closing detection network model based on a training set, and adjusting network parameters of the door opening and closing detection network model according to the test precision of the door opening and closing detection network model on a verification set to obtain a trained door opening and closing detection network model;
step three, detection:
step 301, obtaining a video frame detection result: collecting real-time images of elevator door areas of each video frame in elevator monitoring videos, inputting the real-time images into a door opening and closing detection network model, and outputting elevator door opening degrees P corresponding to the real-time images by the door opening and closing detection network model k K represents the number of images;
step 302, fusing detection results of multiple video frames: identifying the time sequence characteristics of the video frame images, and opening the elevator door P corresponding to the video frame images k In a first-in first-out queue of fixed length in time series, if the elevator door entering the queue is open to a degree P k If the number of the elevator doors is greater than the length of the queue, the opening degree P of the elevator door which enters the queue first is deleted k According to the opening degree P of all elevator doors in the first-in first-out queue k Calculating the final opening degree P ave When P ave >P open When the elevator door is in an open state, judging that the elevator door is in an open state; when P ave <P close When judging that the elevator door is in a closed state, wherein P open Indicating a threshold value of door opening, P close Indicating a door closing threshold.
The elevator door opening and closing detection method based on the neural network is characterized by comprising the following steps of: final opening degree P ave The opening degree P of the elevator door corresponding to all video frame images in the queue k Is a mean value of (c).
The elevator door opening and closing detection method based on the neural network is characterized by comprising the following steps of: setting array weights of a first-in first-out queue, and according to the array weights and the elevator door opening degree P corresponding to the video frame images k Is used for calculating the final opening degree P by weighting the positions of the array ave Wherein the array weight closer to the head of the queue is smaller, and the array weight closer to the tail of the queue is larger.
The elevator door opening and closing detection method based on the neural network is characterized by comprising the following steps of: the first step also comprises preprocessing the image, and the specific process is as follows:
step 101, obtaining corner coordinates (x) of the elevator door in the image si ,y si ) Setting the expected coordinates (x di ,y di ) Calculating a transformation matrix WarpM according to the angular point coordinates and the expected coordinates, wherein i represents the number of points of an abscissa or the number of points of an ordinate;
step 102, affine transformation is carried out on the images of the elevator door areas of each video frame according to the transformation matrix warp, so that preprocessed images are obtained, and the preprocessed images form an image set.
The elevator door opening and closing detection method based on the neural network is characterized by comprising the following steps of: the method for generating the label of the image in the first step comprises the following steps: the computer calculates target=width according to the formula open Width calculation of elevator door opening degree, where width=max (x di )-min(x di )，width open Equal to the distance of the elevator left door edge line from the elevator right door edge line in the coordinate system, the elevator left door edge line from the elevator right door edge line and the expected coordinates (x di ,y di ) In the same coordinate system, each sheetThe elevator door opening degree target of the image serves as a label of the image.
The elevator door opening and closing detection method based on the neural network is characterized by comprising the following steps of: and (3) carrying out upward rounding on the second bit after the decimal point of the target is calculated, and taking the result of the upward rounding as an image tag.
The elevator door opening and closing detection method based on the neural network is characterized by comprising the following steps of: in the second step, the loss function l= - { t.log (P) k )+(1-t)·log(1-P k ) Wherein t represents a true value, P k And (3) representing the prediction probability output by the door opening and closing detection network model, inputting an image of a training set to the door opening and closing detection network model, carrying out back propagation training on the door opening and closing detection network model based on the training set, and when the set iteration times are met, verifying the training effect of the door opening and closing detection network model by adopting a verification set, and adjusting the network parameters of the door opening and closing detection network model according to the test precision of the door opening and closing detection network model on the verification set until the adjusted door opening and closing detection network model meets the set test precision, so as to obtain the trained door opening and closing detection network model.
The invention also comprises an elevator door opening and closing detection system based on the neural network, which comprises the following modules:
a first module for acquiring an image of an elevator door area by using an image acquisition technique;
the second module extracts the coordinates of the corner points, the coordinates of the left door edge line and the coordinates of the right door edge line of the elevator based on the image of the elevator door area, and calculates the opening degree of the elevator door;
a third module for training the neural network to obtain a trained door opening and closing detection network model, calling the door opening and closing detection network model, and calculating the elevator door opening degree P through the door opening and closing detection network model k ；
A fourth module for controlling the opening degree P of the elevator doors according to the plurality of elevator doors in the queue k Calculating the final opening degree P ave ；
A fifth module for according to the final opening procedureDegree P ave And judging whether the elevator door is in a closed state or in a door opening state.
Compared with the prior art, the invention has the following advantages:
1. the invention has simple structure, reasonable design and convenient realization, use and operation.
2. The method acquires the images of the elevator door areas, the elevator door opening degree of each image is used as the label of the image, and the elevator door opening degree is used as the image label, so that the problem of ambiguity caused by classifying the elevator opening and closing states is avoided.
3. The invention accumulates the opening degree P of the elevator door by fusing a plurality of video frames k Calculating to obtain the final opening degree P ave The elevator door switching state is not dependent on a moment in time, but the current time period is integrated, so that the problem of low detection result following degree caused by abnormal blocking of the elevator switching state after switching and before recovery is avoided, the classification accuracy of the elevator switching door is further improved, the overall required calculation force is low, and the elevator door switching device can be rapidly deployed and operated on an embedded platform.
4. In the invention, the opening degree P of the elevator door k Is stored in a queue according to a time sequence, so that the closer to the "head of queue" position the opening degree P of the elevator door is k The longer the time of deposit, the weight of the elevator door is reduced, so that the elevator door opening degree P corresponding to the latest video frame k The weight of the model is larger and is closer to the current real situation.
In conclusion, the elevator door opening degree detection device is simple in structure and reasonable in design, and the elevator door opening degree is used as an image tag, so that the problem of ambiguity caused by classifying the elevator opening and closing states is avoided; elevator door opening degree P accumulated by fusing a plurality of video frames k Calculating to obtain the final opening degree P ave The elevator door switching state is not dependent on a moment, the problem that the detection result is low in following degree due to the fact that the elevator is abnormally blocked to switch and then is recovered to the previous switching state is avoided, the overall required calculation force is low, and the elevator door switching state can be rapidly deployed and operated on an embedded platform.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The method of the present invention will be described in further detail with reference to the accompanying drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The invention will be described in detail below with reference to the drawings in connection with embodiments.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be capable of being practiced otherwise than as specifically illustrated and described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Spatially relative terms, such as "above … …," "above … …," "upper surface at … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial location relative to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as "above" or "over" other devices or structures would then be oriented "below" or "beneath" the other devices or structures. Thus, the exemplary term "above … …" may include both orientations of "above … …" and "below … …". The device may also be positioned in other different ways (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.
Example 1
As shown in fig. 1, the present invention includes the steps of:
step one, acquiring images of elevator door areas, wherein the elevator door opening degree of each image is used as a label of the image, the images form an image set, and the images in the image set are divided into a training set and a verification set.
To better illustrate the present invention, the present embodiment is implemented in a hardware environment: CPU: intel (R) Core (TM) i5-9600K@3.70GHz, memory: 8G or 16G, display card: geForce RTX2080Ti, 11G. Software environment: the running software environment was python3.7+pytorch1.6, and experiments were performed under Windows 10.
When in actual use, randomly generated noise is added into the image, jitter in the running process of the elevator is simulated, and an image sample is generated; preprocessing an image sample, wherein the specific process is as follows:
step 101, obtaining corner coordinates (x) of the elevator door in the image sample si ,y si ) Setting the expected coordinates (x di ,y di ) Based on the corner coordinates and the expected coordinates, a transformation matrix warp is calculated by converting the expected coordinates (x di ,y di ) Is converted into a first matrix of two rows and one column, and the corner coordinates (x si ,y si ) Is converted intoA second matrix of three rows and one column, wherein a third behavior constant of the second matrix, the second matrix multiplied by the transformation matrix warp is equal to the first matrix, wherein i represents an abscissa point number or an ordinate point number, i=2;
step 102, affine transformation is carried out on the images of the elevator door areas of each video frame according to the transformation matrix warp, so that preprocessed images are obtained, and the preprocessed images form an image set.
The corner coordinates refer to the edge corner points of the elevator door pocket. Each elevator door pocket comprises four corner points with coordinates of (x s1 ,y s1 )、(x s1 ,y s2 )、(x s2 ,y s1 ) And (x) s2 ,y s2 ) The identification of the corner points comprises automatic identification based on computer vision and manual calibration; because of the long focus lens used for acquiring the image or the long focus end of the zoom lens, particularly after the focal length converter is used, the image is easy to distort, so that the corner coordinates of the elevator door are shifted. Thus, with image preprocessing, the four corner points of the elevator door in all images are fixed at four desired coordinates (x di ,y di ) And (3) upper part. (x) s1 ,y s1 ) Correspondence (x) d1 ,y d1 )，(x s1 ,y s2 ) Correspondence (x) d1 ,y d2 )，(x s2 ,y s1 ) Correspondence (x) d2 ,y d1 )，(x s2 ,y s2 ) Correspondence (x) d2 ,y d2 ) The distance between different angular points is conveniently calculated under the same coordinate, and the next calculation is convenient
The label generation method of the image comprises the following steps: the computer calculates target=width according to the formula open Width calculates the door opening degree target of the elevator, where width=max (x di )-min(x di )，width open Equal to the distance of the elevator left door edge line from the elevator right door edge line in the coordinate system, the elevator left door edge line from the elevator right door edge line and the expected coordinates (x di ,y di ) The elevator door opening degree target of each image is used as a label of the image under the same coordinate system.
Elevator left door edge in image based on computer vision identificationThe coordinates of the left door edge line and the right door edge line of the elevator are obtained in the same coordinate system, and the distance between the left door edge line and the right door edge line of the elevator can be obtained through simple operation, and the distance is the opening distance width of the elevator door open 。
In actual use, the second bit after the decimal point of the target is calculated is rounded upwards, so that the number of targets serving as image labels is small, the number of images corresponding to the image labels is increased, and the detection accuracy is improved. Meanwhile, the calculation of the image label target is simple, the operation amount is simplified, the opening degree of the elevator door is used as the image label, and the problem of ambiguity caused by classifying the elevator switch state is avoided.
Step two, establishing a switch door detection network model based on a neural network: and establishing a door opening and closing detection network model based on the neural network, performing back propagation training on the door opening and closing detection network model based on the training set, and adjusting network parameters of the door opening and closing detection network model according to the test precision of the door opening and closing detection network model on the verification set to obtain a trained door opening and closing detection network model.
The selection of the door opening and closing detection network model is not particularly required, and a convolutional neural network which is pre-trained on a picture classification task, such as a vgg model, a resnet-34 model, a resnet-50 model or a resnet-56 model, can be used. In this embodiment, the network model for detecting the opening and closing of the door adopts a resnet18 network.
In the second step, the loss function l= - { t.log (P) k )+(1-t)·log(1-P k ) Wherein t represents a true value, P k And representing the prediction probability output by the door opening and closing detection network model, inputting an image of a training set to the door opening and closing detection network model, carrying out back propagation training on the door opening and closing detection network model based on the training set, setting training iteration times, stopping training when the set iteration times are met, and storing the model.
And verifying the training effect of the door opening and closing detection network model by adopting a verification set, if the loss of the door opening and closing detection network model is not reduced, indicating that the network parameters of the door opening and closing detection network model are optimal, finishing model training, and if the loss of the door opening and closing detection network model is continuously reduced, adjusting the network parameters of the door opening and closing detection network model according to the test precision of the door opening and closing detection network model on the verification set, repeating the steps until the loss of the adjusted door opening and closing detection network model on a cross entropy loss function is minimum, finishing training, enabling the door opening and closing detection network model to reach an optimal state, and obtaining the trained door opening and closing detection network model.
In practice, the focal loss function may be used to improve the cross entropy loss function.
Step three, detection:
step 301, obtaining a video frame detection result: collecting real-time images of elevator door areas of each video frame in elevator monitoring videos, inputting the real-time images into a door opening and closing detection network model, and outputting elevator door opening degrees P corresponding to the real-time images by the door opening and closing detection network model k K represents the number of images.
Step 302, fusing detection results of multiple video frames: identifying the time sequence characteristics of the video frame images, and opening the elevator door P corresponding to the video frame images k Storing in a fixed-length first-in first-out queue according to time sequence, the queue is characterized by that every element entering into the queue is first-out, and as if it were in the queue, the element at the head of the queue is dequeued, and the queue only allows deletion operation at "head of queue", if the elevator door entering into the queue is opened to the extent P k If the number of the elevator doors is greater than the length of the queue, the opening degree P of the elevator door which enters the queue first is deleted k 。
According to the opening degree P of all elevator doors in the first-in first-out queue k Calculating the final opening degree P ave Final opening degree P ave The opening degree P of the elevator door corresponding to all video frame images in the queue k When P is the average value of ave >P open When the elevator door is in an open state, judging that the elevator door is in an open state; when P ave <P close When judging that the elevator door is in a closed state, wherein P open Indicating a threshold value of door opening, P close Indicating a door closing threshold.
Elevator door opening degree P accumulated by fusing a plurality of video frames k Calculating to obtain the final opening degree P ave The elevator door switching state is not dependent on a moment in time, but the current time period is integrated, so that the problem of low detection result following degree caused by abnormal blocking of the elevator switching state after switching and before recovery is avoided, the classification accuracy of the elevator switching door is further improved, the overall required calculation force is low, and the elevator door switching device can be rapidly deployed and operated on an embedded platform.
Example two
In this embodiment, an array weight of the fifo queue is set, the array weight is assigned to a person, and the elevator door opening degree P corresponding to the video frame image is determined according to the array weight k Is used for calculating the final opening degree P by weighting the positions of the array ave Wherein the array weight closer to the head of the queue is smaller, and the array weight closer to the tail of the queue is larger.
In actual use, the elevator door is opened to a degree P k Is stored in a queue according to a time sequence, so that the closer to the "head of queue" position the opening degree P of the elevator door is k The longer the time of deposit, the weight of the elevator door is reduced, so that the elevator door opening degree P corresponding to the latest video frame k The weight of the model is larger and is closer to the current real situation.
Example III
The invention also comprises an elevator door opening and closing detection system based on the neural network, which comprises the following modules:
a first module for acquiring an image of an elevator door area by using an image acquisition technique;
the second module extracts the coordinates of the corner points, the coordinates of the left door edge line and the coordinates of the right door edge line of the elevator based on the image of the elevator door area, and calculates the opening degree of the elevator door;
a third module for training the neural network to obtain a trained door opening and closing detection network model, and calling the door opening and closing detection network model to detect the door through opening and closingNetwork measuring model for calculating elevator door opening degree P k ；
A fourth module for controlling the opening degree P of the elevator doors according to the plurality of elevator doors in the queue k Calculating the final opening degree P ave ；
A fifth module for controlling the final opening degree P ave And judging whether the elevator door is in a closed state or in a door opening state.
The foregoing is merely an embodiment of the present invention, and the present invention is not limited thereto, and any simple modification, variation and equivalent structural changes made to the foregoing embodiment according to the technical matter of the present invention still fall within the scope of the technical solution of the present invention.
Claims (8)
1. The elevator door opening and closing detection method based on the neural network is characterized by comprising the following steps of: the method comprises the following steps:
step one, acquiring images of elevator door areas, wherein the opening degree of an elevator door of each image is used as a label of the image, the images form an image set, and the images in the image set are divided into a training set and a verification set;
step two, establishing a switch door detection network model based on a neural network: establishing a door opening and closing detection network model based on a neural network, performing back propagation training on the door opening and closing detection network model based on a training set, and adjusting network parameters of the door opening and closing detection network model according to the test precision of the door opening and closing detection network model on a verification set to obtain a trained door opening and closing detection network model;
step three, detection:
step 301, obtaining a video frame detection result: collecting real-time images of elevator door areas of each video frame in elevator monitoring videos, inputting the real-time images into a door opening and closing detection network model, and outputting elevator door opening degrees P corresponding to the real-time images by the door opening and closing detection network model k K represents the number of images;
step 302, fusing detection results of multiple video frames: identifying the time sequence characteristics of the video frame images, and opening the elevator door P corresponding to the video frame images k According to time sequence, storing a lengthIn a fixed first-in first-out queue, if the elevator door entering the queue is open to a degree P k If the number of the elevator doors is greater than the length of the queue, the opening degree P of the elevator door which enters the queue first is deleted k According to the opening degree P of all elevator doors in the first-in first-out queue k Calculating the final opening degree P ave When P ave >P open When the elevator door is in an open state, judging that the elevator door is in an open state; when P ave <P close When judging that the elevator door is in a closed state, wherein P open Indicating a threshold value of door opening, P close Indicating a door closing threshold.
2. The elevator door opening and closing detection method based on the neural network as claimed in claim 1, wherein: final opening degree P ave The opening degree P of the elevator door corresponding to all video frame images in the queue k Is a mean value of (c).
3. The elevator door opening and closing detection method based on the neural network as claimed in claim 1, wherein: setting array weights of a first-in first-out queue, and according to the array weights and the elevator door opening degree P corresponding to the video frame images k Position weighting at array calculates final opening degree P ave Wherein the array weight closer to the head of the queue is smaller, the array weight closer to the tail of the queue is larger, and the sum of all weights is 1.
4. The elevator door opening and closing detection method based on the neural network as claimed in claim 1, wherein: the first step also comprises preprocessing the image, and the specific process is as follows:
step 101, obtaining corner coordinates (x) of the elevator door in the image si ,y si ) Setting the expected coordinates (x di ,y di ) Calculating a transformation matrix WarpM according to the angular point coordinates and the expected coordinates, wherein i represents the number of points of an abscissa or the number of points of an ordinate;
step 102, affine transformation is carried out on the images of the elevator door areas of each video frame according to the transformation matrix warp, so that preprocessed images are obtained, and the preprocessed images form an image set.
5. The elevator door opening and closing detection method based on the neural network as claimed in claim 1, wherein: the method for generating the label of the image in the first step comprises the following steps: the computer calculates target=width according to the formula open Width calculation of elevator door opening degree, where width=max (x di )-min(x di )，width open Equal to the distance of the elevator left door edge line from the elevator right door edge line in the coordinate system, the elevator left door edge line from the elevator right door edge line and the expected coordinates (x di ,y di ) The elevator door opening degree target of each image is used as a label of the image under the same coordinate system.
6. The elevator door opening and closing detection method based on the neural network according to claim 5, wherein: and (3) carrying out upward rounding on the second bit after the decimal point of the target is calculated, and taking the result of the upward rounding as an image tag.
7. The elevator door opening and closing detection method based on the neural network as claimed in claim 1, wherein: in the second step, the loss function l= - { t.log (P) k )+(1-t)·log(1-P k ) Wherein t represents a true value, P k And (3) representing the prediction probability output by the door opening and closing detection network model, inputting an image of a training set to the door opening and closing detection network model, carrying out back propagation training on the door opening and closing detection network model based on the training set, and when the set iteration times are met, verifying the training effect of the door opening and closing detection network model by adopting a verification set, and adjusting the network parameters of the door opening and closing detection network model according to the test precision of the door opening and closing detection network model on the verification set until the adjusted door opening and closing detection network model meets the set test precision, so as to obtain the trained door opening and closing detection network model.
8. An elevator door opening and closing detection system based on a neural network, for implementing the method as claimed in claim 1, comprising the following modules:
a first module for acquiring an image of an elevator door area by using an image acquisition technique;
the second module extracts the coordinates of the corner points, the coordinates of the left door edge line and the coordinates of the right door edge line of the elevator based on the image of the elevator door area, and calculates the opening degree of the elevator door;
a third module for training the neural network to obtain a trained door opening and closing detection network model, calling the door opening and closing detection network model, and calculating the elevator door opening degree P through the door opening and closing detection network model k ；
A fourth module for controlling the opening degree P of the elevator doors according to the plurality of elevator doors in the queue k Calculating the final opening degree P ave ；
A fifth module for controlling the final opening degree P ave And judging whether the elevator door is in a closed state or in a door opening state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310093099.3A CN116071682A (en) | 2023-02-01 | 2023-02-01 | Elevator door opening and closing detection method and system based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310093099.3A CN116071682A (en) | 2023-02-01 | 2023-02-01 | Elevator door opening and closing detection method and system based on neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116071682A true CN116071682A (en) | 2023-05-05 |
Family
ID=86183461
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310093099.3A Pending CN116071682A (en) | 2023-02-01 | 2023-02-01 | Elevator door opening and closing detection method and system based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116071682A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117115740A (en) * | 2023-09-05 | 2023-11-24 | 北京智芯微电子科技有限公司 | Method, device and equipment for detecting elevator door opening and closing state based on deep learning |
| CN117630344A (en) * | 2024-01-25 | 2024-03-01 | 西南科技大学 | Method for detecting slump range of concrete on line in real time |
-
2023
- 2023-02-01 CN CN202310093099.3A patent/CN116071682A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117115740A (en) * | 2023-09-05 | 2023-11-24 | 北京智芯微电子科技有限公司 | Method, device and equipment for detecting elevator door opening and closing state based on deep learning |
| CN117630344A (en) * | 2024-01-25 | 2024-03-01 | 西南科技大学 | Method for detecting slump range of concrete on line in real time |
| CN117630344B (en) * | 2024-01-25 | 2024-04-05 | 西南科技大学 | Method for detecting slump range of concrete on line in real time |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112380952B (en) | Power equipment infrared image real-time detection and identification method based on artificial intelligence | |
| KR101607224B1 (en) | Dynamic object classification | |
| CN116071682A (en) | Elevator door opening and closing detection method and system based on neural network | |
| CN108921051B (en) | Pedestrian attribute identification network and technology based on cyclic neural network attention model | |
| CN105389567B (en) | Group abnormality detection method based on dense optical flow histogram | |
| CN109359536B (en) | Passenger behavior monitoring method based on machine vision | |
| CN113496104B (en) | Precipitation prediction correction method and system based on deep learning | |
| CN105488456B (en) | Method for detecting human face based on adaptive threshold adjustment rejection sub-space learning | |
| JP5459674B2 (en) | Moving object tracking system and moving object tracking method | |
| EP2518661A2 (en) | System and method for human detection and counting using background modeling, hog and haar features | |
| CN113269073A (en) | Ship multi-target tracking method based on YOLO V5 algorithm | |
| US20070003141A1 (en) | System and method for automatic person counting and detection of specific events | |
| CN111932583A (en) | Space-time information integrated intelligent tracking method based on complex background | |
| CN106940790B (en) | People flow congestion prediction method and system | |
| CN106570490B (en) | A kind of pedestrian's method for real time tracking based on quick clustering | |
| CN113192105B (en) | Method and device for indoor multi-person tracking and attitude measurement | |
| CN112926522B (en) | Behavior recognition method based on skeleton gesture and space-time diagram convolution network | |
| KR20180138558A (en) | Image Analysis Method and Server Apparatus for Detecting Object | |
| CN112257799A (en) | Method, system and device for detecting household garbage target | |
| CN111444816A (en) | Multi-scale dense pedestrian detection method based on fast RCNN | |
| CN114821154A (en) | Grain depot ventilation window state detection algorithm based on deep learning | |
| Andiani et al. | Face recognition for work attendance using multitask convolutional neural network (MTCNN) and pre-trained facenet | |
| CN106384089A (en) | Human body reliable detection method based on lifelong learning | |
| CN113780145A (en) | Sperm morphology detection method, sperm morphology detection device, computer equipment and storage medium | |
| CN106682604B (en) | Blurred image detection method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |