CN114332694A

CN114332694A - Elevator smoke identification method and system, terminal equipment and storage medium

Info

Publication number: CN114332694A
Application number: CN202111566166.6A
Authority: CN
Inventors: 李成文; 钟晨初; 董晓楠; 李学锋; 田文龙
Original assignee: Suzhou Huichuan Control Technology Co Ltd
Current assignee: Suzhou Huichuan Control Technology Co Ltd
Priority date: 2021-12-20
Filing date: 2021-12-20
Publication date: 2022-04-12

Abstract

The invention discloses an elevator smoke identification method, an elevator smoke identification system, terminal equipment and a storage medium. The method comprises the following steps: acquiring a video frame image in an elevator; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; carrying out image binarization processing on the segmented image to obtain a segmented binary image; and carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result. The invention improves the image recognition precision and the calculation efficiency, and improves the accuracy of judging whether suspected smoke in the elevator is real smoke.

Description

Elevator smoke identification method and system, terminal equipment and storage medium

Technical Field

The invention relates to the field of elevator safety, in particular to an elevator smoke identification method and system based on an optical flow method, terminal equipment and a storage medium.

Background

In recent years, with the increase of high-rise buildings, people rely on elevators more and more strongly. Fire accidents caused by smoke extraction, line faults and the like in the lift car type elevator are frequent nowadays. Existing fire detection technologies mainly detect smoke, harmful gases, and temperature through related sensors. For example, the smoke detector has the characteristics of high sensitivity, high response speed, strong anti-interference capability, low cost, long service life, wide application and the like; however, once the smoke detector is in a false alarm state, the smoke detector cannot intuitively provide pictures for property management personnel, and the property management personnel need to further verify the field situation.

Most elevator cars are internally provided with monitoring cameras, but the traditional video monitoring system needs to artificially monitor monitoring videos in real time, so that the situation of misjudgment or missed judgment is easy to occur. With the development of computer vision, digital image processing and pattern recognition technologies, video-based fire detection technologies are gradually researched and developed; compared with the traditional temperature and smoke sensing detection technology, the video-based image detection technology has many advantages in fire detection, such as suitability for large-area large environment, suitability for severe environment (much dust and high humidity), capability of providing visual fire information and the like.

In order to guarantee real-time performance, the conventional fire monitoring method compresses an image, so that key information is lost, and the identification precision is reduced.

Disclosure of Invention

The embodiment of the invention mainly aims to provide an elevator smoke identification method, an elevator smoke identification system, terminal equipment and a storage medium, and aims to improve the image identification precision and the calculation efficiency and improve the accuracy of judging whether suspected smoke in an elevator is real smoke.

In order to achieve the above object, an embodiment of the present invention provides a conventional elevator smoke recognition method, where the conventional elevator smoke recognition method includes:

acquiring a video frame image in an elevator;

carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image;

carrying out image binarization processing on the segmented image to obtain a segmented binary image;

and carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result.

Optionally, the elevator smoke recognition method is applied to an elevator smoke recognition system, the elevator smoke recognition system includes an optical flow classification model, and the processing procedure of performing smoke recognition on the basis of the segmented binary image based on the optical flow classification model to obtain a smoke recognition result and determining whether the suspected smoke is smoke based on the smoke recognition result includes:

carrying out topdressing on a plurality of continuous frames of the segmentation binary image to obtain a topdressing result;

calculating the diffusion range of the profile of the suspected smoke along with time according to the actual result;

calculating the diffusion rate of the contour points according to the diffusion range;

classifying the diffusion rates to obtain a classification center and calculating the diffusion rate of the classification center;

and if the diffusion rate is greater than a preset threshold value, judging the suspected smoke to be smoke.

Optionally, the elevator smoke recognition system further includes a cloud server, and the step of performing smoke recognition based on the segmented binary image to obtain a smoke recognition result and determining whether the suspected smoke is smoke based on the smoke recognition result includes:

and carrying out smoke identification on the suspected smoke through a cloud server based on the segmentation binary image and a pre-established optical flow classification model, and judging whether the suspected smoke is smoke or not based on the smoke identification result.

Optionally, if the diffusion rate is greater than a preset threshold, the step of determining that the suspected smoke is smoke includes:

and informing the property terminal computer through the cloud server so as to take counter measures for the property.

Optionally, the step of performing image segmentation on the region of interest of the video frame image to obtain a segmented image includes:

performing image segmentation on the region of interest of the video frame image by using a deep learning network through an edge computing device to obtain a segmented image;

and carrying out image binarization processing on the segmented image through the edge computing device to obtain a segmented binary image.

Optionally, the step of acquiring video frame images in the elevator comprises:

receiving an access instruction which is triggered by a user through the property terminal computer and used for accessing the state of a camera in the elevator through the cloud server;

sending the access instruction to the edge computing device through the cloud server;

and acquiring data in the elevator through a camera corresponding to the edge computing device to obtain a video frame image in the elevator.

Optionally, the step of performing, by the cloud server, smoke recognition on the suspected smoke based on the divided binary image and a pre-created optical flow classification model, and determining whether the suspected smoke is smoke based on the smoke recognition result further includes:

and establishing the optical flow classification model based on a plurality of frames of images collected in advance.

In addition, to achieve the above object, the present invention also provides an elevator smoke recognition system, including:

the image acquisition module is used for acquiring a video frame image in the elevator;

the image segmentation module is used for carrying out image segmentation on the interesting region of the video frame image to obtain a segmented image;

the image binarization module is used for carrying out image binarization processing on the segmented image to obtain a segmented binary image;

and the smoke identification module is used for carrying out smoke identification on the basis of the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not on the basis of the smoke identification result.

the edge calculating device is used for acquiring a video frame image in the elevator; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; carrying out image binarization processing on the segmented image to obtain a segmented binary image;

and the cloud server is used for carrying out smoke identification on the basis of the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not on the basis of the smoke identification result.

In addition, to achieve the above object, the present invention also provides a terminal device, including: a memory, a processor and an elevator smoke recognition method stored on the memory and operable on the processor, the program of elevator smoke recognition implementing the steps of the elevator smoke recognition method as described above when executed by the processor.

Furthermore, to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a program for elevator smoke recognition, which when executed by a processor implements the steps of the elevator smoke recognition method as described above.

Furthermore, an embodiment of the present invention also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the elevator smoke identification method as described above is implemented.

The elevator smoke identification method, the elevator smoke identification system, the terminal equipment and the storage medium provided by the embodiment of the invention are used for acquiring a video frame image in an elevator; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; carrying out image binarization processing on the segmented image to obtain a segmented binary image; and carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result. The invention improves the image recognition precision and the calculation efficiency, and improves the accuracy of judging whether suspected smoke in the elevator is real smoke.

Drawings

Fig. 1 is a functional module schematic diagram of a terminal device to which an elevator smoke recognition device belongs;

fig. 2 is a schematic diagram of the architecture of an elevator smoke recognition system according to an embodiment of the present invention;

fig. 3 is a schematic flow chart of a first embodiment of the elevator smoke recognition method of the invention;

fig. 4 is a schematic flow chart of a second embodiment of the elevator smoke recognition method of the invention;

fig. 5 is a schematic flow chart of a third embodiment of the elevator smoke recognition method of the invention;

fig. 6 is a functional block diagram of the elevator smoke recognition system of the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The main solution of the embodiment of the invention is as follows: acquiring a video frame image in an elevator; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; carrying out image binarization processing on the segmented image to obtain a segmented binary image; and carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result. Therefore, the precision of the image to be detected is improved by carrying out image segmentation and image binarization processing on the image; by carrying out smoke identification on the divided binary image, the accuracy of judging whether suspected smoke in the elevator is real smoke is improved, and misjudgment is avoided.

The technical terms related to the embodiment of the invention are as follows:

a convolutional neural network: (CNN, systematic Neural Networks), which is a kind of feed-forward Neural network containing convolution calculation and having a deep structure, is one of the representative algorithms of deep learning. The convolutional neural network has the characteristic learning ability and can carry out translation invariant classification on input information according to the hierarchical structure of the convolutional neural network.

The input layer of the convolutional neural network can process multidimensional data, and the input layer of the one-dimensional convolutional neural network receives a one-dimensional or two-dimensional array, wherein the one-dimensional array is usually a time or frequency spectrum sample; the two-dimensional array may include a plurality of channels; an input layer of the two-dimensional convolutional neural network receives a two-dimensional or three-dimensional array; the input layer of the three-dimensional convolutional neural network receives a four-dimensional array.

The hidden layer of the convolutional neural network comprises 3 types of common structures of a convolutional layer, a pooling layer and a full-link layer, and in some more modern algorithms, there may be complex structures such as an inclusion module and a residual block. In a common architecture, convolutional and pooling layers are characteristic of convolutional neural networks. The convolution kernel in the convolutional layer contains weight coefficients, while the pooling layer does not, and therefore in the literature, the pooling layer may not be considered a separate layer. Taking LeNet-5 as an example, the order of 3 types of common structures in the hidden layer is usually: input-convolutional layer-pooling layer-full-link layer-output.

The convolutional neural network is usually a fully-connected layer upstream of the output layer, and thus has the same structure and operation principle as the output layer in the conventional feedforward neural network. For the image classification problem, the output layer outputs the classification label using a logistic function or a normalized exponential function. In an object recognition problem, the output layer may be designed to output the center coordinates, size, and classification of the object. In the image semantic segmentation, the output layer directly outputs the classification result of each pixel.

Region of interest: (ROI) in machine vision and image processing, a region to be processed is delineated from a processed image in a mode of a square frame, a circle, an ellipse, an irregular polygon and the like, and the region is called as a region of interest. In the field of image processing, the region of interest is an image region selected from an image, which is the focus of your image analysis. The area is delineated for further processing.

Image binarization: (Image Binarization) is a process of setting the gray value of a pixel point on an Image to be 0 or 255, that is, displaying an obvious black-and-white effect on the whole Image. In digital image processing, a binary image plays a very important role, and binarization of an image greatly reduces the amount of data in the image, thereby making it possible to highlight the contour of a target.

An optical flow method: optical flow is a concept in the detection of motion of objects in the field of view. To describe the motion of an observed object, surface or edge caused by motion relative to an observer. The optical flow method is very useful in the fields of pattern recognition, computer vision and other image processing, and can be used for motion detection, object cutting, calculation of collision time and object expansion, motion compensation coding, or stereo measurement through the surface and the edge of an object, and the like.

U-net: U-Net is one of the older algorithms for semantic segmentation using fully convolutional networks, and is a classic fully convolutional network, i.e. there is no fully-connected operation in the network.

K-means: the method comprises the steps of dividing data into K groups in advance, randomly selecting K objects as initial clustering centers, calculating the distance between each object and each seed clustering center, and allocating each object to the nearest clustering center. The cluster centers and the objects assigned to them represent a cluster. The cluster center of a cluster is recalculated for each sample assigned based on the objects existing in the cluster. This process will be repeated until some termination condition is met. The termination condition may be that no (or minimum number) objects are reassigned to different clusters, no (or minimum number) cluster centers are changed again, and the sum of squared errors is locally minimal.

OTSU: the central idea of OTSU is that the threshold T should maximize the inter-class variance of both target and background classes. For an image, when the segmentation threshold of the foreground and the background is t, the ratio of foreground points to the image is w0, the average value is u0, the ratio of background points to the image is w1, and the average value is u 1. The average of the entire image is u-w 0 u0+ w1 u 1. And establishing an objective function g (t) w0 (u0-u) 2+ w1 (u1-u) 2, and g (t) which is the inter-class variance expression when the segmentation threshold is t. The OTSU algorithm makes g (t) take the global maximum, and when g (t) is maximum, the corresponding t is called the optimal threshold. The OTSU algorithm is also known as the variance between maximum classes.

And (3) Kittle: the Kittler algorithm has the effect similar to that of the Otsu method, but has higher speed, and is more suitable for being applied to images with higher pixel quality. The central idea is to calculate the average value of the gradient gray scale of the whole image, and use the average value as the threshold value.

The SVM (Support Vector Machine) is a generalized linear classifier (generalized linear classifier) that performs binary classification on data in a supervised learning manner, and a decision boundary of the SVM is a maximum edge distance hyperplane for solving a learning sample. The SVM calculates an empirical risk (empirical risk) using a hinge loss function (change loss) and adds a regularization term to a solution system to optimize a structural risk (structural risk), which is a classifier with sparsity and robustness. SVMs can be classified non-linearly by a kernel method, which is one of the common kernel learning (kernel learning) methods.

In the conventional fire monitoring method, a deep LSTM network is utilized to perform time sequence feature learning on an optical flow picture to obtain a classification model, and then the picture is classified and identified to finally achieve the purpose of fire monitoring. However, in order to ensure real-time performance, the images are compressed, so that critical information is lost, and the identification precision is reduced; meanwhile, the image pixel value is adopted for calculation, so that the influence of the highlight light spot on the identification precision is large.

And calculating the average value and variance of an array formed by the sizes of all corner point optical flow velocities in the foreground image and the average value and variance of an array formed by the directions of the optical flow velocities, and judging whether the fire smoke exists or not according to a set threshold value. However, the calculation efficiency is reduced by performing optical flow calculation on the whole image, and similarly, the influence of highlight light spots can be reduced to a certain extent by adopting the average value of the pixel points, but the influence on the identification precision still exists.

The invention provides a solution, which solves the problems of low identification precision and low calculation efficiency in the prior art, accurately identifies the contour information of suspected smoke by an image segmentation method and an image binarization method, further tracks by using an optical flow method, achieves the aim of accurately identifying the smoke according to a diffusion range and a diffusion rate, and improves the identification precision.

Specifically, referring to fig. 1, fig. 1 is a functional module schematic diagram of a terminal device to which an elevator smoke recognition device of the present invention belongs. The elevator smoke recognition device can be a device which is independent of the terminal equipment, can process pictures and train a network model, and can be borne on the terminal equipment in a hardware or software mode. The terminal device can be an intelligent mobile terminal with a data processing function, such as a mobile phone and a tablet personal computer, and can also be a fixed terminal device or a server with a data processing function.

In this embodiment, the terminal device to which the elevator smoke recognition apparatus belongs at least includes an output module 110, a processor 120, a memory 130, and a communication module 140.

The memory 130 stores an operation method and an elevator smoke identification program, and the elevator smoke identification device can perform image segmentation and image binarization processing on a collected video image in the elevator, accurately identify the contour information of suspected smoke, track the contour information to obtain a judgment result and store the judgment result in the memory 130; the output module 110 may be a display screen or the like. The communication module 140 may include a WIFI module, a mobile communication module, a bluetooth module, and the like, and communicates with an external device or a server through the communication module 140.

Wherein the elevator smoke recognition program in the memory 130 when executed by the processor implements the steps of:

acquiring a video frame image in an elevator;

Further, the elevator smoke recognition program in the memory 130 when executed by the processor further performs the steps of:

the image segmentation is carried out on the interested region of the video frame image, and the step of obtaining the segmented image comprises the following steps:

According to the scheme, the video frame image in the elevator is obtained; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; carrying out image binarization processing on the segmented image to obtain a segmented binary image; and carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result. Therefore, the contour information of the suspected smoke is more accurately identified through an image segmentation method and an image binarization method, the track is further tracked by an optical flow method, and the aim of accurately identifying the smoke is fulfilled according to the diffusion range and the diffusion rate. Through the mode that the edge computing node and the cloud server are combined, the computing efficiency can be further improved.

Referring to fig. 2, fig. 2 is a schematic diagram of an architecture of an elevator smoke recognition system according to an embodiment of the present invention.

As shown in fig. 2, the elevator smoke recognition system mainly includes: edge computing device 3, cloud server 1, wherein:

the edge calculating device 3 is used for acquiring a video frame image in the elevator; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; and carrying out image binarization processing on the segmented image to obtain a segmented binary image.

And the cloud server 1 is used for performing smoke identification based on the divided binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke based on the smoke identification result.

Specifically, as shown in fig. 2, as an embodiment, the elevator smoke recognition system includes: the system comprises a cloud server 1, a cloud server database 2, universal

edge computing devices

3 and 7, local data 4 and 8,

cameras

5 and 6 and a property terminal computer 9. Wherein:

the cloud server 1 is respectively connected with the cloud database 2, the edge computing device 3 and the edge computing device 7; the edge computing device 3 is respectively connected with the local database 4, the elevator camera 5 and the elevator camera 6; the edge computing device 7 is respectively connected with the local database 8 and the property terminal computer 9.

When a user accesses the state of the camera 5 by remotely accessing the cloud server 1 through the network of the property terminal computer 9, first, the cloud server 1 broadcasts an access instruction to the edge computing device 3 and the edge computing device 7, and the edge computing device 3 and the edge computing device 7 match the access instruction.

Thus, by the matching, it is determined that the property terminal computer 9 connected to the edge calculation device 7 needs to access the state of the camera 5.

Secondly, after the peripheral interface or the network module of the edge computing device 3 receives the access instruction, data acquisition is performed through the camera 5 and the camera 6 corresponding to the edge computing device 3. The camera 5 and the camera 6 transmit data acquired in real time to the peripheral interface or the network module of the edge computing device 3 connected thereto, and the peripheral interface or the network module uploads the received data to the memory of the corresponding edge computing device 3.

Thirdly, the computing unit of the edge computing device 3 preprocesses the image data by using an image segmentation method and an image binarization method, and uploads the computing result to the cloud server 1 through the network module of the edge computing device 3.

Finally, the cloud server 1 identifies the smoke by using the optical flow classification model, stores the result into the cloud database 2, and also stores the calculation result into the local database 4 through the peripheral interface of the edge calculation device 3.

It should be noted that the number of the elevator cameras, the number of the edge computing devices, and the number of the local databases may be set according to actual situations, and this embodiment is not particularly limited.

According to the scheme, the video frame image in the elevator is obtained; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; carrying out image binarization processing on the segmented image to obtain a segmented binary image; and carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result. Therefore, firstly, the contour information of suspected smoke of the video frame image is identified by a computing unit of the edge computing device by using an image segmentation method and an image binarization method, so that the contour information of the smoke is accurately reserved, and the computing efficiency is greatly improved; then, the cloud server identifies the smoke by using the optical flow classification model, so that the identification accuracy is greatly guaranteed, and the calculation efficiency is improved.

Based on the above terminal device architecture but not limited to the above architecture, embodiments of the method of the present invention are presented.

Referring to fig. 3, fig. 3 is a schematic flow chart of a first embodiment of the elevator smoke identification method of the invention. The elevator smoke identification method comprises the following steps:

and step S101, acquiring a video frame image in the elevator.

The execution main body of the method can be an elevator smoke recognition device, and also can be a terminal device, a cloud server or an elevator smoke recognition system.

In order to monitor suspected smoke in an elevator, firstly, a video frame image in the elevator is obtained.

As an implementation manner, in this embodiment, first, a camera in the elevator collects a video in the elevator; and then uploading the data frame by frame to the edge computing device and a local database through a network for storage.

Therefore, the video is uploaded to the edge computing device and the local database frame by frame through the network, so that the video image can be analyzed frame by frame subsequently, and the detection efficiency is improved.

As another implementation manner, in this embodiment, a user triggers an access instruction for accessing the state of the camera in the elevator through the property terminal computer, and receives the access instruction for accessing the state of the camera in the elevator, which is triggered by the user through the property terminal computer, through the cloud server; sending the access instruction to the edge computing device through the cloud server; and carrying out data acquisition on the elevator through a camera corresponding to the edge computing device to obtain a video frame image in the elevator.

Specifically, a user remotely accesses the cloud server through the network of the property terminal computer to access the state of the camera, the cloud server broadcasts an access instruction to the edge computing device, and the edge computing device matches the access instruction. And after the peripheral interface or the network module of the edge computing device receives the access instruction, data acquisition is carried out through a camera corresponding to the edge computing device. The camera transmits the data collected in real time to a peripheral interface or a network module of the edge computing device connected with the camera, and the peripheral interface or the network module uploads the received data to a memory of the corresponding edge computing device.

And step S102, carrying out image segmentation on the interesting region of the video frame image to obtain a segmented image.

As an embodiment, in this embodiment, the edge calculation device performs image segmentation on the suspected smoke ROI of the single frame image by using a deep learning network, and determines whether there is a suspected smoke region according to the image segmentation result.

Specifically, in this embodiment, the edge computing device performs image segmentation on the suspected smoke ROI of the video frame image by using a convolutional neural network.

If the suspected smoke area exists, obtaining a segmented image;

if the smoke-like area is clear, the process returns to step S101: video frame images in the elevator are acquired. The next moment is identified.

As another embodiment, in this embodiment, the edge computing device performs image segmentation on the suspected smoke ROI of the single frame image using U-net.

If the suspected smoke area exists, obtaining a segmented image;

Therefore, the suspected smoke ROI of the single-frame image is subjected to image segmentation by the edge computing device through the deep learning network, smoke is preliminarily extracted, and the computing efficiency is improved.

And step S103, carrying out image binarization processing on the segmented image to obtain a segmented binary image.

As an implementation manner, in this embodiment, the edge computing device performs a binary classification process of 0-1 on the segmented image through the classifier, so as to obtain a segmented binary image, and uploads the segmented binary image to the cloud database.

Specifically, the edge computing device performs 0-1 binarization classification processing on the segmented image through a K-means classifier to obtain a segmented binary image, and uploads the segmented binary image to a cloud database through a network module of the edge computing device.

As another implementation manner, in this embodiment, the edge computing device performs binarization classification processing on the segmented image through an OTSU algorithm to obtain a segmented binary image, and uploads the segmented binary image to the cloud database.

As another implementation manner, in this embodiment, the edge computing device performs binarization classification processing on the segmented image through a Kittle algorithm to obtain a segmented binary image, and uploads the segmented binary image to the cloud database.

Therefore, the edge calculation device utilizes the classifier to carry out binarization processing on the segmented image, so that the influence of the highlight light spots is isolated, the integrity of the external contour of the follow-up smoke is ensured, the burden of later-stage work is reduced, and the calculation efficiency is improved.

And step S104, performing smoke identification based on the segmentation binary image to obtain a smoke identification result, and judging whether the suspected smoke is smoke based on the smoke identification result.

As an implementation manner, in this embodiment, the cloud server performs smoke recognition on the divided binary image, and if it is determined that the suspected smoke is smoke for multiple times within a preset time, the cloud server sends the suspected smoke to the property through the property terminal computer in time, so that the property can take measures.

Further, the recognition result is stored in a cloud database, and the result is stored in a local database through a peripheral interface of the edge computing device.

It should be noted that the preset time is set according to actual situations, and this embodiment is not specifically limited.

According to the scheme, the video frame image in the elevator is obtained; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; carrying out image binarization processing on the segmented image to obtain a segmented binary image; and carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result. Therefore, through image segmentation and image binarization, the original image is avoided being calculated, meanwhile, the contour information of the smoke is accurately kept, the calculated amount is greatly reduced, then the smoke is identified through the optical flow classification model, and the identification accuracy is greatly guaranteed.

Referring to fig. 4, fig. 4 is a schematic flow chart of a second embodiment of the elevator smoke identification method of the invention. The elevator smoke identification process comprises the following steps:

step 1, a camera acquires an RGB (red, green and blue) image of a pit video of an elevator;

step 2, segmenting the ROI image of suspected smoke;

and 3, judging whether a suspected smoke area exists or not.

As an implementation manner, in this embodiment, a camera in an elevator collects a pit video of the elevator; and then uploading the data frame by frame to the edge computing device and a local database through a network for storage.

The edge computing device utilizes a deep learning network to carry out image segmentation on the suspected smoke region ROI of the single-frame image, and judges whether a suspected smoke region exists according to the image segmentation result.

If the suspected smoke area exists, obtaining a segmented image;

if the smog-like area is clear, returning to the step 1: the camera obtains elevator pit video RGB image.

Therefore, the suspected smoke ROI of the single-frame image is subjected to image segmentation by the edge computing device through the deep learning network, smoke is preliminarily extracted, the burden of later work is reduced, and the computing efficiency is improved.

And 4, K-Means binarization.

As an implementation manner, in this embodiment, the edge computing device performs binary classification processing of 0-1 on the segmented image through a K-means classifier to obtain a segmented binary image, and uploads the segmented binary image to the cloud database through a network module of the edge computing device.

Step 5, classifying the model by an optical flow method, and calculating the diffusion rate of the contour points;

step 6, judging whether the smoke exists;

and 7, alarming.

As an implementation manner, in this embodiment, first, the cloud server calculates a diffusion rate of a contour point of a segmented binary image by using an optical flow classification model; and then, the cloud server judges whether the suspected smoke is the smoke according to the diffusion rate of the contour points of the divided binary image.

If the suspected smoke is judged to be smoke, the cloud server sends an alarm signal to the property in time through the property terminal computer so that the property can take countermeasures.

If the suspected smoke is judged not to be smoke, returning to the step 1: the camera obtains elevator pit video RGB image.

Referring to fig. 5, fig. 5 is a schematic flow chart of a third embodiment of the elevator smoke recognition method of the invention. Based on the embodiment shown in fig. 2, in this embodiment, the elevator smoke recognition method is applied to an elevator smoke recognition system, the elevator smoke recognition system includes an optical flow classification model, and the step S104 is performed based on the optical flow classification model: the processing procedure of carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result comprises the following steps:

step S1041, carrying out space pursuit on a plurality of continuous frames of the segmentation binary image to obtain a space pursuit result;

step S1042, calculating a diffusion range of the profile of the suspected smoke over time according to the actual result.

As an implementation manner, in the present embodiment, a plurality of frames of images are collected in advance to construct an optical flow classification model; the optical flow method classification model comprises a preset threshold value.

Specifically, the cloud server learns the historical data through the deep learning network to obtain a smoke rate threshold value serving as a preset threshold value.

Further, the cloud server performs smoke recognition on the continuous N-frame segmentation binary image through an optical flow classification model, where N is a positive integer, and is determined according to an actual situation, and this embodiment is not particularly limited.

Specifically, the cloud server performs boundary pursuit on the continuous N-frame segmentation binary image by using an optical flow method to obtain a boundary pursuit result, wherein the continuous N-frame segmentation binary image comprises a time-sequential segmentation binary image.

Further, the cloud server calculates the diffusion range of the suspected smoke contour along with the time according to the actual result, and obtains the diffusion range.

Therefore, by setting the preset threshold value, a judgment basis is set for judging suspected smoke, the accuracy of identifying the suspected smoke is improved, and the calculation efficiency is improved.

Step S1043, calculating the diffusion rate of the contour points according to the diffusion range;

and step S1044, classifying the diffusion rates to obtain a classification center and calculating the diffusion rate of the classification center.

As an implementation manner, in this embodiment, first, the diffusion rate of the contour points is calculated according to the diffusion range of the suspected smoke; then, classifying the diffusion rate of the contour points by using a classifier to obtain the diffusion rate of a classification center; finally, the maximum classified central diffusion rate is compared to a preset threshold.

Wherein, the classifier comprises an SVM classifier, a naive Bayes classifier and a K nearest neighbor algorithm classifier.

Specifically, firstly, calculating the diffusion rate of the contour points according to the diffusion range of the suspected smoke; then, classifying the diffusion rates of the contour points by using an SVM classifier to obtain the diffusion rate of a classification center; finally, the maximum classified central diffusion rate is compared to a preset threshold.

As another embodiment, firstly, the diffusion rate of the contour points is calculated according to the diffusion range of the suspected smoke; then, classifying the diffusion rate of the contour points by using a naive Bayes classifier to obtain the diffusion rate of a classification center; finally, the maximum classified central diffusion rate is compared to a preset threshold.

As another embodiment, first, the diffusion rate of the contour points is calculated according to the diffusion range of the suspected smoke; then, classifying the diffusion rate of the contour points by using a classifier of a K nearest neighbor algorithm to obtain the diffusion rate of a classification center; finally, the maximum classified central diffusion rate is compared to a preset threshold.

Step S1045, if the diffusion rate is greater than a preset threshold, determining that the suspected smoke is smoke.

As an implementation manner, in this embodiment, if the maximum classification center diffusion rate is greater than the preset threshold, it is determined that the suspected smoke is smoke, the cloud server sends an instruction to control the alarm to send an alarm signal, and the property terminal computer sends the alarm signal to the property in time, so that the property can take a countermeasure.

If the maximum classification center diffusion rate is not greater than the preset threshold, determining that the suspected smoke is not smoke, and returning to the step S101: and acquiring a video frame image in the elevator, and identifying the next moment.

According to the scheme, the video frame image in the elevator is obtained; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; carrying out image binarization processing on the segmented image to obtain a segmented binary image; and carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result. Therefore, the sequential images are traced by an optical flow method, the diffusion contour of the target is predicted, the diffusion range and the diffusion rate of the target are calculated, and the calculation efficiency is greatly improved.

Referring also to fig. 6, fig. 6 is a functional block diagram of the elevator smoke recognition system of the present invention. The embodiment of the invention also provides an elevator smoke recognition system, which comprises:

the image acquisition module 10 is used for acquiring video frame images in the elevator;

an image segmentation module 20, configured to perform image segmentation on the region of interest of the video frame image to obtain a segmented image;

an image binarization module 30, configured to perform image binarization on the segmented image to obtain a segmented binary image;

and the smoke identification module 40 is configured to perform smoke identification based on the divided binary image to obtain a smoke identification result, and determine whether the suspected smoke is smoke based on the smoke identification result.

The principle and implementation process for realizing elevator smoke recognition in this embodiment refer to the above embodiments, and are not described herein again.

In addition, the embodiment of the present invention further provides a terminal device, where the terminal device includes a memory, a processor, and an elevator smoke recognition program stored on the memory and operable on the processor, and when executed by the processor, the elevator smoke recognition program implements the steps of the elevator smoke recognition method described above.

Because the elevator smoke recognition program is executed by the processor, all technical solutions of all the embodiments are adopted, so that the elevator smoke recognition program at least has all beneficial effects brought by all the technical solutions of all the embodiments, and detailed description is omitted.

Furthermore, an embodiment of the present invention also provides a computer-readable storage medium, on which an elevator smoke recognition program is stored, which when executed by a processor implements the steps of the elevator smoke recognition method as described above.

Furthermore, an embodiment of the present invention also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the elevator smoke identification method according to the above embodiment is implemented.

Compared with the prior art, the elevator smoke identification method, the elevator smoke identification system, the terminal equipment and the storage medium provided by the invention have the advantages that the video frame image in the elevator is obtained; carrying out image segmentation on the region of interest of the video frame image to obtain a segmented image; carrying out image binarization processing on the segmented image to obtain a segmented binary image; and carrying out smoke identification based on the segmentation binary image to obtain a smoke identification result and judging whether the suspected smoke is smoke or not based on the smoke identification result. Therefore, the contour information of the suspected smoke is more accurately identified through an image segmentation method and an image binarization method, the track is further tracked by an optical flow method, and the aim of accurately identifying the smoke is fulfilled according to the diffusion range and the diffusion rate. Through the mode that the edge computing node and the cloud server are combined, the computing efficiency can be further improved.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or method that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, a controlled terminal, or a network device) to execute the method of each embodiment of the present invention.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. An elevator smoke recognition method, characterized in that the method comprises the steps of:

acquiring a video frame image in an elevator;

2. The elevator smoke recognition method according to claim 1, wherein the elevator smoke recognition method is applied to an elevator smoke recognition system, the elevator smoke recognition system comprises an optical flow classification model, and the processing procedure of performing smoke recognition on the basis of the segmented binary image based on the optical flow classification model to obtain a smoke recognition result and judging whether the suspected smoke is smoke or not based on the smoke recognition result comprises:

3. The elevator smoke recognition method according to claim 1, wherein the elevator smoke recognition system further comprises a cloud server, and the steps of performing smoke recognition based on the segmented binary image, obtaining a smoke recognition result, and determining whether the suspected smoke is smoke based on the smoke recognition result comprise:

and performing smoke identification on the suspected smoke through the cloud server based on the segmentation binary image and the pre-established optical flow classification model, and judging whether the suspected smoke is smoke or not based on the smoke identification result.

4. The elevator smoke identification method according to claim 3, wherein the step of determining the suspected smoke as smoke if the diffusion rate is greater than a preset threshold is followed by the step of:

5. The method of claim 3, the elevator smoke recognition system further comprising an edge computing device,

the step of performing image segmentation on the region of interest of the video frame image to obtain a segmented image comprises:

performing image segmentation on the region of interest of the video frame image by using a deep learning network through the edge computing device to obtain a segmented image;

6. The elevator smoke recognition method of claim 5, wherein the step of obtaining video frame images within an elevator comprises:

7. The elevator smoke recognition method according to claim 3, wherein the step of performing, by the cloud server, smoke recognition on the suspected smoke based on the segmented binary image and the optical flow classification model created in advance further includes, before the step of determining whether the suspected smoke is smoke based on the smoke recognition result:

8. An elevator smoke identification system, comprising:

9. An elevator smoke identification system, comprising:

10. Terminal device, characterized in that the terminal device comprises a memory, a processor and an elevator smoke recognition method stored on the memory and operable on the processor, the program of elevator smoke recognition realizing the steps of the elevator smoke recognition method according to any of claims 1-7 when executed by the processor.

11. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a program for elevator smoke recognition, which program, when being executed by a processor, carries out the steps of the elevator smoke recognition method according to any one of claims 1-7.