JP6943401B2 - Fire monitoring system - Google Patents

Description

The present invention relates to a fire monitoring system that determines a fire by a neural network from an image of a monitoring area captured by a surveillance camera such as a fire detector and gives an alarm.

Conventionally, a system for detecting a fire using a sensor that monitors a specific physical quantity such as a smoke detector or a heat detector has been put into practical use.

On the other hand, conventionally, various devices and systems have been proposed in which a fire is detected by performing image processing on an image of a surveillance area captured by a surveillance camera.

In such a fire monitoring system, early detection of a fire is important from the viewpoint of initial fire extinguishing and evacuation guidance.

For this reason, in the conventional device (Patent Document 1), as phenomena caused by smoke accompanying a fire from an image, a decrease in transmittance or contrast, a convergence of a brightness value to a specific value, and a narrowing of the brightness distribution range to disperse the brightness. The decrease in brightness, the change in the average brightness due to smoke, the decrease in the total amount of edges, and the increase in intensity in the low frequency band are derived, and these are comprehensively judged to enable smoke detection.

Japanese Unexamined Patent Publication No. 2008-046916 Japanese Unexamined Patent Publication No. 7-245757 JP-A-2010-238028 Japanese Unexamined Patent Publication No. 6-325270

However, a fire detection system using a sensor that monitors a specific physical quantity has a problem that even if a monitoring standard is satisfied due to a non-fire event, it is regarded as a fire and the fire cannot be correctly determined.

Further, in a conventional fire monitoring system that detects a fire from an image of smoke associated with a fire, the characteristics of smoke such as transparency, contrast, and edge in the smoke image are determined in advance, and the image captured by the surveillance camera is used. It is necessary to generate smoke characteristics by processing, and there are various situations of smoke generation due to fire, and it is extremely difficult to find out what kind of smoke characteristics there are, and it is a decisive factor. Since it is difficult to find the characteristic, a fire monitoring system that accurately judges smoke caused by a fire from a monitoring image and outputs a fire alarm is in the process of being put into practical use.

On the other hand, in recent years, for example, a large number of images of cats and dogs are labeled, and they are trained by a multi-layer neural network equipped with a convolutional neural network, so-called deep learning is performed, and new images are learned. A technique for presenting to a pre-existing multi-layer neural network and determining whether it is a cat or a dog is disclosed.

Further, deep learning is being studied to be used not only for image analysis but also for natural language processing and behavior analysis.

Such a multi-layered neural network is provided in a fire detector that determines a fire from the input information by using an image of a monitoring area captured by a surveillance camera as input information, and during learning, a large number of fires and non-fires occur. If the input information is prepared and trained by the multi-layered neural network, and the input information is input to the trained multi-layered neural network during monitoring, it is estimated with high accuracy whether it is a fire or not from the output and an alarm is issued. It becomes possible to construct a fire monitoring system that outputs.

However, the phenomenon that appears as a fire differs between a smoked fire and a flame fire, and in addition to whether or not the flame rises, the speed, direction, color, etc. of smoke differ, and the image captured by the surveillance camera changes. Also, in a flame fire, the presence or absence of sparks, the color of the flame, the way the flame rises, etc. change depending on the type of fire. Therefore, when a smoked fire and a flaming fire are regarded as the same fire and each fire image is input to the multi-layered neural network of the fire detector for training, the fire image captured by the surveillance camera is input. There is a risk that the determination accuracy of the case will drop, and in particular, it may be difficult to distinguish it from non-fire events caused by steam or smoke caused by cooking.

An object of the present invention is to provide a fire monitoring system capable of determining a fire with high accuracy even if the type of fire is different by using a fire detector provided with a multi-layer neural network.

(Fire detection by deep learning)
The present invention is a fire monitoring system that detects a fire based on the input information of the monitoring area by using a fire detector composed of a multi-layer neural network.
Multiple types of fire detectors learned by deep learning for each type of fire,
A fire judgment unit that judges a fire based on the estimated fire value output when input information is input to multiple types of fire detectors,
Is provided.

(Fire judgment by weighting 1)
The fire judgment unit calculates a weighted fire estimated value by multiplying each of the fire estimated values by a plurality of types of fire detectors by a predetermined weight according to the type of fire, and one of the weighted fire estimated values is a predetermined threshold value. If it exceeds or exceeds, it is judged as a fire.

(Fire judgment 2)
The fire determination unit calculates the fire likelihood as the sum of the fire estimates by a plurality of types of fire detectors, and determines that the fire is a fire when the fire likelihood is equal to or greater than a predetermined threshold.

(Fire judgment by weighting 2)
The fire judgment unit calculates the fire likelihood as the sum of multiplying each of the fire estimates by multiple types of fire detectors by a predetermined weight according to the type of fire, and the fire likelihood exceeds or exceeds a predetermined threshold. If it is judged as a fire.

(Fire judgment 2 with non-fire detection added)
In addition, a non-fire detector that learned non-fire was installed.
The fire judgment unit calculates the fire likelihood from the sum of the fire estimates by multiple types of fire detectors and the non-fire estimates by the non-fire detectors, and if the fire likelihood exceeds or exceeds a predetermined threshold, it is considered as a fire. judge.

(Fire judgment by non-fire detection and weighting 2)
In addition, a non-fire detector that learned non-fire was installed.
The fire judgment unit calculates the fire likelihood as the sum of the fire estimates by multiple types of fire detectors and the non-fire estimates by the non-fire detectors multiplied by a predetermined weight according to the type of fire and non-fire. If the fire likelihood exceeds or exceeds a predetermined threshold, it is judged as a fire.

(Fire estimates and non-fire estimates)
The fire estimates of multiple types of fire detectors are positive values, and the non-fire estimates of non-fire detectors are negative values.

(Setting the weight according to the environment)
The weight according to the type of fire is set according to the environment of the monitoring area indicating the susceptibility to fire.

(Type of fire)
As multiple types of fire detectors
Smoked fire detectors that learned about smoked fires and
Oil fire detectors learned from oil fires and
An electric fire detector that learned about electric fires, and
Chemical fire detectors learned from chemical fires and
Is provided.

(Multi-layer neural network)
Multi-layer neural networks
It consists of a feature extraction unit and a recognition unit.
The feature extraction unit is a convolutional neural network having a plurality of convolutional layers that generate feature information from which the features of the input information are extracted by inputting the input information.
Recognition unit, the total binding neural network for outputting a fire estimate by inputting the feature information output from the convolutional neural network.

(Learning by backpropagation)
The fire detector trains a multi-layer neural network by backpropagation based on an error between a value output when learning information is input to the multi-layer neural network and an expected value of a predetermined value.

(Clarification of fire judgment grounds)
In addition to the fire judgment, the fire judgment unit notifies the type of fire on which the fire judgment is based.

(Basic effect)
The present invention is a plurality of fire monitoring systems learned by deep learning for each type of fire in a fire monitoring system that detects a fire based on input information in a monitoring area using a fire detector composed of a multi-layered neural network. the type of fire detector, because the fire determining fire determination section based on the fire estimation value output if you enter the input information to the plurality of types of fire detectors, are provided, smoldering fire Ya in the monitored area whether appearing different types of fires such as flaming fire occurs as fire flames, smoke velocity, orientation, be different events such as color, input information detected by the monitoring area for each type of fire If input to the learned fire detectors, fire estimate fire detector input information contents corresponding to the type of learned fire has been input is output is increased to obtain a high fire determination result accurate It is possible to ensure the separation from non-fire events such as steam and smoke associated with cooking, and high reliability can be obtained.

(Effect of input information)
In addition, since the plurality of fire detectors input the physical quantity detected by the sensor and / or the image of the monitoring area captured by the imaging unit as input information and output the fire estimated value, the physical quantity detected by the sensor By inputting images of the monitoring area captured by the image pickup unit or the image of the monitoring area into a plurality of types of trained fire detectors, it is possible to determine a fire with high accuracy.

(Effect of fire judgment 1)
Moreover, the fire judgment portion, since any fire estimate of a plurality of types of fire detectors has to be determined to fire if it exceeds more or a predetermined threshold, the input information corresponding to the type of learned fire From the high fire estimate output by the input fire detector, it can be reliably determined as a fire.

(Effect of fire judgment 1 by weighting)
Further, the fire judgment unit calculates a weighted fire estimated value by multiplying each of the fire estimated values by a plurality of types of fire detectors by a predetermined weight according to the type of fire, and one of the weighted fire estimated values is determined. due to so as to determine the fire when the threshold above or beyond the further emphasized, more surely higher fire estimate the fire detector input information is input for the type of learned fire output from the weighting It can be judged as a fire.

(Effect of fire judgment 2)
In addition, the fire judgment unit calculates the fire likelihood as the sum of the estimated fire values by a plurality of types of fire detectors, and determines that the fire is a fire when the fire likelihood exceeds or exceeds a predetermined threshold. obtained a sum of input information corresponding to the type of fire by adding the output fire estimate that the high fire estimate fire different other fire detectors for outputting fire detector is input as a fire likelihood By determining the fire, the fire can be accurately determined, and the non-fire event can be reliably separated from the non-fire event, and the non-fire event can be reliably prevented from being mistakenly determined as a fire.

(Effect of fire judgment 2 by weighting)
In addition, the fire judgment unit calculates the fire likelihood as the sum of multiplying each of the fire estimates by the plurality of types of fire detectors by a predetermined weight according to the type of fire, and the fire likelihood is equal to or higher than the predetermined threshold. due to so as to determine the fire if it exceeds the learning input information corresponding to the type of fire is weighted higher fire estimation value output from the fire detector input, different from the fire detection of the type of fire By weighting the estimated fire output output by the vessel and determining the fire by calculating the sum of these as the fire likelihood, the fire can be judged more accurately and the separation from non-fire events is more reliable. Therefore, it is possible to reliably prevent a non-fire event from being mistakenly determined as a fire.

(Effect of fire judgment 2 including non-fire detection)
Furthermore, a non-fire detector that has learned non-fire is provided, and the fire judgment unit calculates the fire likelihood from the sum of the fire estimates by multiple types of fire detectors and the non-fire estimates by the non-fire detector. , When the fire likelihood exceeds or exceeds a predetermined threshold, it is judged as a fire, so that it is more reliable to distinguish it from a non-fire event, and it is surely prevented that a non-fire event is mistakenly judged as a fire. can.

(Fire judgment by non-fire detection and weighting 2)
Furthermore, a non-fire detector that has learned non-fire is provided, and the fire judgment unit responds to the fire estimation value by multiple types of fire detectors and the non-fire estimation value by the non-fire detection unit according to the type of fire and non-fire. Since the fire likelihood is calculated as the sum of multiplying the predetermined weights and the fire likelihood is determined to be a fire when the fire likelihood exceeds or exceeds the predetermined threshold, it is more reliable to distinguish it from a non-fire event. Therefore, it is possible to reliably prevent a non-fire event from being mistakenly determined as a fire.

(Effects of fire estimates and non-fire estimates)
In addition, since the fire estimates of multiple types of fire detectors are set to positive values and the non-fire estimates of non-fire detectors are set to negative values, the sum of the multiple types of fire estimates and non-fire estimates is used. Alternatively, the value of the fire likelihood obtained as the weighted sum is higher in the case of a fire and lower in the case of a non-fire event, so that it can be accurately determined as a fire and can be reliably separated from the non-fire event.

(Effect of setting weight according to environment)
In addition, since the weight according to the type of fire is set according to the environment of the monitoring area indicating the susceptibility to fire, a large weight is set if the fire environment is prone to fire, and the fire environment is less likely to cause a fire. If so, a small weight is set, and it is possible to make a highly accurate fire judgment according to the environment in which a fire is likely to occur or the environment in which a fire is unlikely to occur. For example, in places where oil is used, such as in a boiler room, the weighting of fire detectors that have learned oil fires is high, and in server rooms, etc., the weighting of fire detectors that have learned electric fires is high. By increasing the weighting of the fire detector that has learned the chemical fire, it is possible to make a highly accurate fire judgment by setting the weight according to the environment of the monitoring area.

(Effect depending on the type of fire)
As multiple types of fire detectors, smoky fire detectors learned smoldering fires, oil fire detectors learned oil fires, electric fire detectors learned electric fires, and chemical fires learned. Even if the type of fire is different, such as smoldering fire, oil fire, electric fire, or chemical fire, it is not affected by the type of fire and the fire can be judged accurately. can do.

(Effect of functional configuration of multi-layer neural network)
Further, the multi-layer neural network is composed of a feature extraction unit and a recognition unit, and the feature extraction unit is a plurality of convolutional networks that generate feature information from which the features of the input information are extracted by inputting the input information of the monitoring area. a neural network convolution with a layer recognition unit, because the set as the total binding neural network for outputting a fire estimate by inputting the feature information output from the convolutional neural network, characterized by a convolutional neural network automatic By pre-processing the input information of the monitoring area, the characteristics of the fire input information, for example, the characteristics of the input information without the need for pre-processing such as extracting the contour etc. in the image. It is possible to estimate the fire with high accuracy by the recognition unit that is extracted and subsequently performed.

(Effect of learning by backpropagation)
The fire detector trains a multi-layer neural network by backpropagation (error backpropagation) based on the error between the value output when the training information is input to the multi-layer neural network and the expected value of a predetermined value. Therefore, when a large number of input information prepared for each type of fire is input as fire input information, the estimated value of the fire is given as the expected value of the output and the error between the output value and the expected value is obtained by backpropagation processing. Weights and biases in a multi-layer neural network are learned so as to minimize, and a fire can be estimated with higher accuracy from the input information and an alarm can be made.

Similarly, when the non-fire input information prepared in advance is input, the estimated value of non-fire is given as the expected value of the output, and the back propagation process minimizes the error between the output value and the expected value. The weights and biases in the neural network of the above are learned, and non-fire can be estimated with higher accuracy from the input information, and false reports can be reliably prevented.

(Effect of clarifying the grounds for fire judgment)
In addition to the fire judgment, the fire judgment department notifies the type of fire that was the basis of the fire judgment, so by knowing the type of fire that occurred, fire extinguishing activities and evacuation instructions corresponding to the type of fire can be issued. Make it possible to do.

Explanatory drawing which showed embodiment of fire monitoring system Explanatory drawing which showed the functional configuration of the smoldering fire detector of FIG. Explanatory diagram showing the functional configuration of the multi-layer neural network shown in FIG. A flowchart showing an embodiment of a fire judgment process by the judgment unit of FIG. A flowchart showing another embodiment of the fire determination process by the determination unit of FIG. Explanatory drawing showing other embodiments of a fire monitoring system A flowchart showing an embodiment of a fire judgment process by the judgment unit of FIG. A flowchart showing another embodiment of the fire determination process by the determination unit of FIG. A flowchart showing another embodiment of the fire determination process by the determination unit of FIG.

[Overview of fire monitoring system]
FIG. 1 is an explanatory diagram showing an embodiment of a fire monitoring system. As shown in FIG. 1, a surveillance camera 16 that functions as an imaging unit is installed in the monitoring area 14 of a facility such as a building, and the monitoring camera 16 captures a moving image of the monitoring area 14. The surveillance camera 16 captures an RGB color image at, for example, 30 frames / second and outputs it as a moving image. Further, one frame has, for example, a pixel arrangement of 4056 × 4056 pixels in length and width.

In addition, on-off type fire detectors 18 are installed in each of the monitoring areas 14, detects the temperature or smoke concentration due to the fire, issues a report when the predetermined threshold level is exceeded, and outputs a fire alarm signal. I try to do it.

A fire determination device 10 and a fire receiver 12 of a fire alarm system are installed in the disaster prevention monitoring center and the manager's room of the facility with respect to the monitoring area 14. The fire determination device 10 and the fire receiver 12 may be integrated. A surveillance camera 16 installed in the surveillance area 14 is connected to the fire determination device 10 by a signal cable 22, and a moving image image captured by the surveillance camera 16 is input.

A sensor line 25 is pulled out from the fire receiver 12 to the monitoring area 14, and a fire detector 18 is connected to each of the sensor lines 25.

The fire determination device 10 is provided with a smoked fire detector 20-1, an oil fire detector 20-2, an electric fire detector 20-3, a chemical fire detector 20-4, and a non-fire detector 20-5. , Each has a multi-layer neural network. In the following description, a plurality of types of fire detectors 20-1 include a smoked fire detector 20-1, an oil fire detector 20-2, an electric fire detector 20-3, and a chemical fire detector 20-4. It may be called 20-4.

Multilayered neural network of the smoldering fire detector 20-1 is learned Ibushisho fire, multi-layer neural network of the oil fire detector 20-2 is oil fires is learned, an electric fire detector 20- 3 of multilayered neural networks have electrical fires are learned, multilayered neural network of chemical fire detector 20-4 are chemicals fire is learned, further, the non-fire detector 20-5 multilayered neural network Is learning non-fire.

Signal cables from surveillance cameras branch to smoked fire detector 20-1, oil fire detector 20-2, electric fire detector 20-3, chemical fire detector 20-4 and non-fire detector 20-5. The video image sent from the surveillance camera 16 is input in frame units to determine the smoked fire estimated value, oil fire estimated value, electric fire estimated value, chemical fire estimated value, and non-fire estimated value. Output to 24.

The determination unit 24 is a smoked fire output from a smoked fire detector 20-1, an oil fire detector 20-2, an electric fire detector 20-3, a chemical fire detector 20-4, and a non-fire detector 20-5. Fire estimated value W, oil fire estimated value X, electric fire estimated value Y, chemical fire estimated value Z and non-fire estimated value N are read, and when a fire is judged based on these estimated values, a fire judgment signal is received. It is output to the machine 12, and for example, a fire sign alarm or the like indicating a fire sign is output. In the following description, the smoked fire estimated value W, the oil fire estimated value X, the electric fire estimated value Y, and the chemical fire estimated value Z may be referred to as a plurality of types of fire estimated values W, X, Y, Z.

As an embodiment of fire determination by the determination unit 24, a smoked fire detector 20-1, an oil fire detector 20-2, an electric fire detector 20-3, a chemical fire detector 20-4, and a non-fire detector 20- The fire likelihood L is calculated as the sum of the smoked fire estimated value W, the oil fire estimated value X, the electric fire estimated value Y, the chemical fire estimated value Z, and the non-fire estimated value N output from 5.

Here, the smoked fire estimated value W, the oil fire estimated value X, the electric fire estimated value Y, and the chemical fire estimated value Z have positive values, and the non-fire estimated value N has negative values. L is
L = W + X + Y + Z + N (1)
Is calculated as. Then, the determination unit 24 determines that the fire is a fire when the fire likelihood L exceeds a predetermined threshold value Lth.

In addition, as another embodiment of fire determination by the determination unit 24, a smoked fire detector 20-1, an oil fire detector 20-2, an electric fire detector 20-3, a chemical fire detector 20-4, and a non-fire Depending on the type of fire and non-fire, the smoked fire estimated value W, oil fire estimated value X, electric fire estimated value Y, chemical fire estimated value Z, and non-fire estimated value N output from the detector 20-5. predetermined weights a, b, c, d, the fire likelihood L as a sum obtained by multiplying the e L = Wa + Xb + Yc + Zd + N e (2)
When the fire likelihood L exceeds a predetermined threshold value Lth, it is determined as a fire.

Here, the weights a, b, c, d, and e for weighting the smoked fire estimated value W, the oil fire estimated value X, the electric fire estimated value Y, the chemical fire estimated value Z, and the non-fire estimated value N are the weights of the fire. It is set according to the environment of the monitoring area 14 indicating the ease of occurrence.

For example, when there is a fire source, cloth, or the like that causes a smoked fire in the monitoring area 14, the weight a with respect to the smoked fire estimated value W is set to, for example, a = 2, and the other weights b to e. Is set to 1 or a value less than 1. When the monitoring area 14 uses oil as in the boiler room, the weight b for the estimated oil fire value X is set to, for example, b = 2, and the other weights a, c to e are 1 or It is set to a value smaller than 1.

When the monitoring area 14 is the server room, the weight c for the estimated electric fire value Y is set to, for example, c = 2, and the other weights a, b, d, and e are set to values smaller than 1 or 1. Will be done. When the monitoring area 14 is a chemical storage room or a laboratory, the weight d with respect to the chemical fire estimated value Z is set to, for example, d = 2, and the other weights a to c, e are values smaller than 1 or 1. Is set to.

Further, when there is no equipment or object causing a fire in the monitoring area 14, the weight e with respect to the non-fire estimated value N is set to, for example, e = 2, and the other weights a to d are values smaller than 1 or 1. Is set to.

In this way, weights are set for each estimated fire value according to whether the environment of the monitoring area 14 is a fire-prone environment or a fire-prone environment, and if the environment is a fire-prone environment, the fire likelihood L based on the fire estimated value is set. If the environment is such that a fire is unlikely to occur, the degree of contribution to the fire likelihood L based on the estimated fire value is reduced, and a more accurate fire judgment is possible.

[Fire judgment device]
(Functional configuration of fire judgment device)
FIG. 2 is an explanatory diagram showing a functional configuration of a smoked fire detector using a multi-layer neural network that estimates a fire from an image captured by a surveillance camera.

As shown in FIG. 2, the smoked fire detector 20-1 is composed of an image input unit 26, a multi-layer neural network 28, a learning image holding unit 30, and a learning control unit 32, and these functions are provided by the neural network. It is realized by executing a program by the CPU of the computer circuit corresponding to the processing.

The smoked fire detector 20-1 inputs the image of the monitoring area captured by the surveillance camera 16 to the multi-layer neural network 28 via the image input unit 26, and outputs the smoked fire estimated value W.

The learning control unit 32 reads a moving image of a smoked fire from a recording device or the like (not shown), temporarily stores and holds the moving image in the learning image holding unit 30, and sequentially stores images in frame units from the moving image held in the learning image holding unit 30. It is read out and input to the multi-layer neural network 28 as a supervised smoked fire image via the image input unit 26. Learn bias.

When the image of the smoked fire in the monitoring area captured by the surveillance camera 16 was input to the multi-layer neural network 28 that had been learned using the image of the smoked fire with a teacher, the image of the smoked fire was supported. The smoked fire estimated value W is output. The smoldering smoldering fires estimate W output from the fire detector 20-1 becomes a value close to 1 or 1 smoldering estimate W For similar smoldering fires image and used for learning, smoldering In the case of a fire image other than a fire, the smoked fire estimated value W is a value smaller than 1, and in the case of a non-fire image, the smoked fire estimated value W is 0 or a value close to 0.

The oil fire detector 20-2, the electric fire detector 20-3, and the chemical fire detector 20-4 shown in FIG. 1 also have the same image input unit 26 as the smoked fire detector 20-1 shown in FIG. , A multi-layer neural network 28, a learning image holding unit 30, and a learning control unit 32. The learning control unit 32 temporarily stores and holds a moving image of an oil fire, an electric fire, or a chemical fire in the learning image holding unit 30. After that, the images are sequentially read out in frame units and input to the multi-layer neural network 28 as an image of an oil fire, electric fire, or chemical fire with a teacher, and a multi-layer system is used by a learning method such as the back propagation method (error back propagation method). Train the weights and biases of the neural network 28.

An image of an oil fire, an electric fire, or a chemical fire in a surveillance area captured by a surveillance camera 16 on a multi-layer neural network 28 that has been trained using images of an oil fire, an electric fire, or a chemical fire with a teacher. If you enter the respective fire type of oil fires estimate value X corresponding to an image, electrical fires estimate Y or chemical fire estimate Z is output. The oil fire estimated value X, the electric fire estimated value Y, or the chemical fire estimated value Z output from the oil fire detector 20-2, the electric fire detector 20-3, or the chemical fire detector 20-4 is used for learning. and a fire estimate value X, Y, Z in the case of similar fire type of image becomes a value close to 1 or 1, fire estimate value X in the case of fire images other than the type of learned fire, Y, Z is 1 In the case of non-fire images, the estimated fire values X, Y, and Z are 0 or close to 0.

For example, if the flame does not rise and only smoke is generated, the smoked fire estimated value W becomes a prominent value, and if the flame rises and black smoke is generated, the oil fire estimated value X becomes a prominent value and sparks. It is expected that the estimated electric fire value Y will be a prominent value when the fire is scattered, and the estimated chemical fire value Z will be a prominent value when a flame other than red rises.

[Multi-layer neural network]
FIG. 3 is an explanatory diagram showing the functional configuration of the multi-layer neural network shown in FIG. 2, the outline is shown in FIG. 3 (A), and the details are schematically shown in FIG. 3 (B).

As shown in FIG. 3A, the multi-layer neural network 28 of this embodiment is composed of a feature extraction unit 38 and a recognition unit 40. The feature extraction unit 38 is a convolutional neural network, and the recognition unit 40 is a fully connected neural network.

The multi-layer neural network 28 is a neural network that performs deep learning, is a neural network that has a deep layer that connects a plurality of intermediate layers, and performs expression learning that is feature extraction.

A normal neural network requires work by artificial trial and error to extract features for determining a fire from an image, but in a multi-layer neural network 28, a convolutional neural network is used as a feature extraction unit 38. Then, the pixel value of the image is input, the optimum feature is extracted by learning, and the optimum feature is input to the fully connected neural network of the recognition unit 40 to identify whether it is a fire or a non-fire.

As schematically shown in FIG. 3B, the fully connected neural network of the recognition unit 40 is composed of an input layer 46 , an intermediate layer 50, a repetition of the fully connected 48, and an output layer 52.

(Convolutional neural network)
FIG. 3B schematically shows the structure of the convolutional neural network constituting the feature extraction unit 38.

Convolutional neural networks have slightly different characteristics from ordinary neural networks and incorporate biological structures from the visual cortex. The visual cortex contains a receptive field, which is a collection of small cells that are sensitive to a small area of the visual field, and the behavior of the receptive field can be imitated by learning weighting in the form of a matrix. This matrix is called the weight filter (kernel) and is sensitive to similar subregions of an image, similar to the biological role played by receptive fields.

The convolutional neural network can express the similarity between the weight filter and the subarea by the convolution operation, and through this operation, the appropriate feature of the image can be extracted.

As shown in FIG. 3B, the convolutional neural network first performs convolution processing on the input image 42 by the weight filter 43. For example, the weight filter 43 is a matrix filter having a predetermined weighting of 3 × 3 in the vertical and horizontal directions, and by performing a convolution calculation while aligning the filter center with each pixel of the input image 42, 9 pixels of the input image 42 can be obtained. convolution to one pixel of feature map 44a as the subregions, a plurality of feature maps 44a is generated.

Subsequently, the pooling calculation is performed on the feature map 44a obtained by the convolution calculation. The pooling operation is a process of removing features unnecessary for identification and extracting features necessary for identification.

Subsequently, the convolution calculation and the pooling calculation using the weight filters 45a and 45b are repeated in multiple stages to obtain the feature maps 44b and 44c, and the feature map 44c of the last layer is input to the recognition unit 40 to perform a normal full coupling. A recognition unit 40 using a neural network estimates whether it is a fire or a non-fire.

Note that the pooling calculation in the convolutional neural network does not perform the pooling calculation because the features unnecessary for distinguishing between fire and non-fire are not always clear and the necessary features may be deleted. You may do so.

[Learning of multi-layer neural network]
(Backpropagation)
A neural network composed of an input layer, a plurality of intermediate layers, and an output layer is provided with a plurality of units in each layer and connected to a plurality of units in other layers, and weights and bias values are set for each unit. The vector product of multiple input values and weights is calculated, the bias values are added to obtain the sum, and this is passed through a predetermined activation function and output to the unit of the next layer. Forward propagation is performed in which the value propagates until it reaches.

To change the weights and biases of such neural networks, a learning algorithm known as backpropagation is used. In backpropagation, supervised learning when a data set of input value x and expected output value (expected value) y is given to the network, and unsupervised learning when only input value x is given to the network. In this embodiment, supervised learning is performed.

When performing backpropagation in supervised learning, for example, a function of mean square error is used as an error to compare the estimated value y * and the expected value y, which are the results of forward propagation that has passed through the network. do.

In backpropagation, the magnitude of the error between the estimated value y * and the expected value y is used to propagate the value from the rear to the front of the network while correcting the weight and bias. The corrected amount for each weight and bias is treated as a contribution to the error, calculated by the steepest descent method, and the value of the error function is minimized by changing the weight and bias values.

The procedure for learning by backpropagation for a neural network is as follows.
(1) The input value x is input to the neural network, forward propagation is performed, and the estimated value y * is obtained.
(2) Calculate the error with an error function based on the estimated value y * and the expected value y.
(3) Backpropagation is performed on the network while updating the weight and bias.

This procedure is repeated using different combinations of input values x and expected values y to minimize the value of the error function until the neural network weight and bias errors are minimized as much as possible.

[Fire judgment based on multiple types of fire estimates]
(Fire judgment based on the sum of fire estimates)
FIG. 4 is a flowchart showing an embodiment of a fire determination process by the determination unit of FIG. As shown in FIG. 4, the determination unit 24 is output from the smoked fire detector 20-1, the oil fire detector 20-2, the electric fire detector 20-3, and the chemical fire detector 20-4 in step S1. A plurality of types of fire estimation values W, X, Y, and Z are read, then the non-fire estimation value N output from the non-fire detector 20-5 is read in step S2, and the fire likelihood L is calculated in step S3. Calculate according to the above equation (1).

Subsequently, the determination unit 24 compares the fire likelihood L with the predetermined threshold value Lth in step S4, and if it determines that the threshold value is equal to or higher than the threshold value Lth (or exceeds the threshold value Lth), the determination unit 24 proceeds to step S5 and sends the fire determination signal to the fire receiver 12. To output a fire sign alarm, etc.

Here, assuming that the estimated fire value is a value in the range of 0 to 1, when an image of a smoked fire is input, for example, the estimated smoked fire value W = 1.0.
Oil fire estimate X = 0.4
Electric fire estimate Y = 0.3
Chemical fire estimate Z = 0.3
Non-fire estimate N = -0.1
In this case, the fire likelihood L is L = 1.9, and for example, by setting the threshold value Lth to Lth = 0.5, the fire can be reliably determined.

On the other hand, when a non-fire image is input, for example, a smoked fire estimated value W = 0.1.
Oil fire estimate X = 0.0
Electric fire estimate Y = 0.0
Chemical fire estimate Z = 0.1
Non-fire estimate N = -1.0
In this case, the fire likelihood L becomes L = -0.8, and a sufficient difference occurs with respect to the threshold value Lth = 0.5, so that it is surely prevented that a non-fire is mistakenly judged as a fire. can.

Subsequently, the determination unit 24 proceeds to step S6, and the type of fire on which the fire determination is based in the processing of steps S3 to S4, that is, the smoldering fire and the oil fire which are the highest values among the estimated fire values W to Z. , Electric fire or chemical fire is controlled to be displayed on the display of, for example, the fire receiver 12. By displaying the type of fire that was the main factor in the fire judgment in this way, it is possible to appropriately carry out fire extinguishing activities and evacuation guidance in response to smoldering fires, oil fires, electric fires, or chemical fires. It becomes.

(Fire judgment based on the total weighting of fire estimates)
FIG. 5 is a flowchart showing another embodiment of the fire determination process by the determination unit of FIG. As shown in FIG. 5, the determination unit 24 reads the plurality of types of fire estimates W, X, Y, Z output from the plurality of types of fire detectors 20-1 to 20-4 in step S11, and subsequently. , The non-fire estimated value N output from the non-fire detector 20-5 is read in step S12.

Subsequently, the determination unit 24 proceeds to step S13, and weights the plurality of types of fire estimates W, X, Y, Z and non-fire estimates N by multiplying them by preset weights a, b, c, d, and e. Obtain the estimated fire value and the estimated non-fire value Wa, Xb, Yc, Zd, and Ne. Subsequently, in step S14, the determination unit 24 obtains the sum of the weighted fire estimated value and the non-fire estimated value Wa, Xb, Yc, Zd, and Ne as the fire likelihood L according to the above equation (2).

For example, the determination unit 24 sets weights a to e according to the likelihood of a fire in the monitoring area 14, and when monitoring an electric fire in a server room, for example, the weight c for the electric fire is c = 2, for example. is set to 2.0, the weight a, b, d for the type of fire the other is set to each of 1, further weights e against non-fire also set to e = 1.

Here, when an image of an electric fire is input, i, for example, an estimated value of smoked fire W = 0.3.
Oil fire estimate X = 0.4
Electric fire estimate Y = 1.0
Chemical fire estimate Z = 0.3
Non-fire estimate N = -0.1
If, then the weighted fire estimate is
Wa = 0.3
Xb = 0.4
Yc = 2.0
Zd = 0.3
Ne = -0.1
Will be. In this case, the fire likelihood L calculated according to the above equation (2) is L = 2.9.

Subsequently, the determination unit 24 compares the fire likelihood L with the predetermined threshold value Lth in step S15, and if it determines that the fire likelihood Lth is equal to or higher than the threshold value Lth (or exceeds the threshold value Lth), proceeds to step S16 and transmits the fire determination signal to the fire receiver 12. To output a fire sign alarm, etc. In this case, for example, by setting the threshold value Lth as high as Lth = 1.5, the fire can be reliably determined.

On the other hand, when a non-fire image is input, for example, a smoked fire estimated value W = 0.1.
Oil fire estimate X = 0.0
Electric fire estimate Y = 0.0
Chemical fire estimate Z = 0.1
Non-fire estimate N = -1.0
If, then the weighted fire estimate is
Wa = 0.1
Xb = 0.0
Yc = 0.0
Zd = 0.1
Ne = -1.0
Will be.

In this case, the fire likelihood L is L = −0.8, and a sufficient difference from the threshold value Lth = 1.5 is generated, so that it is possible to surely prevent a non-fire from being mistakenly determined as a fire.

Subsequently, the determination unit 24 proceeds to step S17, and the type of fire that became the main factor of the fire determination in the processing of steps S13 to S15, that is, the smoldering fire and oil that are the highest among the estimated fire values W to Z. Control is performed so that either a fire, an electric fire, or a chemical fire is displayed on the display of, for example, the fire receiver 12 as a basis for determining the fire. By displaying the type of fire that is the basis of the fire judgment in this way, it is possible to appropriately carry out fire extinguishing activities and evacuation guidance in response to smoldering fires, oil fires, electric fires, or chemical fires. Become.

[Other Embodiments of Fire Monitoring System]
FIG. 6 is an explanatory diagram showing another embodiment of the fire monitoring system. As shown in FIG. 6, in the present embodiment, the fire determination device 10 includes a smoked fire detector 20-1, an oil fire detector 20-2, an electric fire detector 20-3, and a chemical fire detector. 20-4 is provided, which is different from the embodiment shown in FIG. 1 in that the non-fire detector 20-5 is not provided. Other configurations and functions are the same as those of the embodiment of FIG. 1 except for the determination unit 24.

(Fire judgment by comparing thresholds of multiple types of fire estimates)
As an embodiment of fire determination by the determination unit 24, a plurality of types of fires output from a smoked fire detector 20-1, an oil fire detector 20-2, an electric fire detector 20-3, and a chemical fire detector 20-4. When any of the estimated fire values W, X, Y, and Z exceeds or exceeds the predetermined threshold value TH, it is determined to be a fire and a fire determination signal is output.

FIG. 7 is a flowchart showing an embodiment of the fire determination process by the determination unit of FIG. As shown in FIG. 7, the determination unit 24 is output from the smoked fire detector 20-1, the oil fire detector 20-2, the electric fire detector 20-3, and the chemical fire detector 20-4 in step S21. When a plurality of types of estimated fire values W, X, Y, Z are read, subsequently compared with a predetermined threshold value TH in step S22, and determined to be equal to or higher than the threshold value TH (or exceeds the threshold value TH) in step S23, step S24 Proceeds to, a fire determination signal is output to the fire receiver 12, and a fire sign alarm or the like is output.

Subsequently, the determination unit 24 proceeds to step S25, and the type of fire that became the main factor of the fire determination in steps S22 to S23, that is, the smoldering fire and the oil fire, which are the highest values among the estimated fire values W to Z,. Control is performed so that either an electric fire or a chemical fire is displayed on the display of, for example, the fire receiver 12 as the main cause of the fire.

(Fire judgment based on the sum of fire estimates)
FIG. 8 is a flowchart showing another embodiment of the fire determination process by the determination unit of FIG. As shown in FIG. 8, the determination unit 24 is output from the smoked fire detector 20-1, the oil fire detector 20-2, the electric fire detector 20-3, and the chemical fire detector 20-4 in step S31. Read multiple types of fire estimates W, X, Y, Z.

Subsequently, the determination unit 24 obtains the fire likelihood L as the sum of the estimated fire values W, X, Y, and Z in step S32, compares the fire likelihood L with the predetermined threshold Lth in step S33, and is equal to or greater than the threshold Lth (or). When it is determined (exceeding the threshold value Lth), the process proceeds to step S34, the fire determination signal is output to the fire receiver 12, and a fire sign alarm or the like is output.

Subsequently, the determination unit 24 proceeds to step S35, and the type of fire that became the main factor of the fire determination in steps S32 to S33, that is, the smoldering fire and the oil fire, which are the highest values among the estimated fire values W to Z,. Control is performed so that either an electric fire or a chemical fire is displayed on the display of, for example, the fire receiver 12 as the main cause of the fire.

(Fire judgment based on the total weighting of fire estimates)
FIG. 9 is a flowchart showing another embodiment of the fire determination process by the determination unit of FIG. As shown in FIG. 9, the determination unit 24 reads the plurality of types of fire estimates W, X, Y, Z output from the plurality of types of fire detectors 20-1 to 20-4 in step S41, and subsequently. Proceeding to step S 42 , the fire estimates Wa, Xb, Yc, Zd are weighted by multiplying the plurality of types of fire estimates W, X, Y, Z by preset predetermined weights a, b, c, d. Ask for. The weights a, b, c, and d are set according to the likelihood of a fire in the monitoring area 14.

Subsequently, the determination unit 24 sets the fire likelihood L as the sum of the weighted fire estimates Wa, Xb, Yc, and Zd in step S43.
L = Wa + Xb + Yc + Zd
Ask as.

Subsequently, the determination unit 24 compares the fire likelihood L with the predetermined threshold value Lth in step S44, and if it determines that the threshold value is equal to or higher than the threshold value Lth (or exceeds the threshold value Lth), the determination unit 24 proceeds to step S45 and sends a fire determination signal to the fire receiver 12. Output and output a fire sign alarm, etc.

Subsequently, the determination unit 24 proceeds to step S46, and the type of fire that became the main factor of the fire determination in steps S42 to S44, that is, the smoldering fire and the oil fire, which are the highest values among the estimated fire values W to Z,. Control is performed so that either an electric fire or a chemical fire is displayed on the display of, for example, the fire receiver 12 as the main cause of the fire.

The weights a, b, c, and d may be set by the user. In this case, it is preferable to store the preset values according to the typical environment so that the user can reflect the settings only by selecting the environment to be installed.

for example,
{Environment: Boiler room a = 0.5, b = 2.0 , c = 0.5, d = 0.5}
{Environment: Coin laundry a = 2.0 , b = 0.5, c = 0.5, d = 0.5}
{Environment: Server room a = 0.5, b = 0.5, c = 2.0 , d = 0.5}
Etc. are stored as preset values.

In addition, the system is made to learn the cause of the fire according to the type of fire in advance as an image, the system identifies the cause of the fire based on the image of the site, and then the cause of the fire is included for each type of fire. The weights a, b, c, and d may be set depending on whether or not. It is more preferable that the preset values and the ones that can be corrected by the user after the system automatically sets the weights.

[Modification of the present invention]
(Learning function)
Fire detector shown in the above embodiments, although taking a case having a learning function of the multi-layered neural network as an example, the learning of multi-layer neural network, another computer equipment such as a server having a learning function It may be done using and the resulting trained multi-layer neural network may be mounted on a fire detector for use.

(Type of fire)
In the above embodiment, a case where a smoked fire detector, an oil fire detector, an electric fire detector and a chemical fire detector are provided in the fire judgment device is taken as an example, and at least two types of fire detectors are used. Should be provided.

Further, the plurality of types of fire detectors are not limited to smoked fire detectors, oil fire detectors, electric fire detectors and chemical fire detectors, and other types of fire detectors may also be used for monitoring. It may be a fire detector corresponding to the type of area, for example, the type of kitchen, office, boiler room, and the like.

(Arson monitoring)
The above embodiment takes fire monitoring in the monitoring area as an example, but in addition to this, a fire configured by a multi-layer neural network for arson monitoring performed by installing sensors such as a surveillance camera and a flame detector outdoors. A detector may be provided so that the fire detector can be learned from deep learning and the arson can be monitored.

(Feature extraction)
In the above embodiment, the image is input to the convolutional neural network to extract the features due to the fire, but the preprocessing for extracting the features such as contour and shading from the input image is performed without using the convolutional neural network. It is also possible to extract a predetermined feature and input the extracted image into a fully connected neural network functioning as a recognition unit to estimate whether it is a fire or a non-fire. This makes it possible to reduce the processing load of image feature extraction.

(About learning method)
The above embodiments have been learned by the back-propagation learning method of the multilayered neural network is not limited to this.

(Composite of image and sensor)
In the above embodiment, fire monitoring using images is taken as an example, but image data and sensor data may be handled in parallel as input information. For image data, for example, a black-and-white value per pixel is treated as an input term, and for sensor data, for example, a detection value for each sensor is treated as an input term. In this case, the term of the intermediate layer from which the features of the image are extracted in the intermediate layer and the term of the intermediate layer affected by the sensor data affect the terms of the intermediate layer after the next stage for determining fire detection. It is desirable as an educational result to be able to give, but it is not limited to this if fire monitoring can be enabled.

(Infrared lighting and infrared image imaging)
In the above embodiment, the surveillance area is imaged by the surveillance camera in the state of using the illumination of the surveillance area and / or in the state of natural light. Infrared image is taken by a surveillance camera, the multi-layer neural network of the fire detector is learned by back propagation, and the infrared image of the surveillance area is input to the trained multi-layer neural network to determine whether it is fire or non-fire. It may be judged.

By inputting the infrared image of the monitoring area into the fire detector in this way, it is possible to monitor the fire using the monitoring image without being affected by the lighting state of the monitoring area, the change in brightness during the day and night, and the like.

(others)
Further, the present invention is not limited to the above-described embodiment, includes appropriate modifications that do not impair its purpose and advantages, and is not further limited by the numerical values shown in the above-described embodiment.

10: Fire judgment device 12: Fire receiver 14: Monitoring area 16: Surveillance camera 18: Fire detector 20-1: Smoked fire detector 20-2: Oil fire detector 20-3: Electric fire detector 20- 4: Chemical fire detector 20-5: Non-fire detector 22: Signal cable 24: Judgment unit 26: Image input unit 28: Multilayer neural network 30: Learning image holding unit 32: Learning control unit 38: Feature extraction unit 40 : Recognition unit 42: Input images 43, 45a, 45b: Weight filters 44a, 44b, 44c: Feature map 46: Input layer 48: Fully coupled 50: Intermediate layer 52: Output layer

Claims (9)

In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
Multiple types of fire detectors learned by deep learning for each type of fire,
A fire determination unit that determines a fire based on a fire estimate output when the input information is input to the plurality of types of fire detectors.
Is provided,
The fire determination unit calculates a weighted fire estimated value by multiplying each of the fire estimated values by the plurality of types of fire detectors by a predetermined weight according to the type of fire, and any one of the weighted fire estimated values is used. A fire monitoring system characterized in that a fire is determined when a predetermined threshold is exceeded or exceeded.
In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
Multiple types of fire detectors learned by deep learning for each type of fire,
A fire determination unit that determines a fire based on a fire estimate output when the input information is input to the plurality of types of fire detectors.
Is provided,
The fire determination unit calculates the fire likelihood as the sum of the estimated fire values of the plurality of types of fire detectors, and determines that the fire is a fire when the fire likelihood is equal to or greater than or exceeds a predetermined threshold. Fire monitoring system.
In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
Multiple types of fire detectors learned by deep learning for each type of fire,
A fire determination unit that determines a fire based on a fire estimate output when the input information is input to the plurality of types of fire detectors.
Is provided,
The fire determination unit calculates the fire likelihood as the sum of multiplying each of the fire estimates by the plurality of types of fire detectors by a predetermined weight according to the type of fire, and the fire likelihood is equal to or greater than a predetermined threshold. Or, a fire monitoring system characterized in that a fire is determined when the value is exceeded.
In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
Multiple types of fire detectors learned by deep learning for each type of fire,
Non-fire detectors learned non-fire and
A fire estimation value that is a positive value output when the input information is input to the plurality of types of fire detectors and a non-fire estimation value that is a negative value output when the input information is input to the non-fire detectors. Fire judgment unit that judges fire based on the value,
Is provided,
The fire determination unit calculates the fire likelihood from the sum of the fire estimates by the plurality of types of fire detectors and the non-fire estimates by the non-fire detectors, and the fire likelihood exceeds or exceeds a predetermined threshold value. A fire monitoring system characterized in that a fire is determined in case of a fire.
In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
Multiple types of fire detectors learned by deep learning for each type of fire,
Non-fire detectors learned non-fire and
A fire estimation value that is a positive value output when the input information is input to the plurality of types of fire detectors and a non-fire estimation value that is a negative value output when the input information is input to the non-fire detectors. Fire judgment unit that judges fire based on the value,
Is provided,
The fire determination unit calculates the fire likelihood as the sum of the fire estimation value by the plurality of types of fire detectors and the non-fire estimation value by the non-fire detection unit multiplied by a predetermined weight according to the type of fire and the non-fire. A fire monitoring system for calculating and determining a fire when the fire likelihood exceeds or exceeds a predetermined threshold.
In the fire monitoring system according to claim 1 , 3 or 5.
A fire monitoring system characterized in that the weight according to the type of fire is set according to the environment of the monitoring area indicating the susceptibility to a fire.
In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
Multiple types of fire detectors learned by deep learning for each type of fire,
A fire determination unit that determines a fire based on a fire estimate output when the input information is input to the plurality of types of fire detectors.
Is provided,
As the above-mentioned multiple types of fire detectors
Smoked fire detectors that learned about smoked fires and
Oil fire detectors learned from oil fires and
An electric fire detector that learned about electric fires, and
Chemical fire detectors learned from chemical fires and
A fire monitoring system characterized by the fact that
In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
Multiple types of fire detectors learned by deep learning for each type of fire,
A fire determination unit that determines a fire based on a fire estimate output when the input information is input to the plurality of types of fire detectors.
Is provided,
The multi-layer neural network is composed of a feature extraction unit and a recognition unit.
The feature extraction unit, and the input information neural network convolution having a plurality of convolutions layers for generating feature information characteristic of the input information is extracted by inputting,
The recognition unit is a fire monitoring system characterized in that it is a fully connected neural network that outputs the estimated fire value by inputting the feature information output from the convolutional neural network.
In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
Multiple types of fire detectors learned by deep learning for each type of fire,
A fire determination unit that determines a fire based on a fire estimate output when the input information is input to the plurality of types of fire detectors.
Is provided,
The fire detector trains the multi-layer neural network by backpropagation based on an error between a value output when learning information is input to the multi-layer neural network and an expected value of a predetermined value. A featured fire monitoring system.

Images