JP6942029B2 - Fire monitoring system - Google Patents

Description

The present invention relates to a fire monitoring system that determines and warns a fire by a neural network from an image of a monitoring area captured by a surveillance camera.

Conventionally, a system for detecting a fire using a sensor that monitors a specific physical quantity such as a smoke detector or a heat detector has been put into practical use.

On the other hand, conventionally, various devices and systems have been proposed in which a fire is detected by performing image processing on an image of a surveillance area captured by a surveillance camera.

In such a fire monitoring system, early detection of a fire is important from the viewpoint of initial fire extinguishing and evacuation guidance.

For this reason, in the conventional device (Patent Document 1), as phenomena caused by smoke accompanying a fire from an image, a decrease in transmittance or contrast, a convergence of a brightness value to a specific value, and a narrowing of the brightness distribution range to disperse the brightness. The decrease in brightness, the change in the average brightness due to smoke, the decrease in the total amount of edges, and the increase in intensity in the low frequency band are derived, and these are comprehensively judged to enable smoke detection.

Japanese Unexamined Patent Publication No. 2008-046916 Japanese Unexamined Patent Publication No. 7-245757 JP-A-2010-238028 Japanese Unexamined Patent Publication No. 6-325270

However, a fire detection system using a sensor that monitors a specific physical quantity has a problem that even if a monitoring standard is satisfied due to a non-fire event, it is regarded as a fire and the fire cannot be correctly determined.

Further, in a conventional fire monitoring system that detects a fire from an image of smoke associated with a fire, the characteristics of smoke such as transparency, contrast, and edge in the smoke image are determined in advance, and the image captured by the surveillance camera is used. It is necessary to generate smoke characteristics by processing, and there are various situations of smoke generation due to fire, and it is extremely difficult to find out what kind of smoke characteristics there are, and it is a decisive factor. Since it is difficult to find the characteristic, a fire monitoring system that accurately judges smoke caused by a fire from a monitoring image and outputs a fire alarm is in the process of being put into practical use.

On the other hand, in recent years, for example, a large number of images of cats and dogs are labeled, and they are trained by a multi-layer neural network equipped with a convolutional neural network, so-called deep learning is performed, and new images are learned. A technique for presenting to a pre-existing multi-layer neural network and determining whether it is a cat or a dog is disclosed.

Further, deep learning is being studied to be used not only for image analysis but also for natural language processing and behavior analysis.

Such a multi-layered neural network is provided in a fire detector that determines a fire from the input information by using an image of a monitoring area captured by a surveillance camera as input information, and during learning, a large number of fires and non-fires occur. If the input information is prepared and trained by the multi-layered neural network, and the input information is input to the trained multi-layered neural network during monitoring, it is estimated with high accuracy whether it is a fire or not from the output and an alarm is issued. It becomes possible to construct a fire monitoring system that outputs.

By the way, the monitoring environment of a fire detector equipped with a multi-layer neural network is intended for places where it is difficult to monitor with a normal fire detector. For example, a special monitoring environment such as a boiler room, an electric room, or a server room is assumed. Will be done.

However, when a multi-layered neural network of fire detectors is learned from fire images in a general monitoring environment such as a facility or building, an oil fire that occurs in a special monitoring environment such as a boiler room, an electric room, or a server room. There is a risk that the accuracy of fire judgment by the fire detector will drop for electric fires and the like.

An object of the present invention is to provide a fire monitoring system capable of determining a fire with high sensitivity by using a fire detector provided with a multi-layer neural network for a fire that is likely to occur from a monitoring environment.

(Fire monitoring system of the first invention)
The present invention is a fire monitoring system that detects a fire based on the input information of the monitoring area by using a fire detector composed of a multi-layer neural network.
An environmental fire discriminator that determines the types of fires that are likely to occur from the monitoring environment in the monitoring area,
A learning control unit that learns a multi-layered neural network of a fire detector by deep learning based on the type of fire determined by the environmental fire discrimination unit.
A fire judgment unit that judges a fire by comparing the fire estimate output when the input information of the monitoring area is input to the trained fire detector with a predetermined threshold value, and
Is provided.

(Priority of environmental fire)
When multiple types of fires are discriminated from the monitoring environment in the monitoring area, the environmental fire discrimination unit sets priorities in the order of fires that are likely to occur.
The learning control unit learns a multi-layer neural network of a fire detector by deep learning by one or a plurality of types of fires having a high priority determined by the environmental fire discrimination unit.

(Fire monitoring system of the second invention)
In another embodiment of the present invention, in a fire monitoring system that detects a fire based on input information in a monitoring area by using a fire detector composed of a multi-layer neural network.
An environmental fire discriminator that determines the types of fires that are likely to occur from the monitoring environment in the monitoring area,
A plurality of types of fire detectors learned by deep learning for each type of fire by the learning control unit,
A fire judgment unit that determines a fire by comparing each of the fire estimates output when input information is input to a plurality of types of trained fire detectors with a predetermined threshold value corresponding to the type of fire.
A threshold setting unit that sets a threshold value corresponding to the type of fire in the fire determination unit so that the type of fire determined by the environmental fire determination unit can be easily determined.
Is provided.

(Setting the threshold value according to the type of fire that is likely to occur)
The threshold value setting unit sets a threshold value corresponding to the type of fire determined by the environmental fire discrimination unit to a value lower than the threshold value corresponding to other types of fire.

(Automatic judgment of fires that are likely to occur)
The environmental fire discrimination unit
An image analysis unit composed of a multi-layer neural network that analyzes the image of the monitoring area captured by the imaging unit and outputs the elements contained in the image as words.
The environment of the monitoring area is estimated by comparing the word of the element included in the image output from the image analysis unit with the word indicating the predetermined monitoring environment stored in the dictionary in advance, and it is generated corresponding to the estimated monitoring environment. A fire type determination unit that determines the types of fires that are easy to make, and a fire type determination unit
Is provided.

(Input information)
Fire detector outputs a fire estimate by inputting an image of a monitoring region captured by the physical quantity and or imaging unit, which is detected by the sensor as the input information.

(Monitoring environment and type of fire)
The monitoring environment includes a boiler room, an electric room, a server room, a cooking room, and a laboratory.
The types of fires determined by the environmental fire discrimination unit include smoked fires, oil fires, electric fires and chemical fires.

(Functional configuration of multi-layer neural network of fire detector)
The multi-layer neural network of the fire detector consists of a feature extraction unit and a recognition unit.
The feature extraction unit is a convolutional neural network having a plurality of convolutional layers that generate feature information from which the features of the input information are extracted by inputting the input information.
The recognition unit inputs the feature information output from the convolutional neural network.
By doing so, it becomes a fully connected neural network that outputs fire estimates.

(Learning by backpropagation)
The learning control unit uses backpropagation based on the error between the value output when learning information is input to the multi-layer neural network of the fire detector and the expected value of a predetermined value, and the multi-layer neural network of the fire detector. Train the network.

(Clarification of fire judgment grounds)
In addition to the fire judgment, the fire judgment unit displays the fire type on which the fire judgment is based.

(Basic effect of the first invention)
The present invention is a type of fire that is likely to occur from the monitoring environment of the monitoring area in a fire monitoring system that detects a fire based on the input information of the monitoring area by using a fire detector composed of a multi-layered neural network. The environmental fire discrimination unit that determines the number of fires, the learning control unit that learns the multi-layered neural network of the fire detector by deep learning due to the type of fire determined by the environmental fire discrimination unit, and the input information of the monitoring area have been learned. Since a fire judgment unit is provided to judge a fire by comparing the estimated fire value output when input to the fire detector with a predetermined threshold value, it occurs uniquely to the monitoring environment monitored by the fire detector. For easy fires, it is possible to judge fires with higher sensitivity, and it is possible to take appropriate measures by early detection of fires.

(Effect of priority of environmental fire)
Further, when the environmental fire discrimination unit discriminates a plurality of types of fires from the monitoring environment in the monitoring area, the priority is set in the order of the fires that are likely to occur, and the learning control unit has the priority determined by the environmental fire discrimination unit. Since the multi-layered neural network of the fire detector is learned by deep learning by one or more types of fires with a high degree of fire, flame fires such as oil fires and electric fires and smoldering fires can be seen from the monitoring environment. Multiple types of fires such as, etc. are assumed, but by setting the priority according to the likelihood of occurrence and learning the fire detector by one or more types of fires with high priority, the monitoring environment It is possible to judge a fire with higher sensitivity for a fire that is unique to the fire.

(Basic effect of the second invention)
In another embodiment of the present invention, in a fire monitoring system that detects a fire based on the input information of the monitoring area by using a fire detector composed of a multi-layered neural network, the monitoring environment of the monitoring area. and determining environmental fire discriminating section types prone fire from a plurality of types of fire detectors learned by deep learning for each type of fire by the learning control unit, learned plurality of types of fire detecting input information A fire judgment unit that judges a fire by comparing each of the estimated fire values output when input to the device with a predetermined threshold value corresponding to the type of fire, and a threshold value corresponding to the type of fire in the fire judgment unit. Since a threshold setting unit is provided to make it easier to determine the type of fire determined by the environmental fire determination unit, the threshold for determining a fire from the estimated fire values output by multiple types of fire detectors is monitored. By setting the threshold value to determine the fire that is likely to occur peculiar to the environment, it is possible to judge the fire that is likely to occur peculiar to the monitoring environment with higher sensitivity.

(Effect of setting a threshold value corresponding to the type of fire that is likely to occur)
In addition, the threshold value setting unit sets the threshold value corresponding to the type of fire determined by the environmental fire discrimination unit to a lower value than the threshold value corresponding to other types of fire, so that the threshold value is set to a low value. It becomes easier to judge the set type of fire estimate as a fire, and it is possible to judge a fire with higher sensitivity for a fire that is likely to occur peculiar to the monitoring environment.

(Effect of automatic judgment of fires that are likely to occur)
In addition, the environmental fire discrimination unit is composed of an image analysis unit composed of a multi-layered neural network that analyzes the image of the monitoring area captured by the imaging unit and outputs the elements contained in the image as words, and the image analysis unit. The environment of the monitoring area is estimated by comparing the word of the element included in the output image with the word indicating the predetermined monitoring environment stored in advance in the dictionary, and the type of fire that is likely to occur corresponding to the estimated monitoring environment. Since a fire type determination unit is provided to determine the type of fire, by analyzing the surveillance image captured by the surveillance camera, which is the imaging unit, the characteristics of one or more objects existing in the surveillance environment are extracted and converted into words. It is possible to automatically determine the type of fire that is likely to occur by comparing the extracted words with a predetermined word indicating the monitoring environment of the dictionary and estimating that it is the monitoring environment of the monitoring area when they match or are similar. ..

(Effect of input information)
In addition, since the plurality of fire detectors output the fire estimation value by inputting the physical quantity detected by the sensor and / or the image of the monitoring area captured by the imaging unit as input information , the fire detector was detected by the sensor. By inputting physical quantities and images of the monitoring area captured by the imaging unit into a plurality of types of trained fire detectors, it is possible to determine a fire with high accuracy.

(Effects of monitoring environment and type of fire)
The monitoring environment includes a boiler room, an electric room, a server room, a cooking room, and a laboratory, and the types of fires determined by the environmental fire discrimination unit include smoldering fires, oil fires, electric fires, and chemical fires. Therefore, for example, in the case of a boiler room as a monitoring environment, an oil fire is determined as a fire that is likely to occur, and it is possible to determine a fire with higher sensitivity to an oil fire, and in the case of an electric room or a server room. Electric fires are determined as fires that are likely to occur, and it is possible to determine fires that are more sensitive to electric fires.

(Effect of functional configuration of multi-layer neural network of fire detector)
In addition, the multi-layer neural network of the fire detector is composed of a feature extraction unit and a recognition unit, and the feature extraction unit generates feature information from which the features of the input information are extracted by inputting the input information of the monitoring area. a neural network convolution having a plurality of convolutions layer, recognition unit, because the set as the total binding neural network for outputting a fire estimate by inputting the feature information output from the convolutional neural network convolution neural network By automatically extracting the features, the features of the fire input information by preprocessing from the input information of the monitoring area, for example, in the image, input without the need for preprocessing such as extracting the contour etc. The characteristics of the information are extracted, and the subsequent recognition unit makes it possible to estimate the fire with high accuracy.

(Effect of learning by backpropagation)
The learning control unit uses backpropagation (backpropagation of errors) based on the error between the value output when learning information is input to the multi-layer neural network of the fire detector and the expected value of a predetermined value. Since the multi-layered neural network of the above is trained, when a large number of input information prepared for each type of fire is input as learning information, the estimated value of the fire is given as the expected value of the output and backpropagation processing is performed. The weights and biases in the multi-layer neural network are learned so as to minimize the error between the output value and the expected value, and the fire can be estimated with higher accuracy from the input information and an alarm can be made.

(Effect of clarifying the grounds for fire judgment)
In addition to the fire judgment, the fire judgment unit displays the type of fire on which the fire judgment is based. Therefore, by knowing the type of fire that has occurred, fire extinguishing activities and evacuation instructions are given according to the type of fire. Make it possible.

Explanatory drawing which showed the 1st Embodiment of a fire monitoring system Explanatory drawing which showed functional structure of fire detector and learning control part of FIG. Explanatory diagram showing the functional configuration of the multi-layer neural network shown in FIG. Flow chart showing environmental fire monitoring control according to the embodiment of FIG. Explanatory drawing which showed the functional structure of the environmental fire discrimination part of FIG. Explanatory drawing which showed functional structure of convolutional neural network and recurrent neural network provided in image analysis part of FIG. Explanatory drawing which showed the 2nd Embodiment of a fire monitoring system Explanatory drawing which showed functional structure of fire detector and learning control part of FIG. Explanatory drawing which showed the functional structure of the fire judgment part of FIG. 7 together with a fire detector, an environmental fire judgment part and a threshold setting part. Flow chart showing environmental fire monitoring control according to the embodiment of FIG.

[First Embodiment of Fire Monitoring System]
FIG. 1 is an explanatory diagram showing a first embodiment of a fire monitoring system. As shown in FIG. 1, a surveillance camera 16 that functions as an imaging unit is installed in the monitoring area 14 of a facility such as a building, and the monitoring camera 16 captures a moving image of the monitoring area 14. The surveillance camera 16 captures an RGB color image at, for example, 30 frames / second and outputs it as a moving image. Further, one frame has, for example, a pixel arrangement of 4056 × 4056 pixels in length and width.

In addition, on-off type fire detectors 18 are installed in each of the monitoring areas 14, detects the temperature or smoke concentration due to the fire, issues a report when the predetermined threshold level is exceeded, and outputs a fire alarm signal. I try to do it.

As described above, the monitoring area 14 to be monitored by the present embodiment includes a special monitoring environment such as a boiler room, an electric room, a server room, a cooking room, and a laboratory.

A fire determination device 10 and a fire receiver 12 of a fire alarm system are installed in the disaster prevention monitoring center and the manager's room of the facility with respect to the monitoring area 14. The fire determination device 10 and the fire receiver 12 may be integrated. A surveillance camera 16 installed in the surveillance area 14 is connected to the fire determination device 10 by a signal cable 22, and a moving image image captured by the surveillance camera 16 is input.

A sensor line 25 is pulled out from the fire receiver 12 to the monitoring area 14, and a fire detector 18 is connected to each of the sensor lines 25.

The fire determination device 10 is provided with a fire detector 20, an environmental fire determination unit 24, a learning control unit 26, and a fire determination unit 28. The fire detector 20 includes a multi-layer neural network, inputs an image of the surveillance area 14 captured by the surveillance camera 16, and outputs a fire estimate value.

The environmental fire discrimination unit 24 inputs the image of the monitoring area 14 captured by the monitoring camera 16, estimates the monitoring environment of the monitoring area 16 such as the boiler room, the electric room, and the server room, and estimates the boiler room, the electric room, and the estimated environment. Determine the types of fires that are likely to occur from the monitoring environment such as the server room, such as oil fires and electric fires.

The learning control unit 26 learns the multi-layered neural network of the fire detector 20 by deep learning due to the type of fire determined by the environmental fire determination unit 24. For example, when an oil fire in the boiler room is determined by the environmental fire discrimination unit 24, a multi-layered neural network is learned from the image of the oil fire, and the oil fire peculiar to the boiler room as a monitoring environment is more sensitive. Enables fire judgment.

Further, when a plurality of types of fires are discriminated from the monitoring environment of the monitoring area 14, the environmental fire discrimination unit 24 sets the priority in the order of the fires that are likely to occur. In this case, the learning control unit 26 learns the multi-layered neural network of the fire detector 20 by deep learning by one or a plurality of types of fires having high priority determined by the environmental fire determination unit 24.

This corresponds to the fact that multiple types of fires may easily occur depending on the monitoring environment. For example, in a server room, etc., in addition to the electric fire caused by the server machine stored in the server rack, there is a possibility that a smoldering fire may occur because, for example, an instruction manual used for operation management is stored. In this case, the environmental fire discrimination unit 24 has the determined electric fire as the first priority, the determined smoked fire has the second priority, and the learning control unit 26 has the electric fire and the smoked fire. By both, the multi-layered neural network of the fire detector 20 is trained by deep learning.

Therefore, the fire detector 20 can determine a fire with high sensitivity for both an electric fire and a smoked fire that are likely to occur from the monitoring environment.

[Fire detector]
FIG. 2 is an explanatory diagram showing a functional configuration of a fire detector and a learning control unit using the multi-layer neural network shown in FIG.

As shown in FIG. 2, the fire detector 20 is composed of an image input unit 30 and a multi-layer neural network 32, and the learning controller 26 is composed of a learning image storage unit 34 and a control unit 36. It is realized by executing a program by the CPU of a computer circuit corresponding to the processing of the neural network.

The fire detector 20 inputs an image of the monitoring area captured by the surveillance camera 16 to the multilayer neural network 32 via the image input unit 30, and outputs a fire estimation value.

The learning control unit 26 learns a learning image corresponding to the type of fire that is likely to occur in the monitoring environment determined by the environmental fire determination unit 24, for example, in the case of an oil fire, a learning image of the oil fire image in frame units from the video of the oil fire. It is read from the storage unit 34 and input to the multi-layer neural network 32 as a supervised fire image via the control unit 36. And learn bias.

When the image of the fire in the monitoring area 14 such as the boiler room captured by the surveillance camera 16 is input to the multi-layer neural network 32 that has been trained using the image of the oil fire with a teacher, the image of the oil fire is displayed. The corresponding fire estimate W is output. The fire estimate W output from the fire detector 20 is 1 or close to 1 in the case of the same oil fire image used for learning, and the oil fire peculiar to the monitoring environment is more sensitive. Can be determined by.

[Multi-layer neural network]
FIG. 3 is an explanatory diagram showing the functional configuration of the multi-layer neural network shown in FIG. 2, the outline is shown in FIG. 3 (A), and the details are schematically shown in FIG. 3 (B).

As shown in FIG. 3A, the multi-layer neural network 32 of this embodiment is composed of a feature extraction unit 38 and a recognition unit 40. The feature extraction unit 38 is a convolutional neural network, and the recognition unit 40 is a fully connected neural network.

The multi-layer neural network 32 is a neural network that performs deep learning, is a neural network that has a deep layer that connects a plurality of intermediate layers, and performs expression learning that is feature extraction.

A normal neural network requires work by artificial trial and error to extract features for determining a fire from an image, but in a multi-layer neural network 32, a convolutional neural network is used as a feature extraction unit 38. Then, the pixel value of the image is input, the optimum feature is extracted by learning, and the optimum feature is input to the fully connected neural network of the recognition unit 40 to identify whether it is a fire or a non-fire.

As schematically shown in FIG. 3B, the fully connected neural network of the recognition unit 40 is composed of an input layer 46 , an intermediate layer 50, a repetition of the fully connected 48, and an output layer 52.

(Convolutional neural network)
FIG. 3B schematically shows the structure of the convolutional neural network constituting the feature extraction unit 38.

Convolutional neural networks have slightly different characteristics from ordinary neural networks and incorporate biological structures from the visual cortex. The visual cortex contains a receptive field, which is a collection of small cells that are sensitive to a small area of the visual field, and the behavior of the receptive field can be imitated by learning weighting in the form of a matrix. This matrix is called the weight filter (kernel) and is sensitive to similar subregions of an image, similar to the biological role played by receptive fields.

The convolutional neural network can express the similarity between the weight filter and the subarea by the convolution operation, and through this operation, the appropriate feature of the image can be extracted.

As shown in FIG. 3B, the convolutional neural network first performs convolution processing on the input image 42 by the weight filter 43. For example, the weight filter 43 is a matrix filter having a predetermined weighting of 3 × 3 in the vertical and horizontal directions, and by performing a convolution calculation while aligning the filter center with each pixel of the input image 42, 9 pixels of the input image 42 can be obtained. convolution to one pixel of feature map 44a as the subregions, a plurality of feature maps 44a is generated.

Subsequently, the pooling calculation is performed on the feature map 44a obtained by the convolution calculation. The pooling operation is a process of removing features unnecessary for identification and extracting features necessary for identification.

Subsequently, the convolution calculation and the pooling calculation using the weight filters 45a and 45b are repeated in multiple stages to obtain the feature maps 44b and 44c, and the feature map 44c of the last layer is input to the recognition unit 40 to perform a normal full coupling. A recognition unit 40 using a neural network estimates whether it is a fire or a non-fire.

Note that the pooling calculation in the convolutional neural network does not perform the pooling calculation because the features unnecessary for distinguishing between fire and non-fire are not always clear and the necessary features may be deleted. You may do so.

[Learning of multi-layer neural network]
(Backpropagation)
A neural network composed of an input layer, a plurality of intermediate layers, and an output layer is provided with a plurality of units in each layer and connected to a plurality of units in other layers, and weights and bias values are set for each unit. The vector product of multiple input values and weights is calculated, the bias values are added to obtain the sum, and this is passed through a predetermined activation function and output to the unit of the next layer. Forward propagation is performed in which the value propagates until it reaches.

To change the weights and biases of such neural networks, a learning algorithm known as backpropagation is used. In backpropagation, supervised learning when a data set of input value x and expected output value (expected value) y is given to the network, and unsupervised learning when only input value x is given to the network. In this embodiment, supervised learning is performed.

When performing backpropagation in supervised learning, for example, a function of mean square error is used as an error to compare the estimated value y * and the expected value y, which are the results of forward propagation that has passed through the network. do.

In backpropagation, the magnitude of the error between the estimated value y * and the expected value y is used to propagate the value from the rear to the front of the network while correcting the weight and bias. The corrected amount for each weight and bias is treated as a contribution to the error, calculated by the steepest descent method, and the value of the error function is minimized by changing the weight and bias values.

The procedure for learning by backpropagation for a neural network is as follows.
(1) The input value x is input to the neural network, forward propagation is performed, and the estimated value y * is obtained.
(2) Calculate the error with an error function based on the estimated value y * and the expected value y.
(3) Backpropagation is performed on the network while updating the weight and bias.

This procedure is repeated using different combinations of input values x and expected values y to minimize the value of the error function until the error between the weight and bias of the neural network is minimized as much as possible.

[Fire monitoring control]
FIG. 4 is a flowchart showing the fire monitoring control according to the embodiment of FIG. As shown in FIG. 4, the environmental fire discrimination unit 24 estimates that it is, for example, a boiler room by inputting an image of the monitoring area 14 captured by the monitoring camera 16 in step S1 and analyzing the image, followed by a step. Proceeding to S2, an oil fire is determined as an estimated fire that is likely to occur in the boiler room, and is output to the learning control unit 26.

The learning control unit 26 reads a learning image corresponding to the oil fire from the learning image storage unit 34 as shown in FIG. 2 based on the type of fire output from the environmental fire determination unit 24 in step S3, for example, an oil fire. The multi-layer neural network 32 of the fire detector 20 is trained by the learning image of the oil fire.

The fire detector 20 that has been learned in step S3 inputs an image of the monitoring area 14 captured by the surveillance camera 16 in step S4, and outputs a fire estimation value. Fire determination unit 28 reads a fire estimate of the fire detector 20 in step S5, and compared with a predetermined threshold value in step S6, the event of a fire estimate is equal to or greater than a predetermined threshold value (or exceeds the threshold), the step S 7 Proceed, output a fire determination signal to the fire receiver 12, and output a fire sign alarm or the like.

Then fire determination section 28 proceeds to step S 8, environmental fire discriminating section 24 by the fire type is determined as a fire-prone from the monitoring environment, such as oil fires the basis as for example the fire receiver 12 of the fire determination display Controls the display on the top. By displaying the type of fire that is the basis of the fire judgment in this way, fire extinguishing activities and evacuation guidance corresponding to the type of fire such as oil fire, electric fire, chemical fire or smoldering fire can be appropriately performed. It will be possible.

[Automatic judgment of environmental fire]
(Functional configuration of environmental fire discrimination unit)
FIG. 5 is an explanatory diagram showing a functional configuration of the environmental fire discrimination unit of FIG. As shown in FIG. 5, the environmental fire determination unit 24 includes an image input unit 54, a learning data set storage unit 56, an image analysis unit 60, and a fire type determination unit 62. The image analysis unit 60 includes a convolutional neural network 64 and a recurrent neural network 66 as a multi-layer neural network, and also includes a learning control unit 68 for learning the convolutional neural network 64 and the recurrent neural network 66.

The fire type determination unit 62 includes a determination device 70 and a thesaurus dictionary 72. In the thesaurus dictionary 72, words for identifying the monitoring environment are systematically organized and memorized by dividing them into major categories and minor categories for each environment name, and are generated corresponding to environment names such as boiler room and electric room. The types of fires that are easy to do are remembered.

Here, the function of the environmental fire determination unit 24 is realized by executing a program by the CPU of the computer circuit corresponding to the processing of the multi-layer neural network.

The image input unit 54 inputs the image of the monitoring area 14 captured by the surveillance camera 16 and outputs it to the image analysis unit 60. The convolutional neural network 64 provided in the image analysis unit 60 extracts and outputs the feature amount of the input monitoring image. The recurrent neural network 66 inputs the feature amount output from the convolutional neural network 64 , generates and outputs an image explanatory text explaining the outline of the input image.

The determination device 70 of the fire type determination unit 62 determines one or a plurality of words constituting the image description output from the recursive neural network 66 of the image analysis unit 60, and a plurality of environment determinations stored in the thesaurus dictionary 72. When the word in the image description matches or is similar to the environment judgment word in the thesaurus dictionary 72 by comparing with the word, the environment name of the monitoring area 14 such as the boiler room or the electric room is judged, and the environment name corresponding to the judged environment name is judged. Determines the type of fire that is likely to occur and outputs it.

The convolutional neural network 64 and the recursive neural network 66 provided in the image analysis unit 60 use a large number of training data sets consisting of a pair of a training image stored in advance in the training data set storage unit 56 and an image description thereof. It is learned by the learning control unit 68.

As the learning image stored in the learning data set storage unit 56, for example, a large number of monitoring images in a monitoring environment such as a boiler room or an electric room captured by a monitoring camera in a normal monitoring state are stored as learning images.

Further, in the learning data set storage unit 56, image explanatory texts are prepared corresponding to a large number of stored learning images in the monitoring environment, and are stored as a large number of learning data sets composed of a pair of the learning image and the image explanatory text. ing. For example, for the learning image of the boiler room, an image description such as "Boiler, valve, and piping are arranged." Is stored.

The learning control unit 68 prepares a convolutional neural network (not shown) for learning composed of an input layer, a plurality of intermediate layers, and a fully connected output layer, and is first stored in the learning data set storage unit 56. A large number of learning images are read out, input to a convolutional neural network for learning as a learning image without a teacher, and trained by a backpropagation method (backpropagation method).

Subsequently, the learning control unit 68, the resulting weights and bias convolution neural network for learning, is set as the learned neural network 64 convolutional no output layer, learning trained convolutional neural network 64 An image is input to extract a feature amount, and an image description set with the extracted feature amount and the input learning image is input to the recursive neural network 66 as unsupervised learning information, and a backpropagation method (backpropagation method) Learn by backpropagation method).

[Multi-layer neural network of image analysis unit]
FIG. 6 is an explanatory diagram showing a functional configuration of a convolutional neural network and a recurrent neural network provided in the image analysis unit of FIG.

(Convolutional neural network)
As shown in FIG. 6, the convolutional neural network 64 is composed of an input layer 74 and a plurality of intermediate layers 76. In a normal convolutional neural network, after the last intermediate layer 76, a multi-layer neural network that estimates the output from the features of the image by fully connecting the input layer, a plurality of intermediate layers, and the output layer is provided. Since it is only necessary to extract the features of the input image as the form, the multi-layer neural network of the full connection in the subsequent stage is not provided.

As shown in FIG. 3, the convolutional neural network 64 has slightly different characteristics from the normal neural network, and incorporates a biological structure from the visual cortex. The visual cortex contains a receptive field, which is a collection of small cells that are sensitive to a small area of the visual field, and the behavior of the receptive field can be imitated by learning weighting in the form of a matrix. This matrix is called the weight filter (kernel) and is sensitive to similar subregions of an image, similar to the biological role played by receptive fields.

The convolutional neural network 64 can express the similarity between the weight filter and the subarea by a convolutional operation, and through this operation, an appropriate feature of the image can be extracted.

The convolutional neural network 64 performs a convolution process on the input image input to the input layer 74 by a weight filter. For example, the weight filter is a matrix filter with a predetermined weighting of 3 × 3 in the vertical and horizontal directions, and by performing a convolution operation while aligning the center of the filter with each pixel of the input image, 9 pixels of the input image are placed in the next intermediate position. The feature map is generated in the intermediate layer 76 by convolving into one pixel of the feature map which is a small area of the layer 76.

Subsequently, the pooling calculation is performed on the feature map of the intermediate layer 76 obtained by the convolution calculation. The pooling operation is a process of removing features unnecessary for identification and extracting features necessary for identification.

Subsequently, by repeating the convolution operation and the pooling operation using the weight filter for each intermediate layer 76, a feature map is generated up to the last intermediate layer 76, and in the present embodiment, any intermediate layer 76 is generated. The generated feature map is input to the recurrent neural network 66 as the feature amount of the input image.

The convolutional neural network 64 inputs a learning image stored in the learning data set storage unit 56 by the learning control unit 68 shown in FIG. 5 to perform unsupervised learning, and by this learning, a similar image is obtained. Images with clustered features can be generated for grouping.

(Recurrent neural network)
The recurrent neural network 66 shown in FIG. 6 inputs the feature amount of the image extracted by the convolutional neural network 64 together with the word vector to predict the image description.

The recurrent neural network 66 of the present embodiment uses LSTM-LM (Long Short-Term Memory-Language Model), which is a deep learning model for time-series data.

A model of a normal recurrent neural network is composed of an input layer, a hidden layer, and an output layer, and is a model that performs time series analysis using the past history by using the information of the hidden layer as the input of the next time. On the other hand, the LSTM model calculates the probability that each word is selected as the t-th word from the t-1 words that are the past context. That is, the LSTM model inputs three pieces of information, that is, the word information of time 1 to t-1 that is hidden one time ago, the word of time t-1 that is the prediction result one time ago, and the external information, and sequentially. The sentence is generated by repeating the prediction of the next word.

The recurrent neural network 66 of FIG. 6 converts the feature vector of the image extracted by the convolutional neural network 64 into a matrix to be input to the LSTM hidden layer 78, the LSTM input layer 77, and the words stored in the word register 80 in word units. Word vector conversion unit 82 that converts S0 to Sn1 to vectors WeS0 to WeSN1, LSTM hidden layer 78 of N-1 stage, probability conversion unit 84 that converts the output of LSTM hidden layer 78 to appearance probabilities p1 to pN, and outputs words. It is composed of a cost calculation unit 86 that minimizes the cost by calculating from the cost functions logP1 ( S1) to logPN (SN) from the probability of performing.

(Learning of recurrent neural network)
The learning targets of the recurrent neural network 66 are the word vector conversion unit 82 and the LSTM hidden layer 78, and the features from the convolutional neural network 64 use the learned parameters as they are.

The learning data is a learning image I and a word string {St} (t = 0, ... N) of the image description sentence, and the following procedure is performed.
(1) The learning image I is input to the convolutional neural network 64, and the output of the specific intermediate layer 76 is taken out as a feature vector.
(2) The feature vector is input from the LSTM input layer 77 to the LSTM hidden layer 78.
(3) The word string St is input in order from t = 0 to t = N-1, and the probability pt + 1 is obtained in each step.
(4) Minimize the cost obtained from the probability pt + 1 (St + 1) of outputting the word St + 1.

(Generation of image description)
When generating an image description using the trained convolutional neural network 64 and the recurrent neural network 66, the feature vector generated by inputting the image into the convolutional neural network 64 is converted into the recurrent neural network 66. Input and arrange the word strings in descending order of the product of the appearance probability of the words to generate the image description. The procedure is as follows:
(1) The image is input to the convolutional neural network 64, and the output of the specific intermediate layer 76 is extracted as a feature vector.
(2) The feature vector is input from the LSTM input layer 77 to the LSTM hidden layer 78.
(3) The start symbol <S> of the sentence is converted into a vector by using the vector conversion unit 82, and is input to the LSTM hidden layer 78.
(4) Since the probability of appearance of words can be known from the output of the LSTM hidden layer 78, the top M words (for example, M = 20) are selected.
(5) The word output in the previous step is converted into a vector using the vector conversion unit 82, and is input to the LSTM hidden layer 78.
(6) From the output of the LSTM hidden layer 78, the product of the probabilities of the words output so far is obtained, and the upper M word strings are selected.
(7) The processes of (5) and (6) above are repeated until the output of the word becomes a terminal symbol.

In this way, the environmental fire discrimination unit 24 analyzes the surveillance image captured by the surveillance camera to estimate the type of fire that is likely to occur as the surveillance environment of the surveillance area 14, for example, an oil fire for the estimation of the boiler room. The type of fire that was automatically determined by automatically performing processing such as determining an electric fire based on the estimation of the electric room or server room, and further determining a chemical fire based on the estimation of the laboratory. Is output to the learning control unit 24 to learn the fire detector 20, so that it is possible to determine a fire with higher sensitivity to a type of fire that is likely to occur from the monitoring environment.

[Second Embodiment of Fire Monitoring System]
FIG. 7 is an explanatory diagram showing a second embodiment of the fire monitoring system, FIG. 8 is an explanatory diagram showing the functional configurations of the fire detector and the learning control unit of FIG. 7, and FIG. 9 is a function of the fire determination unit of FIG. It is explanatory drawing which showed the structure together with the fire detector, the environmental fire discrimination part and the threshold value setting part.

As shown in FIG. 7, in the fire monitoring system of the present embodiment, the surveillance camera 16 is installed in the monitoring area 14 as in the first embodiment of FIG. 1, and the moving image image captured by the surveillance camera 16 is a fire determination device. A fire detector 18 is installed in the monitoring area 14 which is input to the 10 and is connected to the detector line 25 drawn from the fire receiver 12.

The fire determination device 10 is provided with an oil fire detector 20-1, an electric fire detector 20-2, a chemical fire detector 20-3, and a smoked fire detector 20-4 as a plurality of types of fire detectors. As represented by the oil fire detector 20-1 of FIG. 8, an image input unit 30 and a multi-layer neural network 32 are provided. In the following description, the oil fire detector 20-1, the electric fire detector 20-2, the chemical fire detector 20-3, and the smoked fire detector 20-4 are referred to as a plurality of types of fire detectors 20-1 to 20-1. It may be called 20-4.

The signal cable 22 from the surveillance camera 16 is branched and input to the oil fire detector 20-1, the electric fire detector 20-2, the chemical fire detector 20-3, and the smoked fire detector 20-4 for monitoring. The moving image sent from the camera 16 is input in frame units, and the oil fire estimated value W, the electric fire estimated value X, the chemical fire estimated value Y, and the smoked fire estimated value Z are output to the fire determination unit 28.

Further, the fire determination device 10 is provided with an environmental fire determination unit 24, a threshold value setting unit 27, and a fire determination unit 28.

The environmental fire determination unit 24 is the same as the embodiment shown in FIG. 1, and inputs the image of the monitoring area 14 captured by the monitoring camera 16 to estimate the monitoring environment of the monitoring area 16 such as the boiler room, the electric room, and the server room. At the same time, the types of fires that are likely to occur from the monitoring environment such as the boiler room, electric room, and server room estimated, such as oil fire and electric fire, are determined. The automatic determination of a type of fire that is likely to occur from the environment by the environmental fire determination unit 24 is the same as that shown in FIGS. 5 and 6.

As shown in FIG. 8, the learning control unit 26 includes a learning image storage unit 34 and a control unit 36, learns a multi-layer neural network 32 of the oil fire detector 20-1 from an oil fire image, and detects an electric fire. the multilayered neural network of the vessels 20-2 learned by electrical fires image, a multi-layer neural network of chemical fire detectors 20-3 learned by chemical fire image, further multilayered neural of smoldering fire detector 20-4 The network is learned from smoked fire images.

As shown in FIG. 9, the fire determination unit 28 is composed of the comparators 90-1, 90-2, 90-3, 90-4 and the OR circuit 92, and the comparators 90-1, 90-2, 90- Oil fire estimates output from oil fire detector 20-1, electric fire detector 20-2, chemical fire detector 20-3, and smoked fire detector 20-4 to one of the inputs 3, 90-4. W, an estimated electric fire value X, an estimated chemical fire value Y, and a smoked fire detection value Z are input.

Further, for the other input of the comparators 90-1, 90-2, 90-3, 90-4, a predetermined threshold value corresponding to the type of fire set by the threshold value setting unit 27, for example, an oil fire threshold value Wth, electricity The fire threshold value Xth, the chemical fire threshold value Yth, and the smoldering fire detection value Zth are input.

The comparators 90-1, 90-2, 90-3, 90-4 are fire determination signals when the estimated fire values W, X, Y, Z are equal to or more than or exceed the corresponding thresholds Wth, Xth, Yth, Zth. Is output to the fire receiver 12 via the OR circuit 92 to output a fire sign alarm or the like.

The threshold value setting unit 27 sets the threshold value corresponding to the type of fire determined by the environmental fire determination unit 24 to a value lower than the threshold value corresponding to the other types of fire. When, for example, a boiler room is estimated as a monitoring environment by the environmental fire determination unit 24, and an oil fire is determined as a fire that is likely to occur from the boiler room, the threshold setting unit 27 sets the oil fire threshold Wth, the electric fire threshold Xth, and chemicals. As the fire threshold Yth and the smoldering fire detection value Zth, for example, the oil fire threshold Wth = 0.6
Electric fire threshold Xth = 0.9
Chemical fire threshold Yth = 0.9
Smoked fire threshold Zth = 0.9
Set to.

Therefore, it becomes easy to determine an oil fire set to a low threshold value Wth = 0.6 as a fire, and it is possible to determine a fire with higher sensitivity to a fire that is likely to occur peculiar to the monitoring environment.

In order not to perform a specific type of fire determination, the threshold value corresponding to this may be set to a high value of 1 or more.

[Fire monitoring control]
FIG. 10 is a flowchart showing the fire monitoring control according to the embodiment of FIG. As shown in FIG. 10, the environmental fire discrimination unit 24 estimates that it is, for example, a boiler room by inputting an image of the monitoring area 14 captured by the monitoring camera 16 in step S11 and analyzing the image, followed by a step. Proceeding to S12, an oil fire is determined as an estimated fire that is likely to occur in the boiler room, and is output to the threshold setting unit 27.

The threshold setting unit 27 is based on the type of fire output from the environmental fire determination unit 24 in step S13, for example, an oil fire, and the fire estimation value W from the oil fire detector 20-1 of the fire determination unit 28 shown in FIG. Set the threshold Wth for the comparator 90-1 in which the above is input to a value lower than the thresholds Xth, Yth, Zth for the comparators 90-2 to 90-4 in which the other fire estimation values X, Y, Z are input. However, for oil fires that are likely to occur from the boiler room, it is possible to judge the fire with higher sensitivity.

Subsequently, the image of the surveillance area 14 captured by the surveillance camera 16 in step S14 is input to a plurality of types of fire detectors 20-1 to 20-4, and the fire estimation values W, X, Y, and Z are output. The fire determination unit 28 reads the estimated fire values W, X, Y, Z from a plurality of types of fire detectors 20-1 to 20-4 in step S15, and sets the predetermined threshold values Wth, Xth, Yth, Zth in step S16. In comparison, if any of the estimated fire values is equal to or higher than the corresponding threshold value (or exceeds the threshold value), the process proceeds to step S17, a fire determination signal is output to the fire receiver 12, and a fire sign alarm or the like is output.

Subsequently, the fire determination unit 28 proceeds to step S18 and controls to display the fire type determined as a fire, for example, an oil fire, on the display of, for example, the fire receiver 12. By displaying the type of fire that is the basis of the fire judgment in this way, fire extinguishing activities and evacuation guidance corresponding to the type of fire such as oil fire, electric fire, chemical fire or smoldering fire can be appropriately performed. Is possible.

[Modification of the present invention]
(Learning function)
Fire detector shown in the above embodiments, although taking a case having a learning function of the multi-layered neural network as an example, the learning of multi-layer neural network, another computer equipment such as a server having a learning function It may be done using and the resulting trained multi-layer neutral network may be mounted on a fire detector for use.

(Monitoring environment and fire type)
In the above embodiment, a boiler room, an electric room, a server room, a laboratory, or the like is taken as an example of a monitoring environment, but other appropriate places and uses may be used. Further, in the above embodiment, oil fires, electric fires, chemical fires, and smoked fires are taken as examples of fires that are likely to occur from the monitoring environment, but other appropriate types of fires are included.

(Arson monitoring)
The above embodiment takes fire monitoring in the monitoring area as an example, but in addition to this, a fire configured by a multi-layer neural network for arson monitoring performed by installing sensors such as a surveillance camera and a flame detector outdoors. A detector may be provided so that the fire detector can be learned from deep learning and the arson can be monitored.

(Feature extraction)
In the above embodiment, the image is input to the convolutional neural network to extract the features due to the fire, but the preprocessing for extracting the features such as contour and shading from the input image is performed without using the convolutional neural network. It is also possible to extract a predetermined feature and input the extracted image into a fully connected neural network functioning as a recognition unit to estimate whether it is a fire or a non-fire. This makes it possible to reduce the processing load of image feature extraction.

(About learning method)
The above embodiments have been learned by the back-propagation learning method of the multilayered neural network is not limited to this.

(Composite of image and sensor)
In the above embodiment, fire monitoring using images is taken as an example, but image data and sensor data may be handled in parallel as input information. For image data, for example, a black-and-white value per pixel is treated as an input term, and for sensor data, for example, a detection value for each sensor is treated as an input term. In this case, the term of the intermediate layer from which the features of the image are extracted in the intermediate layer and the term of the intermediate layer affected by the sensor data affect the terms of the intermediate layer after the next stage for determining fire detection. It is desirable as an educational result to be able to give, but it is not limited to this if fire monitoring can be enabled.

(Infrared lighting and infrared image imaging)
In the above embodiment, the surveillance area is imaged by the surveillance camera in the state of using the illumination of the surveillance area and / or in the state of natural light. Infrared image is taken by a surveillance camera, the multi-layer neural network of the fire detector is learned by back propagation, and the infrared image of the surveillance area is input to the trained multi-layer neural network to determine whether it is fire or non-fire. It may be judged.

By inputting the infrared image of the monitoring area into the fire detector in this way, it is possible to monitor the fire using the monitoring image without being affected by the lighting state of the monitoring area, the change in brightness during the day and night, and the like.

(others)
Further, the present invention is not limited to the above-described embodiment, includes appropriate modifications that do not impair its purpose and advantages, and is not further limited by the numerical values shown in the above-described embodiment.

10: Fire judgment device 12: Fire receiver 14: Monitoring area 16: Surveillance camera 18: Fire detector 20: Fire detector 20-1: Oil fire detector 20-2: Electric fire detector 20-3: Chemical fire Detector 20-4: Smoked fire detector 22: Signal cable 24: Environmental fire discrimination unit 26, 68: Learning control unit 27: Threshold setting unit 28: Fire judgment unit 30, 54: Image input unit 32: Multi-layer neural network Network 34: Learning image storage unit 36: Control unit 38: Feature extraction unit 40: Recognition unit 42: Input images 43, 45a, 45b: Weight filters 44a, 44b, 44c: Feature maps 46, 74 : Input layer 48: Fully coupled 50, 76 : Intermediate layer 52: Output layer 56: Image data set storage unit 60: Image analysis unit 62: Fire type determination unit 64: Convolutional neural network 66: Recursive neural network 70: Judgment device 72: Sisorus dictionary 77: LSTM Input layer 78: LSTM hidden layer 80: Word register 82: Word vector conversion unit 84: Probability conversion unit 86: Cost calculation unit

Claims (10)

In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
An environmental fire discrimination unit that determines the type of fire that is likely to occur from the monitoring environment of the monitoring area, and
A learning control unit that learns a multi-layered neural network of the fire detector by deep learning according to the type of fire determined by the environmental fire discrimination unit.
A fire determination unit that determines a fire by comparing the estimated fire value output when the input information of the monitoring area is input to the learned fire detector with a predetermined threshold value.
A fire monitoring system characterized by the fact that
In the fire monitoring system of請Motomeko 1, wherein,
When a plurality of types of fires are discriminated from the monitoring environment of the monitoring area, the environmental fire discrimination unit sets priorities in the order of fires that are likely to occur.
The learning control unit is a fire monitoring system characterized in that the multi-layer neural network is learned by deep learning by one or a plurality of types of fires having a high priority determined by the environmental fire determination unit.
In a fire monitoring system that detects a fire based on the input information of the monitoring area using a fire detector composed of a multi-layer neural network.
An environmental fire discrimination unit that determines the type of fire that is likely to occur from the monitoring environment of the monitoring area, and
A plurality of types of fire detectors learned by deep learning for each type of fire by the learning control unit,
A fire determination unit that determines a fire by comparing each of the fire estimates output when the input information is input to the learned plurality of types of fire detectors with a predetermined threshold value corresponding to the type of fire. ,
A threshold setting unit for setting said threshold value corresponding to the type of the fire in the fire determination unit, so as to facilitate determining the type of fire is determined by the environmental fire discriminating section,
A fire monitoring system characterized by the fact that
In the fire monitoring system according to claim 3,
The threshold value setting unit is a fire monitoring system characterized in that a threshold value corresponding to a type of fire determined by the environmental fire determination unit is set to a value lower than a threshold value corresponding to other types of fire.
In the fire monitoring system according to any one of claims 1 to 4.
The environmental fire discrimination unit
An image analysis unit composed of a multi-layer neural network that analyzes the image of the monitoring area captured by the image pickup unit and outputs the elements contained in the image as words.
The environment of the monitoring area is estimated by comparing the word of the element included in the image output from the image analysis unit with the word indicating a predetermined monitoring environment stored in advance in the dictionary, and the estimated monitoring environment is supported. A fire type determination unit that determines the types of fires that are likely to occur
A fire monitoring system characterized by the fact that
In the fire monitoring system according to any one of claims 1 to 5.
It said fire detector is a fire monitoring system and outputs the fire estimate an image of the monitored area captured by the physical amount detected and or the imaging unit by the sensor by inputting as said input information.
In the fire monitoring system according to any one of claims 1 to 6.
The monitoring environment includes a boiler room, an electric room, a server room, a cooking room, and a laboratory.
A fire monitoring system characterized in that the types of fires determined by the environmental fire determination unit include smoked fires, oil fires, electric fires, and chemical fires.
In the fire monitoring system according to any one of claims 1 to 7.
The multi-layer neural network of the fire detector
It consists of a feature extraction unit and a recognition unit.
The feature extraction unit, and the input information neural network convolution having a plurality of convolutions layers for generating feature information characteristic of the input information is extracted by inputting,
The recognition unit inputs the feature information output from the convolutional neural network.
A fire monitoring system characterized by forming a plurality of fully connected neural networks that output the estimated fire values.
In the fire monitoring system according to any one of claims 1 to 8.
The learning control unit, the back propagation based on an error between the expected value of the output values with a predetermined value when the inputted learning information multilayered neural network of the fire detector, the multilayer of the fire detector A fire monitoring system characterized by training a neural network of equations.
In the fire monitoring system according to any one of claims 1 to 9.
The fire monitoring unit is a fire monitoring system characterized by displaying the type of fire on which the fire determination is based in addition to the fire determination.

Images