JP2019092972A - Water discharge-type fire extinguishing equipment - Google Patents

Abstract

To detect a fire source from an image of a caution zone captured by a monitoring camera and reliably suppress and extinguish a fire by water discharge from a water cannon.SOLUTION: A fire source detection device 34 divides a monitored image of a fire extinguishing object zone captured by a monitoring camera 12, into a plurality of block images, inputs the block images into a multilayer neural network 40, and outputs estimate values representing probabilities belonging to respective classes of a flame block, a smoke block, a water discharge block, and a normal block. A determination control part 42 discriminates the flame block to detect the fire source. A water discharge control part 36 discharges fire-fighting water toward the fire source detected by the fire source detection device 34.SELECTED DRAWING: Figure 5

Description

  BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to a water extinguishing type fire extinguishing system which detects a fire source from an image of a monitored area captured by a monitoring camera by a neural network and suppresses a fire by squirting water from a water gun.

  Conventionally, as a fire extinguishing facility of a facility having a wide monitoring area such as a stadium, an exhibition hall, etc., a water extinguishing type fire extinguishing facility using a water gun is installed. The water discharge type fire extinguishing equipment monitors the monitoring area by optical horizontal scanning and vertical scanning with a scanning fire detector, detects the fire from the light reception signal by the energy from the fire flame, and the fire source position to be controlled The water discharge gun is directed horizontally to the fire source position, and the fire pressure is controlled according to the distance to the fire source to extinguish the water by extinguishing the extinguishing water.

Unexamined-Japanese-Patent No. 2006-223440 Unexamined-Japanese-Patent No. 2006-227783

  However, in such conventional water discharge type fire extinguishing equipment, although the fire source position is calculated by the scanning fire detector and the water extinguishing water is discharged from the water discharge gun, the scanning fire detector is special Is used to scan the monitoring area with high resolution and detect fires by horizontal and vertical scanning of various optical systems, resulting in complicated structure and high cost, and time and effort for inspection and adjustment to maintain accuracy. There is also an operational problem of this.

  On the other hand, in recent years, for example, images of a large number of cats and dogs are labeled, they are learned by a multi-layered neural network equipped with a convolutional neural network, so-called deep learning is performed, and new images are learned. A technique is disclosed that is presented on a multi-layered neural network and determines whether it is a cat or a dog.

  In addition, deep learning is considered not only for image analysis but also for natural language processing and behavioral analysis.

  Such a multi-layered neural network is provided in the fire source detection device, and at the time of learning, images of multiple fires and non-fires are prepared and learned by the multi-layered neural network. A water extinguishing system that suppresses fire by suppressing the fire by discharging water for fire extinguishing from the water gun by estimating the fire source with high accuracy from the output if the image of the monitored area taken is input to the learned multi-layered neural network Can be constructed.

  However, among the images of a large surveillance area such as a stadium or an exhibition hall captured by a surveillance camera, characteristic portions showing fire sources such as flames and smoke are limited to a narrow range, and a multi-layered neural network throughout the entire image Because the characteristic part indicating the fire source is limited to a narrow range even if learning and inputting the image of the monitoring area after learning, the judgment accuracy of the fire source is reduced, and the performance of suppressing and extinguishing the fire by water discharge from the water gun May decrease.

  The present invention improves the detection accuracy of the fire source by the multi-layered neural network from the image of the caution area captured by the surveillance camera, and reliably suppresses the fire by the water discharge from the water discharge gun. Intended to provide.

(Water discharge type fire extinguishing equipment)
The present invention relates to a water discharge type fire extinguishing system,
A fire source detection device for detecting a fire source by inputting a monitoring image of a caution area captured by an imaging unit into a multilayer neural network;
A water gun apparatus for discharging fire extinguishing water toward the fire source detected by the fire source detection device;
Is provided.

(Block division)
The fire source detection device divides the monitoring image of the alert area captured by the imaging unit into a plurality of block images, and inputs the block image to the multilayer neural network to detect the fire source.

(Learning control and divided block identification)
Furthermore, the water discharge type fire extinguishing equipment
A learning image generation unit that divides an input image into a plurality of block images as a learning image and classifies the input image into a flame learning block image, a smoke learning block image, a water discharge learning block image, or a normal learning block image and stores it in a learning image storage unit ,
A learning control unit for learning a multi-layered neural network of a fire source detection device, the flame learning block image, the smoke learning block image, the water discharge learning block image and the normal learning block image stored in the learning image storage unit;
Is provided,
The multi-layered neural network of the fire source detection device learned by the learning control unit inputs a block image, detects whether it is a flame block, a smoke block, a water discharge block or a normal block and outputs it.

(Estimation of fire source position based on flame block and water discharge)
The water discharge device discharges water toward the fire source estimated based on the one or more flame blocks detected by the fire source detection device.

(Estimation of fire source position based on smoke block and water discharge)
The water discharge device discharges water toward the estimated fire source based on the block when one or more smoke blocks are detected without detecting the fire block by the fire source detection device.

(Determination of water release to fire source)
When the water discharge device starts water discharge toward the fire source position, it is judged from the temporal change of the smoke block and the water discharge block detected by the fire source detection device whether the water discharge hits the fire source or not. Inform.

(Determination of water discharge by water discharge image and water discharge incompetence image)
The multi-layered neural network of the water discharge device is learned by the water discharge image when the water discharge hits the fire source and the water discharge non-shooting image when the water discharge does not hit the fire source, and starts water discharge toward the fire source position. In this case, an image of the monitoring area is input to the multi-layered neural network, and it is determined whether or not the fire source has been hit by water discharge and notified.

(Feedback feedback control to the fire source)
The water discharge gun device controls at least one of the water discharge direction and the water discharge pressure so as to obtain the judgment result that the water discharge is hit to the fire source when it is judged that the water discharge to the fire source is not hit.

(Configuration of multi-layered neural network)
Multilayer neural networks are
It consists of feature extraction unit and recognition unit,
The feature extraction unit is a convolutional neural network including a plurality of convolutional layers that receives input information and generates feature information from which features of the input information are extracted,
The recognition unit inputs the feature information output from the convolutional neural network, and outputs an estimated value indicating the probability of belonging to each class of flame block, smoke block, water discharge block, and normal block.

(Generation of block image description)
Furthermore, an image analysis device is provided which inputs the block image into the second multilayer neural network and outputs an image explanatory text indicating the content of the block image.

(Water discharge trajectory control by identification of obstacles)
When the image analysis device identifies no obstacle or an obstacle through which the water discharges from the image description of the block image located on the water discharge path to the fire source, the water gun apparatus directs the fire source to direct activation. If the controller instructs water discharge and identifies an obstacle from which the water discharge can not penetrate from the image description, the water discharge device is instructed to discharge water by turning on the curve to bypass the obstacle.

(Control of discharge amount by identification of combustibles)
The image analysis device instructs the water discharge gun device to increase the amount of water discharge when identifying the combustibles from the image description of the block image located near the fire source, and discharges the water when the noncombustibles are identified from the image description. Instruct the gun device to reduce the amount of water discharge.

(Drain water stop control by identification of person)
When the image analysis device identifies a person from the image description of the block image located on the water discharge path to the fire source, the image analysis device instructs the water discharge device to prohibit water discharge start or stop water discharge in water discharge.

(Drain water stop control by identification of person)
When the image analysis device identifies the fire fighting activity of a person from the image description of the block image located on the water discharge path to the fire source, the image analysis device instructs the water discharge gun device to prohibit the start of water discharge, and the person from the image description When it is identified that it has fallen, the water gun device is instructed to start water discharge.

(Support for water discharge operation based on identification result of image description sentence)
The water discharge gun device detects the operation of the operation unit and starts water discharge,
From the description of the image block, the image analysis device is an obstacle on the water discharge trajectory, combustibles near the fire source, the presence of a person on the water discharge trajectory, a person's fire fighting activity on the water discharge trajectory or a fall on the water discharge trajectory When a person is identified, the identification result is displayed on the display unit provided in the operation unit of the water discharge gun device.

(Multilayer neural network of image analysis system)
The second multilayer neural network of the image analysis device is
A convolutional neural network which extracts feature quantities from the input image of the monitored area and outputs it from a predetermined intermediate layer;
A recursive neural network which receives feature quantities output from a convolutional neural network and generates and outputs an image description of an image of a monitored area;
It consists of

(Basic effect)
The present invention, in a water discharge type fire extinguishing facility, is detected by a fire source detection device for detecting a fire source by inputting a monitoring image of a caution area captured by an imaging unit to a multilayer neural network; Since the water gun apparatus for discharging fire extinguishing water toward the fire source is provided, the fire source is displayed with high accuracy when the monitoring image of the alert area captured by the imaging unit is input to the multilayer neural network. Detect the image and ensure that the fire can be suppressed and extinguished by discharging water from the water gun.

(Effect by block division)
In addition, the fire source detection device divides the monitoring image of the alert area captured by the imaging unit into a plurality of block images and inputs the block image to the multi-layered neural network to detect the fire source. Even if the characteristic parts such as fire and smoke due to fire occupying the entire image are limited to a narrow range, dividing the monitoring image into multiple block images makes the characteristic parts due to fire occupying the whole image of the block image When the block image is input to the multi-layered neural network, the block image indicating the fire source is detected with high accuracy from the block image, and the water can be reliably suppressed and extinguished by the water discharge from the water gun.

  In addition, a general-purpose surveillance camera that functions as an imaging unit can be used instead of a scanning fire detector that is complex in structure and expensive in cost, and the resolution accuracy for detecting a fire source is, for example, 4K. Since the image of the surveillance area taken by the surveillance camera of this camera has a pixel arrangement of 2160 × 3840 pixels in height and width, high resolution accuracy of the image is obtained even when divided into block images, and the fire source position is detected with high accuracy. Reliable suppression and extinction can be achieved by water discharge from the water gun.

(Effects of learning control and divided block identification)
In addition, the discharge-type fire extinguishing equipment further divides the input image into a plurality of block images as a learning image, and classifies the image into a flame learning block image, a smoke learning block image, a water discharge learning block image, or a normal learning block image. A multi-layered neural network of a fire source detection device, a learning image generation unit stored in an image storage unit, a flame learning block image, a smoke learning block image, a water discharge learning block image and a normal learning block image stored in a learning image storage unit And the multi-layered neural network of the fire source detection device learned by the learning control unit inputs a block image and detects whether it is a flame block, a smoke block, a water discharge block or a normal block. And the image at the time of a fire captured by the surveillance camera in a fire experiment etc. The image at the time of water discharge taken by the surveillance camera by the water test is divided into a plurality of block images as a learning image, and the divided block images are a flame block image, a smoke block image, a water discharge block image and a normal block image (flame, smoke, By classifying and storing in a block image without water discharge and storing the stored flame block image, smoke block image, water discharge block image and normal block image into a multi-layered neural network of a fire source detection device and learning, The feature part by fire and water discharge which occupies the whole of the learning block image can be made into a wide range, and when the image of the monitoring area is divided into block images and input, whether it is a fire block, smoke block, water discharge block or normal block Can be detected with high accuracy.

(Estimation of fire source position based on flame block and effect by water discharge)
In addition, since the water discharge gun device discharges water toward the fire source estimated based on one or more flame blocks detected by the fire source detection device, the divided blocks into which the monitoring image is divided are, for example, water discharge guns. When a three-dimensional coordinate position with the position as the origin is set in advance and a single flame block is detected, setting of a water discharge trajectory by the water gun with the three-dimensional coordinate position of the flame block as the fire source position The fire gun can discharge water for fire extinguishing from the water gun to the detected fire source position to ensure the hit.

  When a plurality of sets of fire blocks are detected, for example, setting of a water discharge trajectory by a water discharge gun is performed with the three-dimensional coordinate position indicating the center position of the fire block set as the fire source position. You can be sure to hit the water by extinguishing water from the water gun in position.

(Estimation of fire source position based on smoke block and effect by water discharge)
In addition, since the water discharge gun device is configured to discharge water toward the fire source estimated based on the smoke block, when the fire source detection device detects one or more smoke blocks without detecting the fire block, For example, if the fire source is behind a shadow and can not be seen from the surveillance camera, for example, from the distribution of smoke blocks, the division block adjacent below the lowest point of the smoke block distributed is estimated as the fire source, Release the extinguishing water from the water gun to a position beyond the point to suppress and extinguish the fire.

(Effect by judgment of water release to the fire source)
In addition, when the water discharge device starts water discharge toward the fire source position, it is determined whether the water discharge hits the fire source from the temporal change of the smoke block and the water discharge block detected by the fire source detection device. The time change of smoke when fire extinguishing water from the water gun hits the fire source, for example, because white smoke spreads widely when the fire extinguishing water hits the fire, It can be informed to the operator that the fire extinguishing water hits the fire source from the temporal change of the smoke block. Moreover, even if the water discharge is started, if the temporal change of the smoke block is not detected such that the white smoke spreads, the operator can be notified that the water discharge is not hit, and the operator receives the water gun The water discharge direction can be adjusted by adjusting the water discharge direction and the water discharge distance.

(Effect by judgment of water discharge by water discharge image and water discharge incompetence image)
As another embodiment, the multi-layered neural network of the water discharge device is learned from the water discharge shooting image when the water discharge hits the fire source and the water discharge non-hit image when the water discharge does not hit the fire source, When the water discharge toward the position is started, the image of the monitoring area is input to the multi-layered neural network, and it is determined whether or not the water discharge hits the fire source, and the notification is made. As the smoke spreads when the water is hit, the spread of the smoke when there is no water discharge or when the water is not fired and the spread of the smoke when the water is hit are respectively learned to grasp the spread of the smoke as a feature. It can be determined whether the water discharge has hit.

(Effect of feedback control during water discharge to the fire source)
In addition, the water discharge gun device controls at least one of the water discharge direction and the water discharge pressure so as to obtain the judgment result that the water discharge is hit to the fire source when it is judged that the water discharge to the fire source is not hit. In this case, if no change in the smoke block over time is detected, such as when the fire extinguishing water hits the fire source and white smoke spreads when water discharge is started, for example, the water discharge pressure and the water discharge direction initially set are set. At the starting point, the discharge pressure is increased by a predetermined value and the discharge distance is extended. The discharge direction is changed within a predetermined angle range, and the discharge pressure is detected until a temporal change in the smoke block is detected such that white smoke spreads. Gradually increase the water discharge from the fire source reliably by changing the water discharge distance from the water discharge gun and the water discharge direction even if the water discharge does not hit the fire source at the start of the water discharge. Kill.

(Effect by the construction of multi-layered neural network)
A multi-layered neural network is composed of a feature extraction unit and a recognition unit, and the feature extraction unit is a convolutional neural network including a plurality of convolutional layers for inputting input information and generating feature information from which features of the input information are extracted. Since the recognition unit inputs the feature information output from the convolutional neural network and outputs an estimated value indicating the probability of belonging to each class of flame block, smoke block, water discharge block and normal block, the convolutional neural network The feature of the block image is automatically extracted by the network, so that the feature of the block image such as flame, smoke, water discharge, etc. is extracted from the input information of the monitoring area, for example, the outline etc. in the image Feature of the block image can be extracted without requiring any Every time a flame block, smoke block, the probability of belonging to each class of water discharge block and normal block is outputted, to ensure distinguishable the type of block.

(Effect of block image description generation)
Furthermore, since an image analysis apparatus is provided which inputs a block image to the second multilayer neural network and outputs an image explanatory text indicating the contents of the block image, for example, "flame is appearing" in the case of a flame block image. If a smoke block image is generated, an image description such as “smoke is generated” is generated. If an image is a water discharge block image, an image description such as “water is applied” is generated. The content of the block image can be artificially grasped by analyzing the image description.

(Effect of water trajectory control by identification of obstacles)
In addition, when the image analysis device identifies no obstacle or an obstacle through which the water discharges from the image description of the block image located on the water discharge path to the fire source, the water gun apparatus directs the fire source toward the fire source. In the case where water discharge at startup is instructed and an obstacle that can not penetrate water discharge is identified from the image description, water discharge is instructed by water spray activation for bypassing the obstacle to the water discharge gun device, for example, a fire source is detected Then, before starting water discharge from the water discharge gun, a block image located on the water discharge trajectory to the fire source is input to the image analysis device to generate an image explanatory text, for example, an image such as “a high structure stands” If an explanatory note is generated, the water from the direct activation does not reach the fire source as it strikes the construction, and the water can be discharged to the fire source even if there is an obstacle, by discharging the water according to the curved trajectory. In addition, when an image caption such as "the curtain is stretched" is generated, the curtain can penetrate the water discharge, so the water can be discharged by the direct trajectory so that the obstacle can be penetrated and the fire source can be discharged.

(Effect of discharge control by identification of combustibles)
In addition, when the image analysis device identifies combustibles from the image description of the block image located in the vicinity of the fire source, it instructs the water discharge gun device to increase the amount of water discharge, and identifies the incombustibles from the image description. In the case of fire flammable materials near the fire source, increase the amount of water discharge to prevent the spread of fire, and when there is non-combustible material near the fire source, because Reduce the amount of water discharge to reduce water loss damage.

(Effect of water stop control by identification of person)
In addition, when the image analysis device identifies a person's fire extinguishing activity from the image description of the block image located on the water discharge path to the fire source, the image analysis device instructs the water discharge gun device to prohibit the water discharge start. When it is identified that a person falls down, since the water discharge gun device is instructed to start water discharge, for example, it is possible to safely evacuate the person who has evacuated without interfering with the water discharge.

(Effect of water stop control by identification of person)
In addition, when the image analysis device identifies a person's fire extinguishing activity from the image description of the block image located on the water discharge path to the fire source, the image analysis device instructs the water discharge gun device to prohibit the water discharge start. When it is identified that a person is falling, the water discharging gun device is instructed to start water discharging, so for example, when an image explanatory text such as “male using a fire extinguisher” is generated, the initial fire extinguishing Do not interrupt fire extinguishing activities by not identifying and starting water discharge, while starting water discharge and protecting from fire when an image explanation part such as “person is falling” is generated, water discharge track It enables control such as appropriate water discharge start and water discharge stop according to the situation of the person on the top.

(Effect by support of water discharge operation based on identification result of image explanation sentence)
Also, the water discharge gun device detects the operation of the operation unit and starts water discharge, and the image analysis device, from the description of the image block, obstacles on the water discharge trajectory, combustibles near the fire source, water discharge When the presence of a person on the orbit, the fire fighting activity of the person on the water discharge track, or the fallen person on the water discharge track is identified, the identification result is displayed on the display unit provided on the operation unit of the water discharge device Therefore, in the case of a water discharge type fire extinguishing equipment configured to be operated by a person in charge of water discharge start, the obstacle on the water discharge track, the combustibles near the fire source, the presence of the person on the water discharge track, Support by providing the operator with information on fire extinguishing activities of people on the water discharge track or the situation of the people on the water discharge track, and the person in charge can quickly and appropriately judge the water discharge gun to suppress the fire and extinguish I assume.

(Effect of multi-layered neural network in image analysis system)
Further, the second multi-layered neural network of the image analysis apparatus extracts a feature quantity from the input monitoring area image and outputs the feature quantity from a predetermined intermediate layer, and the feature quantity output from the convolutional neural network Since it is composed of a recursive neural network that inputs and generates and outputs an image description of the image of the surveillance area, the feature value of the block image such as flame, smoke, and water discharge without requiring pre-processing from the block image Is extracted automatically, and then, the extracted feature quantity is input to the recursive neural network to generate an image explanatory text indicating the content of the block image, and a person looks at the block image and makes a judgment. It enables control of the water gun device equivalent to

Explanatory drawing which showed the outline of the water discharge type fire extinguishing equipment which detects a fire source with a surveillance camera and discharges water Explanatory drawing which showed the installation state of the water gun and surveillance camera to an outdoor ball game place Explanatory drawing showing water discharge by direct orbit of water discharge gun Explanatory drawing which showed water discharge by the curved trajectory of the water discharge gun Block diagram showing the functional configuration of the central control panel of FIG. 1 Explanatory drawing which showed the function structure of the multilayer-type neural network provided in the fire source detection apparatus of FIG. Explanatory diagram showing block division of learning image taken by fire experiment in the monitoring area and classification of flame, smoke and normality of block image Explanatory diagram showing block division of learning image taken by water discharge for fire experiment in the monitoring area and classification of flame, smoke, water discharge, normality of block image A flowchart showing learning image generation control by the learning control unit of FIG. 5 The flowchart which showed the fire source detection control by the fire source detection apparatus of FIG. 5 Block diagram showing another embodiment of the central control unit of FIG. 1 Explanatory drawing which showed the function structure of the 2nd multilayer type neural network provided in the image-analysis apparatus of FIG. Flow chart showing image analysis control by the image analysis apparatus of FIG.

[First embodiment of water discharge type fire extinguishing facility]
(Outline of fire monitoring system)
FIG. 1 is an explanatory view showing an outline of a water discharge type fire extinguishing facility in which a fire source is detected by a monitoring camera and discharged.

  As shown in FIG. 1, in the water discharge type fire extinguishing equipment of this embodiment, for example, four water discharge guns 10 are installed with facilities such as stadiums and exhibition halls as fire extinguishing target areas. Four surveillance cameras 12 are installed corresponding to the target area.

  A control panel 14 and an on-site control panel 16 are provided for the water discharge gun 10. The surveillance camera 12 and the control panel 14 are connected by signal lines to a central control panel 18 installed at the monitoring center.

  A central console 20 is provided for the central control panel 18. Further, a pump 22 is provided in the pump chamber, and the pump 22 is controlled by a pump control panel 24. The water pipe 26 from the pump 22 is connected to the water discharge gun 10 via a motorized valve 13 which functions as a water discharge pressure control valve. Further, an air compressor 28 is provided in the pump chamber, and is connected to the water discharge gun 10 by an air pipe 30.

  The central control panel 18 monitors the fire source from the image of the monitoring camera 12 by a fire source detection function using a multi-layered neural network, and the central operation is performed when the fire decision signal by the fire source detection from the image of the monitoring camera 12 is received. The fire occurrence area is displayed on the table 20 to issue an alarm, and the fire source position information is transmitted to a predetermined water gun 10 corresponding to the detected fire source.

  At this time, for example, when the automatic mode is set in the central control panel 18, after setting the water discharge path to point the nozzle direction to the fire source position by horizontal swing and vertical swing of the water discharge gun 10, At startup, pressurized extinguishing water is supplied, and extinguishing water is discharged toward the fire source position by pressure control by the motor-operated valve 13 according to the distance to the fire source.

  At the same time, compressed air is supplied from the air compressor 28, and the compressed air is jetted around the extinguishing water in the water gun 10 to disperse the extinguishing water over a long distance and over a wide area with a low water discharge pressure and a small discharge amount.

  FIG. 2 is an explanatory view showing an installation state of a water gun and a monitoring camera in an outdoor ball game place. Figure 2 is an example of an outdoor ball game center, the central ground portion is defined in the fire control area 32, the four water guns 10 at a predetermined height position behind the spectator seat surrounding it around the fire control area It is arranged around 32.

  A surveillance camera 12 is installed at a high position behind the water gun 10 on the stand side. The surveillance camera 12 captures an RGB color image at, for example, 30 frames / second and outputs it as a moving image. Further, the surveillance camera 12 is compatible with 4K, and one frame has a pixel arrangement of 3840 × 2160 pixels, for example.

  Furthermore, it is desirable that the surveillance camera 12 be installed as high as possible on the stand side so that the fire suppression area 32 can be viewed from obliquely above. Such a high-level arrangement of the surveillance camera 12 minimizes the change in the imaging object with respect to the distance from the surveillance camera 12. In addition, an image captured by the monitoring camera 12 may be converted into an image looking down from the sky of the fire suppression target area 32 and output as necessary.

(Summary of water discharge trajectory)
FIG. 3 is an explanatory view showing water discharge by a direct orbit of the water discharge gun, and FIG. 4 is an explanatory view showing water discharge by a curved injection track of the water discharge gun.

  In FIG. 3, three-dimensional coordinates of the X axis, Y axis, and Z axis with the origin O are set in the fire suppression area 32, and the XY plane is the ground plane shown in FIG. The water discharge point P of the water discharge gun 10 is set on the Z axis which is H. Here, when point Q of the ground plane is detected as a fire source, the water discharge gun 10 is turned by a horizontal turning angle φ with respect to the X axis so that the horizontal water discharge axis W passes the fire source Q, and the water discharge point Since there is no obstacle between P and the fire source Q, vertically rotate the water gun 10 by a vertical turning angle θ1 downward from the water discharge point P so that the water discharge axis from the water gun 10 directly hits the fire source Q The direct orbit 54 is set and water is discharged.

  On the other hand, as shown in FIG. 4, when the obstacle 58 exists between the water discharge point P and the fire source Q, the water discharge gun 10 is vertically turned upward from the water discharge point P by the vertical turning angle θ2, Water is released by setting the curved trajectory 56 so that the water discharge axis from the gun 10 draws a parabola and directly strikes the fire source Q beyond the obstacle 58.

  The direct orbit 54 in FIG. 3 is effective when the distance L to the fire source Q is short, but when the distance L is long, the direct orbit 54 does not hit the fire source Q, so the downward vertical swing angle θ1 is Correct upward and set the fire source Q to a direct trajectory. The relationship between the vertical swing angle θ1 of the direct trajectory and the water discharge distance L is set in advance, and when the fire source Q is detected, the corresponding vertical swing angle θ is obtained by calculating the water discharge distance L. The direct orbit 54 is set.

  In addition, the relationship between the vertical turning angle θ2 and the water discharge distance L is also set in advance with respect to the curved trajectory 58 of FIG. 4, and when the obstacle is identified, the water discharge distance L to the fire source Q is calculated. Then, the corresponding vertical turning angle θ2 is acquired, and the curved trajectory 56 is set.

  Furthermore, when a person is in the vicinity of the fire source Q, water discharge due to the setting of the direct trajectory 54 is dangerous. Therefore, regardless of the presence or absence of an obstacle, the bent trajectory 56 is always set to discharge water. good.

[Configuration of central control panel]
FIG. 5 is a block diagram showing a functional configuration of the central control panel of FIG. 1, taking one system of the water gun 10 and the monitoring camera 12 shown in FIG. 1 as an example.

  As shown in FIG. 5, the central control panel 18 is provided with a fire source detection device 34 and a water discharge control unit 36. The fire source detection device 34 divides the monitoring image of the extinguishing target area captured by the monitoring camera 12 functioning as an imaging unit into a plurality of block images, and inputs the block image to the multilayer neural network 40 to detect the fire source Do.

  The water discharge control unit 36 sets, for example, the direct trajectory 54 shown in FIG. 3 or the curved trajectory 56 for avoiding the obstacle shown in FIG. 4 toward the fire source detected by the fire source detection device 34 for extinguishing water. Reject water.

(Configuration of fire source detection device)
The fire source detection device 34 includes an image input unit 38, a multilayer neural network 40, a determination control unit 42, and a block classification image storage unit 44. Further, a learning controller 46 is provided for the multilayer neural network 40. The learning control unit 46 is connected to a learning image storage unit 48, an operation unit 50 such as a keyboard and a mouse, and a display unit 52 having a monitor screen. These functions are performed by the CPU of a computer circuit corresponding to neural network processing. It is realized by program execution.

  The image input unit 38 inputs a moving image of a fire extinguishing target area captured by the monitoring camera 12 as a monitoring image in frame units, divides it into block images of 16 blocks by 16 × 16, for example, and divides each block image into multiple layers It sequentially inputs to the neural network 40, and outputs an estimated value of clustering indicating whether the block image is a flame block, a smoke block or a water discharge block.

  The determination control unit 42 compares each estimated value of the block image indicating flame, smoke, and water discharge output from the multi-layered neural network 40 with a predetermined threshold, and determines the type of the block image of the estimated value exceeding the threshold as the determination result It is stored in the block classification image storage unit 44.

  In addition, when the type determination of all block images divided from the monitoring image is completed, the determination control unit 42 detects a fire block from the type blocks stored in the block classification image storage unit 44 and detects the fire source position ( The fire source position coordinates are determined, and the fire source position is output to the water discharge control unit 36 to perform water discharge control.

  In this case, when a single fire block is detected from the block classification image storage unit 44, the determination control unit 42 outputs the detected position coordinates of the fire block to the water discharge control unit 36 as the fire source position. When a set of fire blocks is detected, for example, the coordinates of the center position of the fire block set are estimated as the fire source position and output to the water discharge control unit 36.

  Further, when the fire block is not detected from the block classification image storage unit 44, the judgment control unit 42 detects a smoke block, and estimates a fire source position from the detected smoke block. The estimation of the fire source position based on the smoke block performs control to estimate the block adjacent to the smoke block at the diffusion start end in the smoke block set as the fire source position, since the smoke is distributed out of the fire source and diffused. .

  Even when the flame is hidden behind the shield and can not be seen from the surveillance camera 12, the fire source position is estimated from the smoke block, and the fire suppression can be surely suppressed by the discharge of water by setting the curved trajectory, for example.

  The learning control unit 46 sequentially reads the flame learning block image, the smoke learning block image, the water discharge learning block image, and the normal learning block image stored in advance in the learning image storage unit 48 at the time of startup of the facility, etc. A supervised block image is input to the multilayer neural network 40 via the unit 38, and the weights and bias of the multilayer neural network 40 are learned by a learning method such as a back propagation method (error back propagation method).

  When a block image obtained by dividing the image of the monitoring area captured by the monitoring camera 12 is input to the multi-layered neural network 40 that has been learned using this supervised block image, classes of flame, smoke, water discharge and normal An estimated value indicating) is output.

Here, assuming that the flame estimated value is y1, the smoke estimated value is y2, the water discharge estimated value is y3, and the normal expected value is y4, ideally
(Y1, y2, y3, y4) = (1, 0, 0, 0) for a fire block
(Y1, y2, y3, y4) = (0, 1, 0, 0) for smoke block
(Y1, y2, y3, y4) = (0, 0, 1, 0) in the case of a water discharge block
(Y1, y2, y3, y4) = (0, 0, 0, 1) for a normal block
It becomes.

  When an actual block image is input to the multilayer neural network 40, the sum of the estimated values y1 to y4 is 1 and each takes a value between 0 and 1, so the control unit determines the estimated values y1 to y4. 42 is input and compared with a predetermined threshold, for example, 0.5, and the class of the estimated value yi equal to or greater than the threshold is determined as the discrimination result. The determination control unit 42 may set the class that is the maximum value among the estimated values y1 to y4 as the determination result.

(Multilayer neural network)
FIG. 6 is an explanatory view showing a functional configuration of the multilayer neural network shown in FIG. 5, schematically shown in FIG. 6 (A), and schematically shown in detail in FIG. 6 (B).

  As shown in FIG. 6A, the multi-layered neural network 40 of the present embodiment is composed of a convolutional neural network 60 functioning as a feature extraction unit and a total combination neural network 61 functioning as a recognition unit.

  The multilayer neural network 40 is a neural network that performs deep learning (deep learning), is a neural network having a deep hierarchy in which a plurality of intermediate layers are connected, and performs expression learning that is feature extraction.

  A normal neural network needs an operation by artificial trial and error to extract features for estimating the features of the image such as flame, smoke, water discharge and normality from the image, but the multilayer neural network 40 has features By using the convolutional neural network 60 that functions as an extraction unit, the pixel values of the image are input, the optimum features are extracted by learning, and input to the all-combined neural network 61 that functions as a recognition unit for flame, smoke, water discharge. Perform clustering that is classified based on image features such as normal and.

  The total connection neural network 61 is composed of an input layer 66, a total connection 68, a repetition of an intermediate layer 70 and a total connection 68, and an output layer 72, as schematically shown in FIG. 6 (B).

  Here, since the fully coupled neural network 61 performs multi-class classification to classify block images into four classes of flame, smoke, water discharge and normal, the final output layer 72 has the same four as the target four classes. The units are arranged, and the inputs to these units are made to be outputs y1, y2, y3 and y4 with a sum of 1 using a soft max function, and the outputs y1, y2, y3 and y4 of each unit are provided. Will indicate the probability of belonging to the class.

(Convolutional neural network)
FIG. 6B schematically shows the structure of a convolutional neural network 60 which functions as a feature extraction unit.

  The convolutional neural network 60 has a slightly different characteristic from a normal neural network, and incorporates biological structures from the visual cortex. The visual cortex contains a receptive field that is a collection of small cells that are sensitive to a small area of the visual field, and the behavior of the receptive field can be mimicked by learning weights in the form of a matrix. This matrix is called a weight filter (kernel) and is sensitive to similar subregions of an image, as well as the role played by the receptive field in biological terms.

  The convolutional neural network 60 can express the similarity between the weight filter and the small area by convolution operation, and through this operation, the appropriate features of the image can be extracted.

  As shown in FIG. 6 (B), the convolutional neural network 60 first performs convolution processing by the weight filter 63 on the input image 62 input as a block image. For example, the weight filter 63 is a matrix filter with predetermined weighting of 3 × 3 in vertical and horizontal directions, and performs nineteen pixels of the input image 62 by performing a convolution operation while aligning the center of the filter with each pixel of the input image 62. A plurality of feature maps 64a are generated by convolution to one pixel of the feature map 64a which is a small area.

  Subsequently, pooling operation is performed on the feature map 64 a obtained by the convolution operation. The pooling operation is a process of removing feature amounts unnecessary for identification and extracting feature amounts necessary for identification.

  Subsequently, the convolution operation and the pooling operation using the weight filters 65a and 65b are repeated in multiple stages to obtain the feature maps 64b and 64c, and the feature map 64c of the last layer is input to the fully coupled neural network 61. The block image input by the recognition unit using the total connection neural network 61 is estimated to belong to the flame, smoke, water discharge, or normal class.

  It should be noted that the pooling operation in the convolutional neural network is a pooling operation because the unnecessary feature quantities are not necessarily clear for estimation of flame, smoke, water discharge, and normality, and the necessary feature quantities may be deleted. You may choose not to

[Learning of multi-layered neural network]
(Back propagation)
A neural network composed of an input layer, a plurality of intermediate layers, and an output layer provides a plurality of units in each layer and combines them with a plurality of units in other layers, and weights (bias) and bias values are set for each unit The vector product of a plurality of input values and weights is calculated and the bias values are added to obtain the sum, which is passed through a predetermined activation function and output to the unit of the next layer. Forward propagation is performed where the value propagates until reaching.

  To change the weights and biases of such neural networks, we use a learning algorithm known as backpropagation. In back propagation, supervised learning when the data set of input value x and expected output value (expected value) y is given to the network and unsupervised learning when the input value x is given only to the network In this embodiment, for example, supervised learning is performed.

  When performing backpropagation in supervised learning, for example, a function of mean square error is used as an error for comparing values of estimated value y * and expected value y, which are the results of forward propagation passed through the network Do.

  In back propagation, using the magnitude of the error between the estimated value y * and the expected value y, the value is propagated from the back to the front of the network while correcting the weight and bias. The amount corrected for each weight and bias is treated as a contribution to the error, calculated by the steepest descent method, and the value of the weight and bias is changed to minimize the value of the error function.

The procedure of learning by back propagation for a neural network is as follows.
(1) The input value x is input to the neural network, and forward propagation is performed to obtain the estimated value y *.
(2) Calculate an error with an error function based on the estimated value y * and the expected value y.
(3) Perform back propagation in the network while updating weights and biases.

  This procedure is repeated using combinations of different input values x and expected values y until the error of weight and bias of the neural network is minimized to minimize the value of the error function.

  In the supervised learning control of the multi-layered neural network 40 shown in FIG. 6B, the expected values (y1, y2, y3, y4) = (1, 0, 0, 0) of the flame learning block image, the smoke learning block Expected value of image (y1, y2, y3, y4) = (0, 1, 0, 0), Expected value of water discharge learning block image (y1, y2, y3, y4) = (0, 0, 1, 0) Furthermore, the expected values (y1, y2, y3, y4) = (0, 0, 0, 1) of the normal learning block image are used to perform the back propagation described above.

  When back propagation is performed by unsupervised learning, the value is propagated while correcting the weight and bias from the rear to the front of the network using the magnitude of the error between the estimated value y * and the input value x. Also in this case, the corrected amount for each weight and bias is treated as a contribution to the error, calculated by the steepest descent method, and the value of the weight and bias is changed to minimize the value of the error function.

(Function to generate learning image)
When a predetermined learning image generation operation is performed by the operation unit 50, the learning control unit 46 illustrated in FIG. 5 inputs an image as a learning image in frame units from a moving image of a fire experiment or a water discharge experiment captured by the monitoring camera 12. For example, in the case of a 4K image, the image is divided into block images of 16 × 16 blocks of 16 × 16 and displayed on the monitor screen of the display unit 52. Note that the learning image may be recorded on a video recording device (not shown) that is a moving image of a fire experiment or a water discharge experiment captured by the monitoring camera 12 and read from the video recording device to the image input unit 38.

  FIG. 7 is an explanatory view showing block division of a learning image captured by a fire experiment or the like in a monitoring area and classification of flame, smoke, and normal of the block image.

  As shown in FIG. 7A, on the monitor screen of the display unit 52 of FIG. 5, a fire image 74-1 read by the image input unit 38 and divided into blocks is displayed. In the present embodiment, the fire image 74-1 is divided into block images of 26.times.16 256 blocks, but this is an example, and the block size is on the screen corresponding to the water spray area by the water discharge gun 10. It is divided to be the size.

  The fire image 74-1 is an image of an early stage of fire in which, for example, a timber is stacked on a combustible material 76 and ignited as a combustible material, and a flame 78 slightly rises after ignition, and smoke 80 rises thereon.

  Classification of the smoke learning block image S, the flame learning block image F, and the block image to be a normal block image is performed on the block-divided fire image 74-1 by visual judgment of the operator. A block classification image 82-1 shown in FIG. 7B is generated by this classification operation. The normal block image is indicated by a white frame block.

  Here, the normal learning block image is a block having no flame or smoke, the smoke learning block image S is a block in which smoke can be viewed, and the flame learning block image F is a block in which a flame can be viewed.

  Further, the block classified image 82-1 shown in FIG. 7B has block addresses (binary addresses) A1 to A256 specified by row numbers 1 to 16 and column numbers 1 to 16 with the upper left corner as an initial position, for example. Are stored in a data format storing identification information indicating the smoke learning block image S and the flame learning block image F in correspondence with.

  Furthermore, the block addresses A1 to A256 are stored in a data format including coordinate values (Xi, Yi, Zi) indicating the three-dimensional coordinate position of the fire suppression area 32 shown in FIGS. 3 and 4.

  FIG. 8 is an explanatory view showing block division of a learning image captured by water discharge and fire, smoke, water discharge, and classification of normality of the block image for the fire experiment of the fire suppression area.

  In the case of the fire image 74-2 divided into blocks shown in FIG. 8A, the fire is expanded with the passage of time as compared with FIG. 7A, and the flame 78 from the combustible material 76 largely rises. Here, the water discharge 84 from the water discharge gun 10 is in a state of being started, and the smoke 80 containing water vapor is largely diffused by the water discharge 84 hitting the combustion material 76.

  FIG. 8B is a block classification image 82-2 in the fire image 74-2 of FIG. 8A, and in addition to the flame learning block image F and the smoke learning block image S, the water discharge learning block image W is newly added. Is participating.

  7 (A) and 8 (A) are an example of a fire image, and in fact, as shown in FIG. 8 (A) from FIG. 7 (A) from the ignition of the burning matter, it is expanded so that the flame may burn up. Then, since the fire animation that is to be extinguished after receiving water discharge is input, for example, when recording for 5 minutes from ignition is read in frame units, 9000 fire images are obtained, each being 256 × 16 × 16 × 256 By dividing and classifying into block images of blocks, it is possible to generate a maximum of 2,304,000 normal block images, flame learning block images F, smoke learning block images S and water discharge learning block images W.

  The classification of the block image obtained by dividing the fire image may be performed by automatically determining and classifying the sum of the pixel luminance in block units, for example, in addition to the classification by the manual operation by the operator visually .

  Moreover, since imaging of a fire image by fire experiment may not be able to be performed in the fire extinguishing area 32, in this case, the fire experiment conducted using the fire experiment facility on the system manufacturing side was imaged and recorded Use video.

(Learning image generation control)
FIG. 9 is a flowchart showing learning image generation control by the learning control unit of FIG. As shown in FIG. 9, when learning image generation control is started by a predetermined operation, the learning control unit 46 performs a frame unit of a fire moving image captured by the monitoring camera 12 in step S1 and recorded, for example, in a recording device. The image is read as a fire image, divided into block images of a predetermined size in step S2, and displayed on the monitor screen of the display unit 52 as shown in FIG. 7A, for example, in step S3.

  Subsequently, the process proceeds to step S4, and the learning control unit 46 causes the block image to be a flame learning block image, a smoke learning block image, a water discharge learning block image, based on a visual operation by the operator of the block image displayed on the monitor screen. The normal learning block image is classified, and the flame learning block image, the smoke learning block image, the water discharge learning block image, and the normal learning block image among them are stored in the learning image storage unit 48 in step S5.

  Subsequently, in step S6, the learning control unit 46 determines whether all fire images have been processed, and if all the fire images have not been processed, the process from step S1 is repeated to process all the fire images. When it is determined that the process has been performed, the series of processes is ended, and the learning of the multilayer neural network 40 is performed.

(Fire source detection control)
FIG. 10 is a flowchart showing fire source detection control by the fire source detection device of FIG. As shown in FIG. 10, the fire source detection device 34 reads the monitoring image from the monitoring camera 12 into the image input unit 38 in step S11, and divides it into block images of a predetermined size in step S12. Images are input to the multi-layered neural network 40 in order, and the smoke, smoke, water discharge, and normal four classes of estimated values output from the multi-layered neural network 40 are compared with a predetermined threshold in the determination control unit 42 In step S14, the block classification image storage unit 44 is made to store the determination result.

  Subsequently, in step S15, the fire source detection device 34 repeats the processing from step S13 until the input of all block images is determined. If the input of all block images is determined, the process proceeds to step S16. It is determined whether or not a flame block is stored in the storage unit 44. If even one flame block is determined, the process proceeds to step S17, and a fire source block is estimated from the flame block. At this time, when a plurality of sets of flame blocks are determined, the flame block located at the center of the flame block set is estimated as the fire source block.

  Subsequently, the process proceeds to step S18, and the fire source detection device 34 acquires the estimated position coordinates of the fire source block based on the block address, and outputs the fire source position coordinates to the water discharge control unit 36 to Make water discharge.

  If the flame source is not determined from the block classification image storage unit 44 in step S16, the fire source detection device 34 proceeds to step S19, and whether the smoke block is stored in the block classification image storage unit 44 If a smoke block is determined, the process proceeds to step S20, and a fire source block is estimated from the smoke block.

  Here, since a large number of smoke blocks are stored in the block classification image storage unit 44 if it is a fire, the fire source detection device 34 estimates the fire source block from the distribution of the plurality of smoke blocks in step S20. Subsequently, the process proceeds to step S18, the position coordinates of the estimated fire source block are acquired based on the block address, and the fire source position coordinates are output to the water discharge control unit 36 to cause water discharge from the water discharge gun 10.

(A function to judge the water discharge to the fire source)
When the fire source detection device 34 shown in FIG. 5 outputs fire source coordinates to the water discharge control unit 36 and starts water discharge from the water discharge gun 10 toward the fire source, the fire extinguishing target area captured by the monitoring camera 16 The image is divided into block images by the image input unit 38 and input to the multi-layered neural network 40. The block control of the flame, smoke and water discharge is discriminated by the judgment control unit 42 and stored in the block classification image storage unit 44. It is judged from the temporal change of the block and the water discharge block whether or not the water discharge is hit to the fire source, and the judgment result is sent to the on-site operation panel 16 to be notified.

  This is because, as shown in FIG. 8, when the flame 78 hits the water discharge 84 from the water gun 10, the smoke 80 spreads widely, so the fire source detection device 34 changes such a smoke block in time. From this, it is determined that the fire extinguishing water has hit the fire source and the operator is notified.

  In addition, even if the water discharge is started, if the temporal change of the smoke block is not detected such that the white smoke spreads, the fire source detection device 34 can notify the operator that the water discharge is not hit. Thus, the operator can adjust the water discharge direction and the water discharge distance of the water discharge gun 10 to perform an operation of hitting the water discharge.

(Feedback feedback control to the fire source)
The fire source detection device 34 also notifies the water discharge control unit 36 when it is determined from the temporal change of the smoke block and the water discharge block that the fire source has not hit the water discharge, and the fire source hits the water discharge. The water discharge control unit 36 performs control to change at least one of the water discharge direction and the water discharge pressure so as to obtain the determined result.

  That is, the fire source detection device 34 discharges water to the water discharge control unit 36 when the temporal change of the smoke block is not detected such that the fire extinguishing water hits the fire source and the white smoke spreads when the water discharge is started. The water discharge control unit 36, which has instructed the change of the conditions and received it, for example, increases the water discharge pressure by a predetermined value and extends the water discharge distance starting from the water discharge pressure and the water discharge direction initially set. Control is performed to gradually increase the water discharge pressure until it is changed within a predetermined angle range and notification of temporal change in the smoke block is obtained from the fire source detection device 34 such that white smoke spreads. As a result, even if the fire source does not hit the water source at the start of the water discharge, it is possible to make the water source hit the water source by the substantial feedback control by changing the water discharge distance from the water discharge gun and the water discharge direction.

[Second embodiment of water discharge type fire extinguishing facility]
(Overview of Central Control Unit)
FIG. 11 is a block diagram showing another embodiment of the central control unit of FIG. As shown in FIG. 11, in addition to the fire source detection device 34 and the water discharge control unit 36, the central control panel 18 is additionally provided with an image analysis device 90.

  The fire source detection device 34 divides the monitoring image of the alerting area taken by the monitoring camera 12 into a plurality of block images, and inputs the block image into the multilayer neural network 40 to detect the fire source. Further, the water discharge control unit 36 discharges extinguishing water toward the fire source detected by the fire source detection device 34. Since such a fire source detection device 34 and the water discharge control unit 36 are basically the same as the embodiment of FIG. 5, the same reference numerals are given and the description thereof is omitted.

(Outline of image analysis device)
The image analysis device 90 is composed of an image input unit 92, a second multilayer neural network 94, a judgment control unit 96, and a thesaurus dictionary 98, and a learning controller 100 is provided for the second multilayer neural network 94. The learning control unit 100 is connected to a learning data set storage unit 102, an operation unit 104 such as a keyboard or a mouse, and a display unit 106 having a monitor screen, and these functions correspond to computer circuits corresponding to neural network processing. This is realized by the execution of a program by the CPU of

  The water discharge control unit 36 sets the water discharge track setting information of the image analysis device 90 when the water discharge control unit 36 sets a water discharge track for discharging water from the water discharge gun 10 toward the fire source position detected by the fire source detection device 34. The image is output to the image input unit 92.

  The image input unit 92 sets a water discharge track for the block image read and divided from the monitoring camera 12 based on the water discharge track setting information received from the water discharge control unit 36, and cuts out a block image located on the water discharge track. The multi-layered neural network 94 is input to generate an image description of the input block image.

  The image description generated by the second multilayer neural network 94 is output to the determination control unit 96, and comparison and comparison with the words stored in the thesaurus dictionary 98 are performed, and the block image on the water discharge trajectory indicates what The control instruction such as the water discharge start and the water discharge stop based on the result of the identification is output to the water discharge control unit 36 to enable appropriate water discharge control in accordance with the situation on the water discharge track.

  For example, when a flame block image is input to the second multilayer neural network 94, an image description such as "flame is generated" is generated, and when a smoke block image is input, "smoke is generated. The image description such as “” is generated, and further, when the water discharge block image is input, the image description such as “water is poured” is generated, and this image description is used by the determination control unit 96 in the thesaurus dictionary 98. The contents of the block image can be artificially grasped by comparing and analyzing the registered words.

(Water discharge trajectory control by identification of obstacles)
The determination control unit 96 of the image analysis device 90 uses, for example, "a high construction stands" from the image description of the block image located on the water discharge trajectory up to the fire source generated by the second multilayer neural network 94. When the image description is generated, the water control unit 36 is instructed to discharge water by setting the curved trajectory 56 shown in FIG. 4 and the water discharge by the direct trajectory hits the structure and does not reach the fire source, so the water discharge is performed by the curved trajectory. Thus, even if there is an obstacle, it is possible to reliably discharge the fire source.

  In addition, when the second multi-layered neural network 94 generates an image explanatory text such as "the curtain is stretched", the curtain can penetrate the water discharge, so the judgment control unit 96 is configured as shown in FIG. The water discharge control unit 36 is instructed to discharge the water according to the setting of 54, and the curtain serving as the obstacle is penetrated by the direct trajectory to discharge water to the fire source.

(Control of discharge amount by identification of combustibles)
The determination control unit 96 of the image analysis device 90 determines, for example, “cardboard box is stacked up” from the image description of the block image located on the water discharge trajectory up to the fire source generated by the second multilayer neural network 94. When an image explanatory text indicating such a flammable substance is generated, the water discharge control unit 36 is instructed to increase the amount of water discharge to enable a reliable fire extinguishing.

  In addition, when an image explanatory text indicating non-combustible material such as “Concrete blocks are stacked” is generated from the second multilayer neural network 94, the determination control unit 96 instructs the water discharge control unit 36 to decrease the water discharge amount. To reduce water loss damage.

(Drain water stop control by identification of person)
The judgment control unit 96 of the image analysis device 90 is, for example, from the image description of the block image positioned on the water discharge trajectory up to the fire source generated by the second multilayer neural network 94, “a large number of people are moving If the user is evacuated from the fire extinguishing area, the determination control unit 96 instructs the water discharge control unit 36 to prohibit the start of water discharge, and the water discharge is performed. Enables safe evacuation without interfering with evacuation behavior.

(Drain water stop control by identification of person)
The judgment control unit 96 of the image analysis device 90 uses, for example, “Men use a fire extinguisher from the image description of the block image located on the water discharge trajectory up to the fire source generated by the second multilayer neural network 94 When the image description indicating the initial fire extinguishment is generated, the determination control unit 96 instructs the water discharge control unit 36 to prohibit the start of water discharge so that the water extinguishing operation is not disturbed.

  Also, when an image explanatory sentence such as “person is falling” is generated from the second multi-layered neural network 94, for example, this is a situation in which the person who is going to evacuate is falling near the fire source. The judgment control unit 96 instructs the water discharge control unit 36 to start water discharge so as to protect a person who has fallen due to water discharge from fire. The water discharge trajectory in this case is a water discharge instruction that designates a curved trajectory without personal injury.

(Support control of water discharge operation based on identification result of image description sentence)
The judgment control unit 96 of the image analysis device 90 determines from the image description of the block image on the water discharge track generated by the second multilayer neural network 94 the obstacle on the water discharge track, the combustible near the fire source, the water discharge When the presence of a person on the orbit, the fire fighting activity of the person on the water discharge track, or the fallen person on the water discharge track is identified, these identification results are displayed, for example, on the screen of the display unit provided on the site control panel 16 Take control.

  For this reason, when the person in charge judges the start of water discharge by the water discharge gun and operates it, obstacles in the water discharge track, combustibles near the fire source, the presence of people in the water discharge track, fire extinguishing activities or water discharge of people on the water discharge track By displaying information such as the condition of the person on the orbit to the operator, the operator can perform judgment operation of the water discharge gun 10 promptly and appropriately to suppress and extinguish the fire.

(Second Multilayer Neural Network of Image Analysis Unit)
FIG. 12 is an explanatory view showing a functional configuration of a second multilayer neural network provided in the image analysis apparatus of FIG. As shown in FIG. 12, the second multilayer neural network 94 is composed of a convolutional neural network 108 and a recursive neural network 110.

(Convolutional neural network)
As shown in FIG. 12, the convolutional neural network 108 is composed of an input layer 112 and a plurality of intermediate layers 114.

  A conventional convolutional neural network, as shown in the multilayer neural network 40 of FIG. 6, repeats the input layer 66, the total combination 68, the intermediate layer 70 and the total combination 68 after the last middle layer of the convolutional neural network 60, For example, the image is clustered from the feature quantities of the image extracted by the convolutional neural network 60. However, in the present embodiment, the feature quantities of the input image are provided. Since it is only necessary to extract the following, the fully coupled neural network 61 in the latter stage is not provided, and as shown in FIG. 12, feature quantities obtained in an arbitrary intermediate layer 114 are input to the recursive neural network 110. Since the other parts are the same as the convolutional neural network 61 of FIG.

(Learning of convolutional neural networks)
In order to learn the second multilayer neural network 94 by the learning control unit 100 shown in FIG. 11, a learning data set consisting of a pair of block images located on the water discharge trajectory up to the fire source and their image description is used. The preparation is made to be stored in the learning data set storage unit 102.

  For learning of the second multilayer neural network 94 by the learning control unit 100, a multilayer neural network composed of the convolutional neural network 60 and the fully coupled neural network 61 shown in FIG. The learning block image is read out from the set storage unit 102 and input, and learning control by back propagation is performed.

  In back propagation, supervised learning when the data set of input value x and expected output value (expected value) y is given to the network and unsupervised learning when the input value x is given only to the network In this embodiment, for example, supervised learning is performed.

  When performing backpropagation in supervised learning, for example, a function of mean square error is used as an error for comparing values of estimated value y * and expected value y, which are the results of forward propagation passed through the network Do.

  In back propagation, using the magnitude of the error between the estimated value y * and the expected value y, the value is propagated from the back to the front of the network while correcting the weight and bias. The amount corrected for each weight and bias is treated as a contribution to the error, calculated by the steepest descent method, and the value of the weight and bias is changed to minimize the value of the error function.

  When learning control for the multi-layered neural network for learning shown in FIG. 6 is completed, the weights and biases generated in each layer of the already-convoluted convolutional neural network 60 are acquired as learned parameters, and the convolution of FIG. Transfer learning is performed, which is set in the neural network 108 and has been learned.

(Recursive neural network)
The recursive neural network 110 shown in FIG. 12 inputs the feature quantities of the block image extracted using the convolutional neural network 108 together with the word vector to estimate the image description.

  The recursive neural network 110 of the present embodiment uses a Long Short-Term Memory-Language Model (LSTM), which is a deep learning model corresponding to time-series data.

  A model of a normal recursive neural network is a model that is composed of an input layer, a hidden layer, and an output layer, and is a model that performs time series analysis using past history by using information of the hidden layer as an input of the next time. On the other hand, the LSTM model calculates the probability that each word will be selected as the t-th word from t-1 words that will be in the past context. That is, the LSTM model takes three inputs of word information of time 1 to t-1 that is hidden one time ago, time t-1 that is a prediction result of one time earlier, and external information, and sequentially Repeat the next word prediction to generate sentences.

The recursive neural network 110 of FIG. 12 converts the feature vectors of the image extracted by the convolutional neural network 108 into a matrix to be input to the LSTM hidden layer 118, and the word S stored in word units in the register 120. _The vector conversion unit 122 converts ₀ to _SN-1 into vectors WeS _{0 to} WeSN _-1 , converts the output of the LSTM hidden layer 118 of the N−1 stages and the LSTM hidden layer 118 to the appearance probability p _{1 to} p _N The probability conversion unit 124 is configured by a cost calculation unit 126 that calculates the cost from the probability of outputting a word by the cost functions log P ₁ (s ₁ ) to log p _N (S _N ) to minimize the cost.

(Learning of recursive neural networks)
The learning targets of the recursive neural network 110 are the vector conversion unit 122 and the LSTM hidden layer 118, and the extracted parameters (weights and biases) are used as they are for extracting the feature amount from the convolutional neural network 108.

The learning data is the word sequence {St} (t = 0,... N) of the learning image I and its action explanatory text, and the following procedure is performed.
(1) The image I is input to the convolutional neural network 108, and the output of a specific intermediate layer 114 is extracted as a feature vector.
(2) Input the feature vector to the LSTM hidden layer 118.
(3) The word string St is sequentially input from t = 0 to t = N-1, and a probability _{pt + 1} is obtained in each step.
(4) Minimize the cost obtained from the probability pt + 1 (St + 1) of outputting the word St + 1.

(Generation of image description)
When the image description of the input block image is estimated using the trained convolutional neural network 108 and the recursive neural network 110, the vector of feature values generated by inputting the block image into the convolutional neural network 108 is recursed. The word sequence is input to the type neural network 110, and word sequences are arranged in descending order of the product of the word appearance probability to generate an image description. The procedure is as follows.

(1) The image is input to the convolutional neural network 108, and the output of a specific intermediate layer 114 is extracted as a feature vector.
(2) Input a feature vector from the LSTM input layer 116 to the LSTM hidden layer 118.
(3) The sentence start symbol <S> is converted into a vector using the vector conversion unit 122 and input to the LSTM hidden layer 118.
(4) Since the occurrence probability of the word is known from the output of the LSTM hidden layer 118, the top M (for example, M = 20) words are selected.
(5) The word output in the previous step is converted into a vector using the vector conversion unit 122, and is input to the LSTM hidden layer 118.
(6) From the output of the LSTM hidden layer 118, the product of the probabilities of the words output so far is obtained, and the top M word strings are selected.
(7) Repeat the processes of (5) and (6) until the word output becomes a terminal symbol.

(Image analysis control)
FIG. 13 is a flow chart showing image analysis control by the image analysis apparatus of FIG. As shown in FIG. 13, in step S 21, the image analysis device 90 divides block images obtained by dividing the monitoring image captured by the monitoring camera 12 by the image input unit 92 based on the water discharge trajectory instruction information from the water discharge control unit 36. A block image based on the water discharge trajectory up to the fire source, for example, a block image on and around the water discharge trajectory is read and input to the second multilayer neural network 94 in step S22 to generate an image description, generated in step S23 The determined image description is held by the determination control unit 96.

  Subsequently, the process proceeds to step S24, the determination control unit 96 determines whether or not there is an obstacle on the water discharge trajectory by comparison with the registered words of the thesaurus dictionary 98, and proceeds to step S25 when it is determined that there is no obstacle. The water discharge control unit 36 is instructed to discharge water by a direct trajectory. On the other hand, when it is determined in step S24 that there is an obstacle on the water discharge path, the determination control portion 96 proceeds to step S26, and instructs the water discharge control portion 36 to discharge water by the curved injection path. In addition, even if it is determined that there is an obstacle on the water discharge path, the determination control unit 96 instructs the water discharge in the direct path if the obstacle can penetrate the water discharge.

  Subsequently, the determination control unit 96 proceeds to step S27, and in the case where it is determined from the image descriptive text that there is a flammable material in the vicinity of the fire source, proceeds to step S28 and instructs the water discharge control unit 36 to increase the water discharge amount. On the other hand, if the determination control unit 96 determines that there is no flammable material in the vicinity of the fire source in step S27, the process proceeds to step S29. If it determines that there is a nonflammable material in the vicinity of the fire source, the process proceeds to step S30. Instruct to reduce the amount of water discharge.

  Subsequently, the determination control unit 96 proceeds to step S31, and when it is determined from the image description sentence that there is a person on the water discharge path, proceeds to step S32 and instructs the water discharge control unit 36 to prohibit the water discharge start. Note that the determination of the presence of a person on the water discharge track in step S31 also includes the determination of an image description indicating an initial fire extinguishing activity.

  Subsequently, the water discharge control unit 96 proceeds to step S33, and when it is determined from the image description sentence that a person has fallen on the water discharge path, proceeds to step S34 and instructs the water discharge control unit 36 to start water discharge.

  Note that the analysis of the image description in the image analysis control of FIG. 13 and the control by the water discharge control unit 36 based on the analysis result are an example, and the contents of the image description generated from block images on the water discharge trajectory and its vicinity are various. In accordance with the content of the system, the optimum water discharge control will be set.

[Modification of the present invention]
(Discharge gun)
Although the above embodiment exemplifies a water gun performing horizontal turning and vertical turning, a horizontal turning is performed, but a water discharging gun dedicated to curved water discharge having a fixed vertical scanning angle may be installed without performing vertical turning. .

(Recognition of surveillance image) (This part shows claims 2 to 15 as non-block image.)
In the above embodiment, although the block image obtained by dividing the monitoring image is input to the multilayer neural network, the fire is detected by inputting the multilayer image to the multilayer neural network without dividing the monitoring image. Water for fire extinguishing may be discharged toward the source.

In this case, the learning image generation unit classifies the input image as a learning image into a flame learning image, a smoke learning image, a water discharge learning image, or a normal learning image, and stores the input image in the learning image storage unit.
The flame learning device learns the multi-layered neural network by inputting the flame learning image, the smoke learning image, the water discharge learning image, and the normal learning image stored in the learning image storage unit into the fire source detection device, and the fire source detection learned by the learning control unit The multi-layered neural network of the device inputs the monitoring image of the monitoring area, and detects and outputs a flame image, a smoke image, a water discharge image or a normal image.

  Further, as the estimation of the fire source position based on the flame image and the water discharge, the water discharge gun device discharges water toward the fire source estimated based on the flame image detected by the fire source detection device.

  Also, as the estimation of the fire source position based on the smoke image and the water discharge, the water discharge gun apparatus estimates the fire source based on the smoke image when the fire image is not detected by the fire source detection device and the smoke image is detected. Discharge water toward the

  In addition, as a judgment on the water discharge to the fire source, when the water discharge device starts water discharge toward the fire source position, the fire source is derived from the temporal change of the smoke image and the water discharge image detected by the fire source detection device. It is judged whether the water discharge has been hit or not and informed.

  In addition, as another example of the determination of the water discharge to the fire source, the multi-layered neural network of the water discharge device, the water discharge image when the water hits the fire source and the water discharge failure when the water discharge does not hit the fire source The image is learned from the image, and when the water discharge is started toward the fire source position, the water is discharged to the fire source from the judgment result of the water discharge in progress image and the water discharge incompetence image output by inputting the monitoring image into the multilayer neural network. It is judged whether or not it has been hit and informed.

  For example, when the water source hits the fire source, the smoke spreads by learning the spread of the smoke when there is no water discharge from the spread of smoke or the spread of the smoke when the water discharge is not hit, respectively. It can be judged whether or not the water discharge is hit by taking the feature as a feature.

  In addition, as feedback control during water discharge to the fire source, when the water discharge device determines that the water discharge does not hit the fire source, the water discharge is performed so that the judgment result that the water discharge hits the fire source is obtained. Control at least one of the direction and the discharge pressure.

  Further, as the configuration of the multi-layered neural network, the multi-layered neural network is configured of a feature extraction unit and a recognition unit, and the feature extraction unit receives input information and generates feature information from which the features of the input information are extracted. The recognition unit receives feature information output from the convolutional neural network, and indicates the probability of belonging to each class of flame image, smoke image, water discharge image, and normal image. A neural network is provided with a plurality of all coupled layers that output estimated values.

  Further, as generation of the image description, an image analysis device is further provided which inputs the monitoring image to the second multilayer neural network and outputs the image description indicating the content of the image.

  In addition, as water discharge trajectory control based on identification of obstacles, the image analysis device discharges water when it detects no obstacle or an obstacle through which water discharge penetrates from the image description of the image located on the water discharge trajectory up to the fire source. Instruct the gun device to direct water discharge at direct activation toward the fire source, and if the image description identifies an obstacle that can not be penetrated by the water discharge, instruct the water discharge gun device to discharge water at a curved start that bypasses the obstacle. .

  In addition, as the discharge amount control by the identification of the combustibles, the image analysis device instructs the water discharge gun device to increase the amount of water discharge when the combustibles located in the vicinity of the fire source are identified from the image description. If the incombustible material is identified from the above, instruct the water gun device to decrease the water discharge amount.

  When the image analysis device identifies a person located on the water discharge path from the image description to the fire source as water discharge stop control based on identification of a person, the water discharge gun device prohibits water discharge start or stops water discharge in water discharge To indicate.

  In addition, as water discharge stop control by identification of the person, the image analysis device instructs the water discharge gun device to prohibit the water discharge start when the fire fighting activity of the person located on the water discharge trajectory from the image description to the fire source is identified. When it is identified from the image description that a person has fallen, the water gun device is instructed to start water discharge.

  Moreover, as assistance of the water discharge operation based on the identification result of an image description sentence, a water discharge gun apparatus detects operation of an operation part and starts water discharge, and an image analysis apparatus is on a water discharge track from an image description sentence. Obstacles, combustibles near the fire source, the presence of people on the water discharge track, fire-fighting activities of people on the water discharge track or falling people on the water discharge track, the identification results are used to operate the water discharge device Display the screen on the display unit provided in the unit.

(Feature extraction)
In the above embodiment, the block image is input to the convolutional neural network to extract the feature due to fire, but it is not used the convolutional neural network, but the preprocessing such as extracting the feature such as the contour, the gradation from the input block image The predetermined feature may be extracted, and the image from which the feature has been extracted may be input to an all coupled neural network functioning as a recognition unit to estimate a fire. As a result, the processing load on feature extraction of block images can be reduced. The convolutional neural network is preferable because learning is faster than the neural network of full connection, but a neural network of full connection may be used.

(About the learning method)
Although the above embodiment performs learning by back propagation, the learning method of the multilayer neural network is not limited to this.

(Others)
Further, the present invention is not limited to the above-described embodiment, includes appropriate modifications that do not impair the objects and advantages thereof, and is not limited by the numerical values shown in the above-described embodiment.

10: Discharge gun 12: Surveillance camera 14: Control panel 16: Field control panel 18: Central control panel 20: Central control console 22: Pump 24: Pump control panel 26: Water distribution pipe 28: Air compressor 30: Air piping 32: Fire extinguishing Target area 34: Fire source detection device 36: Water discharge control unit 38, 92: Image input unit 40: Multilayer neural network 42, 96: Judgment control unit 44: Block classification image storage unit 46, 100: Learning control unit 48: Learning Image storage unit 50, 104: operation unit 52, 106: display unit 54: direct trajectory 56: curved trajectory 60: convolutional neural network 61: fully coupled neural network 62: input image 63, 65a, 65b: weight filter 64a, 64b, 64c: feature map 66, 112: input layer 68: total coupling 70, 114: middle layer 72: output layer 74-1, 7 -2: fire image 76: combustion material 78: flame 80: smoke 82-1, 82-2: block classification image 84: water discharge 90: image analysis device 94: second multilayer neural network 98: thesaurus dictionary 116: LSTM Input layer 118: LSTM hidden layer 120: register 122: vector converter 124: probability converter 126: cost calculator

Claims (16)

A fire source detection device for detecting a fire source by inputting a monitoring image of a caution area captured by an imaging unit into a multilayer neural network;
A water discharge gun device for discharging fire extinguishing water toward the fire source detected by the fire source detection device;
Water discharge type fire extinguishing equipment characterized by being provided.
In the water discharge type fire extinguishing equipment according to claim 1,
The fire source detection apparatus is characterized in that the surveillance image of the alert area captured by the imaging unit is divided into a plurality of block images, and the block image is input to the multilayer neural network to detect the fire source. Water extinguishing type fire extinguishing equipment.
In the water discharge type fire extinguishing equipment according to claim 2, further,
A learning image generation unit that divides an input image into a plurality of block images as a learning image and classifies the input image into a flame learning block image, a smoke learning block image, a water discharge learning block image, or a normal learning block image and stores it in a learning image storage unit ,
The flame neural network is trained by inputting the flame learning block image, the smoke learning block image, the water discharge learning block image, and the normal learning block image stored in the learning image storage unit into the fire source detection device. A learning control unit,
Is provided,
The multi-layered neural network of the fire source detection device learned by the learning control unit inputs the block image, detects whether it is a flame block, a smoke block, a water discharge block or a normal block and outputs it. Water extinguishing type fire extinguishing equipment.
In the water discharge type fire extinguishing equipment according to claim 3,
The water discharge type fire extinguishing facility, wherein the water discharge gun device discharges water toward a fire source estimated based on one or more of the flame blocks detected by the fire source detection device.
In the water discharge type fire extinguishing equipment according to claim 3,
The water discharge gun device discharges water toward the fire source estimated based on the smoke block, when the fire source detection device detects the one or more smoke blocks without detecting the fire block. Water discharge type fire extinguishing equipment characterized by
In the water discharge type fire extinguishing equipment according to claim 3,
When the water discharge device starts water discharge toward the fire source position, the water discharge hits the fire source from temporal changes in the smoke block and the water discharge block detected by the fire source detection device. A water discharge type fire extinguishing facility characterized by judging whether it is present and notifying.
In the water discharge type fire extinguishing equipment according to claim 1,
The multi-layered neural network of the above-mentioned water discharge gun device is learned by a water discharge in-use image when water discharge hits a fire source and a water discharge non-shooting image when water discharge does not hit a fire source,
When water discharge is started toward the fire source position, an image of the monitoring area is input to the multi-layered neural network, and it is determined whether or not the water discharge hits the fire source and notified. Water extinguishing type fire extinguishing equipment.
In the water discharge type fire extinguishing equipment according to claim 6 or 7,
The water discharge gun device controls at least one of the water discharge direction and the water discharge pressure so as to obtain the judgment result of the water discharge hit to the fire source when judging that the water discharge does not hit the fire source. Water discharge type fire extinguishing equipment characterized by having.
In the water discharge type fire extinguishing equipment according to claim 2,
The multi-layered neural network is
It consists of feature extraction unit and recognition unit,
The feature extraction unit may be a convolutional neural network including a plurality of convolutional layers that receives the input information and generates feature information from which features of the input information are extracted,
The recognition unit inputs the feature information output from the convolutional neural network, and outputs a plurality of estimated values indicating the probability of belonging to each class of the flame block, the smoke block, the water discharge block, and the normal block. A fire monitoring system characterized in that it is a neural network having a total connection layer of
The water discharge type fire extinguishing equipment according to claim 2, further comprising an image analysis device for inputting the block image into a second multilayer neural network and outputting an image explanatory text indicating the contents of the block image. Water discharge type fire extinguishing equipment characterized by.
In the water discharge type fire extinguishing equipment according to claim 10,
When the image analysis device identifies an obstacle without an obstacle or a water penetration object from the image description of the block image located on the water discharge path to the fire source, the water discharge device may be used as the fire source. Direct water discharge by direct activation, and when identifying an obstacle from which water discharge can not penetrate from the image description, water discharge device is instructed by water injection to bypass the obstacle. Water extinguishing type fire extinguishing equipment.
In the water discharge type fire extinguishing equipment according to claim 10,
When the image analysis apparatus identifies a combustible from the image description of the block image located in the vicinity of the fire source, the image analysis apparatus instructs the water discharge gun device to increase the amount of water discharge, and the incombustible from the image description A water discharge type fire extinguishing facility characterized by instructing the water discharge gun device to decrease the water discharge amount when it is identified.
In the water discharge type fire extinguishing equipment according to claim 10,
When the image analysis device identifies a person from the image description of the block image located on the water discharge path to the fire source, the image analysis device instructs the water discharge gun device to prohibit water discharge start or stop water discharge in water discharge Water discharge type fire extinguishing equipment characterized by
In the water discharge type fire extinguishing equipment according to claim 10,
When the image analysis device identifies a person's fire extinguishing activity from the image description of the block image located on the water discharge path to the fire source, the image analysis device instructs the water discharge gun device to prohibit water discharge start, and the image A water discharge type fire extinguishing facility characterized by instructing the water discharge gun device to start water discharge when it is identified from the description that a person has fallen.
In the water discharge type fire extinguishing equipment according to claim 10,
The water discharge gun device detects the operation of the operation unit and starts water discharge,
According to the description of the image block, the image analysis device is based on the description of the image block, the obstacle in the water discharge trajectory, the combustibles in the vicinity of the fire source, the presence of a person in the water discharge trajectory The water discharge type fire extinguishing equipment characterized by displaying said identification result on the screen provided in the operation part of the said water discharge gun apparatus, when the person who is identified is identified.
In the water discharge type fire extinguishing equipment according to claim 10,
The second multilayer neural network of the image analysis device is:
A convolutional neural network which extracts feature quantities from the input image of the monitoring area and outputs the extracted feature quantity from a predetermined intermediate layer;
A recursive neural network which receives the feature quantity output from the convolutional neural network and generates and outputs an image description of an image of the monitoring area;
Water discharge type fire extinguishing equipment characterized by being constituted by.

Images