JP7150934B2 - Fire detection device and fire detection method - Google Patents

Description

The present invention relates to a fire detection apparatus and fire detection method for detecting smoke caused by a fire from an image of a monitored area captured by a camera.

Conventionally, a photoelectric spot-type sensor, which detects smoke using the concentration of smoke around the sensor, is used to detect fire. The generated smoke stays before it reaches the detector installed at a high place such as the ceiling, making early detection difficult.

Therefore, if surveillance cameras installed in various locations can be used to detect smoke from images, it is thought that early detection of fires will be possible. Various devices and systems have been proposed for fire detection.

For this reason, in the conventional apparatus (Patent Document 1), as phenomena caused by the smoke accompanying the fire from the image, the decrease in transmittance or contrast, the convergence of the luminance value to a specific value, the narrowing of the luminance distribution range and the dispersion of luminance , a change in the average luminance value due to smoke, a decrease in the total amount of edges, and an increase in intensity in the low-frequency band.

JP 2008-046916 A JP-A-7-245757 JP 2010-238028 A

However, in the conventional fire detection system that detects fire from images of smoke accompanying fires, the entire image captured by a surveillance camera is processed to detect characteristic changes due to smoke to determine fires. Therefore, there is a problem that the processing load for judging the fire from the entire image increases and the processing takes time.

In smoke detection using surveillance cameras, it is important to detect fires at an early stage. There remains the problem that it is difficult to treat it in the same way as recognition processing.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a fire detection apparatus and a fire detection method that can reliably detect a smoldering fire or smoke generated in the early stages of a fire in an image captured by a surveillance camera, thereby making it possible to determine a fire.

(Fire detection device)
The present invention, in a fire detection device,
imaging means for outputting an image obtained by imaging a monitoring area;
an image input means for converting an image output from the imaging means into a grayscale as necessary and inputting the image;
ridge line density enhancement means for generating a ridge line density emphasized image by emphasizing ridge lines of density in the image by top hat processing from the image input by the image input means;
a fixed background image which is a ridgeline density-enhanced image generated by a ridgeline density emphasizing means by grayscaling an image of a state in which smoke is not generated and captured by an imaging means by an image input means as necessary; and ridgeline density enhancement. difference image generation means for generating a difference image obtained by taking the absolute value of the difference from the edge line density emphasized image generated by the means;
a foreground image extraction means for extracting a foreground image from the difference image generated by the difference image generation means by background subtraction;
fire determination means for determining fire based on the foreground image;
is provided.

(top hat processing)
The ridge line density enhancement means performs, as top-hat processing, contraction processing in which each pixel of the input image is treated as a pixel of interest, and the smallest pixel value within a reference range located within a predetermined circular filter is replaced with the pixel value of the pixel of interest. is performed a predetermined number of times, and then dilation processing is performed the same number of times to replace the largest pixel value in the reference range with the pixel value of the target pixel. generate an image.

Since the features of the fire detection method of the present invention are basically the same as those of the fire detection device described above, the description thereof will be omitted.

(basic effect)
In a fire detection apparatus, the present invention provides an imaging means for outputting an image obtained by imaging a monitoring area, an image input means for converting the image output from the imaging means into a gray scale as necessary, and an image input means. Ridge line density enhancement means for generating a ridge line density emphasized image by emphasizing ridge lines of density in the image from the input image by top hat processing; A difference obtained by taking the absolute value of the difference between the fixed background image and the edge line density emphasized image generated by the edge line density emphasizing means. Difference image generation means for generating an image, foreground image extraction means for extracting a foreground image from the difference image generated by the difference image generation means by background subtraction, and fire determination means for determining a fire based on the foreground image are provided. Therefore, the smoke generated in a smoldering fire or in the early stages of a fire starts from the fire source and rises due to the heat of the smoke particles. Therefore, when this model is viewed from the horizontal direction, the center line of the smoke becomes the density ridgeline, so it is possible to reliably extract the foreground image containing smoke from the image in which the density ridgeline is emphasized using top-hat processing. and

In addition, by generating a difference image that takes the absolute value of the difference between the fixed background image in a smoke-free state and the ridgeline density-enhanced image generated by top-hat processing, the locations where there is a large difference in density from the fixed background image are A grayscale image is generated that is white and black in small areas. By taking the difference in this way, the areas with large changes in gradation are emphasized, and the background model can be appropriately generated by the background subtraction performed next. A suitable foreground image can be extracted.

(Effect of top hat treatment)
Further, the edge line density emphasizing means, as top-hat processing, treats each pixel of the input image as a pixel of interest, and replaces the smallest pixel value within a reference range positioned within a predetermined circular filter with the pixel value of the pixel of interest. After performing the erosion process a predetermined number of times, the dilation process is performed the same number of times to replace the largest pixel value in the reference range with the pixel value of the target pixel. Since it is designed to generate a density-enhanced image, top-hat processing is one of morphological operations, and morphological operations focus on all pixels of a grayscale image, and refer to the pixel value of the target pixel and the pixel values near the target pixel. Among them, the process of replacing the largest pixel value with the pixel value of the pixel of interest is dilation processing, and similarly the process of replacing it with the smallest pixel value is erosion processing. , it is possible to emphasize the ridgeline of density due to smoke in the grayscale image. Further, by performing contraction processing and expansion processing in the reference range located within the circular filter, it is possible to sufficiently emphasize the ridge lines of the smoke while suppressing noise enhancement, thereby generating a ridge-line density-enhanced image.

(Effect of background subtraction method using mixed normal distribution)
Further, the foreground image extracting means generates a histogram of temporal change of each pixel as a mixed normal distribution based on the plurality of difference images for a predetermined time period generated by the difference image generating means, and uses the mixed normal distribution. Since the foreground image is extracted by background subtraction, each pixel of the difference image generated by the top hat process is observed for a certain period of time, and the probability that the pixel value change at each pixel is the background is learned from the temporal change of the pixel value. Each of the learned probabilities is modeled on a normal distribution, and the probability distribution that is the background of each pixel is a mixed normal distribution that combines these normal distributions. , is the foreground, and a foreground image containing smoke can be reliably extracted.

(Effect of additive synthesis of foreground images)
Further, the foreground image extracting means generates a foreground addition image by adding a plurality of foreground images extracted during a predetermined period of time, and the fire determination means determines a fire based on the foreground addition image. By adding the foreground image generated in, for example, one second, which is the most recent from the input image, the ridgeline image of the smoke included in the foreground image is further emphasized, and the fire is judged more accurately based on the ridgeline image of the smoke. becomes possible.

(Effect of resolution reduction by Gaussian pyramid)
In addition, the image input means further reduces the resolution of the image picked up by the image pick-up means to a predetermined value and inputs it to the ridge line density emphasis means. As for the pixel resolution, create multiple scale images with a Gaussian pyramid, convert to grayscale by paying attention to the scale at which changes in smoke are extracted, and scale to 1/4, for example, to obtain a resolution of 640 x 360 pixels. By reducing the number of pixels per frame, the arithmetic processing time of the computer device is shortened, and the smoke ridgeline image is generated by real-time processing for the video input captured by the surveillance camera. Make it possible to judge fire.

Explanatory drawing showing a monitoring area in which the fire detection device of the present invention is installed Explanatory diagram of the smoke model to be detected FIG. 1 is a block diagram showing an outline of the functional configuration of an image processing device; Explanatory diagram showing scaling with a Gaussian pyramid Illustration showing a circular filter used as a structuring element for top-hat processing Explanatory diagram showing a foreground image containing smoke generated by the image processing apparatus of FIG. Flowchart showing smoke monitoring processing by the image processing device of FIG.

[Outline of fire detection device]
FIG. 1 is an explanatory view showing through a monitoring area in which a fire detection device according to the present invention is installed.

As shown in FIG. 1, a monitoring camera 10 functioning as an imaging unit and a lighting device 20 are installed in a monitoring area 18, and the monitoring area 18 is captured by the monitoring camera 10. FIG.

The surveillance camera 10 is installed, for example, in the center of an upper corner of a surveillance area (monitoring space) 18 partitioned vertically, horizontally, and forward and backward, and is arranged with its imaging optical axis obliquely downward so that the surveillance area 18 can be imaged as a whole. and

The monitoring camera 10 captures moving images of the monitoring area 18. The image capturing size is, for example, 1280×720 pixels, and outputs a moving image consisting of a series of color frame images at a frame rate of 30 frames per second or 60 frames per second.

Combustible material such as a trash can placed in the monitoring area 18 is in a situation where a smoldering fire occurs for some reason, and smoke 24 is rising from the fire source 22 . A moving image captured by the surveillance camera 10 is transmitted to an image processing device 12 installed in a janitor's room or the like via a transmission line, and smoke 24 rising from a fire source 22 such as a garbage can is detected by image processing, and a fire is detected. is determined, and a fire detection signal is output to the fire alarm equipment 14 to output a fire alarm.

[Smoke detection principle]
The principle of detecting smoke according to this embodiment will be described as follows. FIG. 2 is an explanatory diagram of a smoke model to be detected. FIG. 2(A) shows the smoke model, and FIG. 2(B) shows the pixel density distribution viewed from the horizontal direction.

As shown in FIG. 1, smoke 24 generated in a smoldering fire or in the early stages of a fire rises from the fire source 22 due to heat, so as shown in FIG. A smoke model 38 is conceivable in which a columnar object extends upward while fluctuating. Therefore, when the smoke model 38 is viewed from the horizontal direction indicated by XX, the centerline of the smoke model 38 becomes the density ridge. Therefore, the image processing apparatus 12 of the present embodiment extracts smoke from a moving image by emphasizing the density ridgeline in the image.

For edge enhancement processing, top hat processing, which is one of morphological operations, is used instead of edge extraction such as a Sobel filter, and background subtraction using mixed normal distribution is used for extraction processing.

[Fire detection device]
(Functional configuration of fire detection device)
FIG. 3 is a block diagram showing an outline of the functional configuration of the fire detection device according to the present invention, FIG. 4 is an explanatory diagram showing scaling with a Gaussian pyramid, and FIG. 5 shows a circular filter used as a structuring element for top-hat processing. FIG. 6 is an explanatory diagram showing a foreground image containing smoke generated by the image processing apparatus of FIG.

As shown in FIG. 3, the fire detection device is composed of a monitoring camera 10 and an image processing device 12. As shown in FIG. The image processing apparatus 12 is composed of a CPU, a memory, a computer circuit having various input/output ports, etc. as its hardware, and has an image input function that functions as image input means as a function realized by execution of a program by the CPU. 26, edge line density enhancement section 28 functioning as edge line density enhancement means, difference image generation section 30 functioning as difference image generation means, foreground image extraction section 32 functioning as foreground image extraction means, and fire determination functioning as fire determination means. A unit 34 and a control unit 36 are provided.

The monitoring camera 10 functioning as imaging means operates in response to an instruction from the control unit 36, transmits color moving image data of the monitoring area at, for example, 30 frames per second or 60 frames per second. stored in a memory (not shown).

(Image input section)
The image input unit 26 grayscales the frame images of the color moving image from the monitor camera 10 and outputs them to the ridge line density enhancement unit 28 . Here, the size of the color frame image captured by the surveillance camera has a resolution of, for example, 1280×720 pixels. , grayscaling with attention to the scale at which the change in smoke is extracted, for example, as shown in FIG. 42 is generated and output to the edge line density emphasizing unit 28 .

By reducing the number of pixels per frame by generating the 1/4 grayscale image 42 by the image input unit 26 as described above, the arithmetic processing time of the image processing device 12 using a computer circuit is shortened, and the surveillance camera 10 Fire can be determined by generating a ridge line image of smoke by real-time processing for moving image input captured by .

(Edge line density emphasis part)
The edge line density emphasizing section 28 generates a edge line density emphasized image by emphasizing a density edge line in the image by top-hat processing for each grayscale frame image input from the image input section 26 .

The top-hat processing used in the edge line density enhancement unit 28 is one of morphological operations. The morphological operation focuses on all pixels of the grayscale image, refers to the pixel value of the target pixel and the pixel values near the target pixel, and calculates the The process of replacing the largest pixel value with the pixel value of the pixel of interest is dilation processing, and the process of replacing it with the smallest pixel value is erosion processing. By taking the difference between the obtained image and the original grayscale image, it is possible to emphasize the ridgeline of density due to smoke in the grayscale image.

When an input image is shrunk a predetermined number of times by top-hat processing and then expanded the same number of times, the reference range of pixels to be referred to for the pixel of interest is a filter defined by a shape such as a rectangle, circle, or cross called a structuring element. Various processing results can be obtained by using the shape and size that match the processing target.

In the present embodiment, in order to determine the optimum filter shape and size of the structuring element, the filter shape is rectangular or circular, and has odd sizes such as 5×5 pixels, 25×25 pixels, and 35×35 pixels. I've tried several combinations of

Here, taking 5×5 as an example, the circular filter 46 is a stepped circle determined by a circle with a diameter of 5 pixels inscribed in a 5×5 square as shown in FIG. There are 12 reference pixels 50 surrounding the perimeter of .

As a result of the trial, the rectangular filter emphasized the ridge of smoke more than the circular filter, but at the same time, it also emphasized a lot of noise. Since the enhancement was sufficient, it was decided to adopt a circular filter in the present embodiment, and it was decided to use a circular filter with a size of 31×31 pixels as the structuring element, which gave the best results. there is

(difference image generator)
Prior to the use of the device, the difference image generation unit 30 generates a 1/4 gray scale image by the image input unit 26 for the image captured by the monitoring camera 10 in a state where smoke is not generated in the monitoring area 18, A ridge line density emphasized image generated using the top hat process of the ridge line density emphasis unit 28 is stored in advance as a fixed background image.

In this state, the difference image generation unit 30 generates a difference image obtained by taking the absolute value of the difference from the pre-stored fixed background image each time the edge line density emphasized image generated by the edge line density emphasis unit 28 is input. It is generated and output to the foreground image extraction unit 32 .

By generating the difference image between the edge-line density-enhanced image and the fixed background image in the difference image generation unit 30 in this way, a grayscale image is generated in which portions where the difference in density from the fixed background image is large are white and portions where the difference is small are black. Then, the background difference using the mixed normal distribution performed by the foreground image extraction unit 32 generates a background model from the grayscale change of the pixel value. and the background model can be generated properly.

(foreground image extractor)
The foreground image extraction unit 32 generates a histogram of temporal change of each pixel as a mixed normal distribution based on the plurality of difference images for a predetermined time period generated by the difference image generation unit 30, and uses the mixed normal distribution. Extract the foreground image by background subtraction.

The background subtraction method for extracting the foreground image by background subtraction using the mixed normal distribution of the foreground image extraction unit 32 is a method proposed by Zivkovic et al. For each pixel, the probability that the change in pixel value is background is learned from the change in target. Each of the learned background probabilities uses a normal distribution as the background model, and the probability distribution of each pixel is a mixed normal distribution that includes those normal distributions. , is the foreground, and the foreground is extracted.

That is, the background subtraction method using the mixed normal distribution is an algorithm for segmenting the foreground and background areas based on the background normal distribution, and each pixel belonging to the background is modeled by the mixed normal distribution with the mixture number K of 3 to 5. It is known as a method of In particular, the technique by Zivkovic et al. is characterized in that the optimum number of mixtures K is selected for each pixel. The process generates a background model from the grayscale variation of pixel values for each pixel of the difference image, then, for each input of the difference image, generates a mask of the foreground region using the background model, and generates a mask of the foreground region. Extract images.

In this embodiment, several trials were conducted on the number of frames for learning the background model and the value σ of the variance of the normal distribution. The case was 7 frames and a variance of σ 2 =6 was used.

Here, as for the number of frames, the number of frames required for the mass of smoke to rise and the cycle of flickering of the lighting are predicted from the actual moving image, and the length including the flickering cycle of the lighting corresponding to 50 Hz or 60 Hz and the smoke The number of frames that is shorter than the number of frames it takes for the cluster to rise results in 15 frames at 60 frames per second and 7 frames at 30 frames per second.

Further, the foreground image extracting unit 32 generates a foreground addition image by adding foreground images generated during a predetermined period of time, for example, one second from the current input image, and generates the next image based on the foreground addition image. The fire determination unit 34 may be made to determine a fire. By adding the foreground images generated, for example, within one second from the current input image in this way, the ridge line image of the smoke included in the foreground image is further emphasized, and the judgment of the fire based on the ridge line image of the smoke is performed. It can be done more accurately.

Through such processing by the foreground image extraction unit 32, for example, as shown in FIG. 6, a foreground extraction image in which the ridgeline of the smoke is emphasized can be obtained.

(Fire determination unit)
The fire determination unit 34 receives a foreground image in which the ridgeline of the smoke is emphasized as shown in FIG. to output a fire alarm.

As a method for judging a fire by the fire judging unit 34, there is a method for judging a fire when the cumulative value of the smoke image is equal to or higher than a predetermined threshold, and a method for obtaining the rising speed of the smoke from the time-series change of the smoke image and determining the threshold speed or higher. A fire is determined using various characteristics of smoke unique to fire, such as a method of determining a fire when the smoke is falling, a method of determining a fire by detecting the angle at which the smoke spreads upward, and the like.

(Smoke monitoring process)
FIG. 7 is a flow chart showing smoke monitoring processing by the image processing apparatus of FIG.

As shown in FIG. 7, in step S1, the control unit 36 reads a color moving image of the surveillance area captured by the surveillance camera 10, that is, a color frame image into the image input unit 26, and in step S2, as shown in FIG. A 1/4 grayscale frame image is generated by the Garcian pyramid, and in step S3, the edge line density enhancement unit 28 is instructed to generate a edge line density emphasized image in which the edge lines in the frame image are emphasized by top hat processing.

Subsequently, the control unit 36 proceeds to step S 4 to generate a difference image based on the difference in pixel absolute value between the fixed background image stored in advance and the input edge line density emphasized image, and outputs the difference image to the foreground image extraction unit 32 .

Subsequently, the control unit 36 advances to step S5 and instructs the foreground image extraction unit 32 to generate a foreground mask from the difference image by background subtraction using a mixed normal distribution to extract the foreground image, and step S6. In step S7, the foreground images from the last predetermined time before are additively synthesized to generate a foreground added image, and in step S7, the fire is determined from the unique characteristics of the smoke of the fire. Note that it is not always necessary to generate the foreground addition image in step S6, and the processing in step S6 may be skipped.

Subsequently, the control unit 36 proceeds to step S8, and when it determines that there is smoke, proceeds to step S9 to transmit a fire detection signal to the fire alarm equipment 14 to output a fire alarm or fire preliminary alarm (pre-alarm). If the control unit 36 determines that there is no smoke in step S8, the process returns to step S1 to process the next frame image.

[Modification of the present invention]
(Surveillance camera)
In the above-described embodiment, since a monitoring camera that outputs color moving images is used, the image input unit 26 converts color frame images into grayscale images. , grayscaling at the image input section is not required.

(Grayscale by Gaussian pyramid)
In the above embodiment, the color frame image input from the surveillance camera is converted into a 1/4 grayscale image by the Gaussian pyramid to lower the resolution, but the processing speed of the computer circuit used in the image processing device 12 is sufficient. If , no scaling with a Gaussian pyramid is necessary, just convert to a grayscale image.

(others)
Moreover, the present invention is not limited to the above-described embodiments, includes appropriate modifications that do not impair the objects and advantages thereof, and is not limited by the numerical values shown in the above-described embodiments.

10: Surveillance camera 12: Image processing device 14: Fire alarm equipment 18: Monitoring area 20: Lighting equipment 22: Fire source 24: Smoke 26: Image input unit 28: Edge line density emphasis unit 30: Difference image generation unit 32: Foreground image Extraction unit 34: Fire determination unit 36: Control unit 38: Smoke model 40: Input frame image 42: 1/4 gray scale image 46: Circular filter 48: Pixel of interest 50: Reference pixel

Claims (2)

imaging means for outputting an image obtained by imaging a monitoring area;
an image input means for converting the image output from the imaging means into a grayscale as necessary and inputting the image;
ridge line density enhancement means for generating a ridge line density emphasized image by emphasizing ridge lines of density in the image by top hat processing from the image input by the image input means;
a fixed background image, which is a ridgeline density emphasized image generated by the ridgeline density emphasizing means by grayscaling the image of a state in which smoke is not generated and captured by the imaging means by the image input means as necessary; difference image generating means for generating a difference image obtained by taking the absolute value of the difference from the edge line density emphasized image generated by the edge line density emphasizing means;
a foreground image extracting means for extracting a foreground image from the differential image generated by the differential image generating means by background subtraction;
fire determination means for determining a fire based on the foreground image;
provided ,
As the top-hat processing, the edge line density emphasizing means sets each pixel of the input image as a pixel of interest, and sets the smallest pixel value in a reference range located within a predetermined circular filter as the pixel value of the pixel of interest. After performing the erosion process for replacing with a predetermined number of times, the image generated by performing the same number of expansion processes for replacing the largest pixel value in the reference range with the pixel value of the target pixel and the input original image. A fire detection device , wherein the ridgeline density-enhanced image is generated by taking a difference .
Outputting an image obtained by imaging the monitoring area by the imaging means,
inputting the image output from the imaging means by converting it into a gray scale as necessary by an image input means;
edge line density emphasizing means for generating a ridge line density emphasized image by emphasizing density ridge lines in the image by top hat processing from the image input by the image input means;
A ridgeline density-enhanced image generated by the ridgeline density emphasizing means by converting the image of a state in which no smoke is generated by the imaging means into a gray scale as required by the image input means by means of the difference image generating means. generating a differential image obtained by taking an absolute value of a difference between a certain fixed background image and the edge line density emphasized image generated by the edge line density emphasizing means;
a foreground image extracting means for extracting a foreground image from the difference image generated by the difference image generating means by background subtraction;
Fire determination means determines a fire based on the foreground image,
As the top-hat processing of the edge line density emphasizing means, each pixel of the input image is set as a pixel of interest, and the smallest pixel value within a reference range located within a predetermined circular filter is set as the pixel value of the pixel of interest. After performing the replacement erosion process a predetermined number of times, the difference between the image generated by performing the same number of expansion processes of replacing the largest pixel value in the reference range with the pixel value of the target pixel and the input original image. A fire detection method , wherein the edge line density-enhanced image is generated by taking

Images