JP7143174B2 - Smoke detection device and smoke identification method - Google Patents

Description

TECHNICAL FIELD The present invention relates to a smoke detection device and a smoke identification method for detecting smoke caused by a fire from images of a surveillance area captured by a camera.

Conventionally, a photoelectric spot-type sensor, which detects smoke using the concentration of smoke around the sensor, is used to detect fire. The generated smoke stays before it reaches the detector installed at a high place such as the ceiling, making early detection difficult.

Therefore, if surveillance cameras installed in various places can be used to detect smoke from images, it is thought that early detection of fires will be possible. Various devices and systems have been proposed for fire detection.

For this reason, in the conventional apparatus (Patent Document 1), as phenomena caused by the smoke accompanying the fire from the image, the decrease in transmittance or contrast, the convergence of the luminance value to a specific value, the narrowing of the luminance distribution range and the dispersion of luminance , a change in the average luminance value due to smoke, a decrease in the total amount of edges, and an increase in intensity in the low-frequency band.

JP 2008-046916 A JP-A-7-245757 JP 2010-238028 A

However, in such conventional fire detection systems that detect fires from images of smoke associated with fires, the entire image captured by a surveillance camera is processed to detect characteristic changes due to smoke to determine whether a fire has occurred. Therefore, there is a problem that the processing load for judging the fire from the entire image increases and the processing takes time.

In smoke detection using a surveillance camera, there is a phenomenon similar to smoke in the monitored area, which is steam emitted from various cooking appliances such as pots and humidifiers. The problem remains that there is a high possibility of erroneously judging smoke from a fire.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a smoke detection apparatus and a smoke identification method that can distinguish between steam and smoke caused by a fire in an image captured by a surveillance camera, thereby enabling reliable smoke determination.

(smoke detector)
The present invention, in a smoke detection device,
imaging means for outputting images obtained by sequentially imaging a monitoring area;
candidate area extracting means for extracting a smoke candidate area for each image output from the imaging means;
an optical flow generation means for setting a plurality of candidate points in a smoke candidate region and generating an optical flow based on a vector indicating temporal motion for each candidate point;
clustering processing means for clustering the optical flow into one or more clusters on a coordinate plane having two-dimensional coordinates of the x component and the y component of the movement amount, and obtaining an average velocity and an amount of variation in velocity for each cluster;
a smoke judgment means for comparing the average velocity and the amount of velocity variation of a cluster with a preset smoke estimation region on a two-dimensional coordinate plane of the x-axis of the average velocity and the y-axis of the velocity variation, and outputting the comparison result; It is characterized by being provided. The output of the comparison result means an output indicating whether the cluster is a smoke area or a non-smoke area.

(candidate area extraction)
The candidate region extracting means is
a difference image generation means for generating a difference image by subtracting the moving average image from the image output from the imaging means;
binarized image generating means for binarizing the difference image generated by the difference image generating means into smoke pixels when the pixel luminance is equal to or higher than a predetermined threshold value or exceeding the threshold value, and into background pixels otherwise;
is provided.

(candidate area extraction)
The differential image generating means adds a value obtained by multiplying the value obtained by multiplying the current input image by a predetermined update rate to the value obtained by subtracting the update rate from 1 to the previous moving average image. to generate.

(Determination of Effective Optical Flow)
The optical flow generating means, when generating the optical flow of the candidate point, determines that it is an effective optical flow when it exists in a predetermined smoke determination region, and outputs it to the clustering processing means.

(Clustering method)
The clustering processing means clusters the optical flow by a predetermined variational mixture Gaussian distribution method (VBGMM method), a clustering method using a Gaussian mixture model, a k-means method (k-means method), or a mean shift method (MeanShift method). to generate one or more clusters.

(Smoke estimation area)
The smoke determination means, on a two-dimensional coordinate plane, defines a smoke estimation area lower limit line that starts from a predetermined minimum value of an average velocity near the origin and rises with a predetermined slope and then becomes a constant slope; It is set between the estimated smoke area upper limit line that rises with a predetermined gradient larger than the estimated smoke area lower limit line, starting from a predetermined minimum value of the amount of velocity variation near the origin.

(Cluster weight setting)
Smoke determination means
weight setting means for setting a predetermined weight to the generated predetermined number of clusters;
a determination value calculating means for obtaining a value obtained by multiplying the number of optical flows included in the cluster by a weight for each cluster, averaging the total sum, and calculating a determination value;
Smoke area determining means for determining a cluster as a smoke area when the determination value is equal to or greater than a predetermined determination threshold or exceeds the determination threshold;
is provided.

(Cluster weight setting 1)
Weight setting means
A weight is set according to the distance from the lower limit of the smoke estimation region of the cluster position plotted by the average velocity and the amount of velocity variation on the two-dimensional coordinate plane.

(Cluster weight setting 2)
Weight setting means
If the position of the cluster plotted on the two-dimensional coordinate plane by the average speed and the amount of speed variation is in the smoke estimation region, a weight of 1 is set, and if it is not in the smoke estimation region, a weight of 0 is set.

(Cluster weight setting 3)
Weight setting means
A value obtained by inputting the distance from the lower limit of the smoke estimation region of the cluster position plotted on the two-dimensional coordinate plane by the average speed and the amount of speed variation to a predetermined evaluation function including the tanh function or the sigmoid function is set as the weight. do.

(Cluster weight setting 4)
The weight setting means calculates a two-dimensional probability in a range of a predetermined average velocity and velocity variation around the position of the cluster with respect to a two-dimensional probability density function of smoke obtained in advance corresponding to the average velocity and the velocity variation. The cluster weight is set to the probability of being smoke calculated by integration of the density function.

(Cluster weight setting 5)
The weight setting means includes a lookup table in which the two-dimensional probability density function of smoke obtained in advance corresponding to the average speed and the amount of speed variation is set, and the two values obtained by referring to the lookup table according to the cluster position are calculated. Set the output of the dimensional probability density function to the cluster weights.

(Smoke identification method)
The present invention provides a smoke identification method comprising:
The imaging means sequentially captures images of the monitored area,
extracting a smoke candidate region for each image output from the imaging means by the candidate region extraction means;
setting a plurality of candidate points in the smoke candidate region by an optical flow generating means, generating an optical flow based on a vector indicating temporal motion for each candidate point;
clustering the optical flow into one or more clusters on a coordinate plane having two-dimensional coordinates of the x component and the y component of the movement amount by clustering processing means, obtaining an average velocity and an amount of variation in velocity for each cluster;
The smoke judging means compares the average speed and the speed variation of the cluster with the preset smoke estimation region on the two-dimensional coordinate plane of the x-axis of the average speed and the y-axis of the speed variation, and outputs the comparison result. .

(basic effect)
A smoke detection apparatus according to the present invention includes an imaging means for outputting an image obtained by sequentially imaging a monitoring area, a candidate area extracting means for extracting a smoke candidate area for each image output from the imaging means, and a smoke candidate area. and an optical flow generating means for generating an optical flow based on a vector indicating temporal movement for each candidate point, and two-dimensional coordinates of the x and y components of the amount of movement. A clustering processing means for clustering into one or more clusters on a coordinate plane and obtaining an average speed and a speed variation amount for each cluster; A smoke judgment means is provided to compare the set smoke estimation region, the average velocity of the cluster, and the amount of velocity variation, and output the comparison result. Furthermore, the optical flow is clustered by a statistical method to generate a cluster that becomes a smoke group, and by evaluating the relationship with the smoke estimation area set in advance on the two-dimensional coordinate plane, the cluster of the optical flow due to steam is obtained. It is possible to identify clusters of optical flow due to smoke and smoke, and it is possible to reliably identify steam (steam) and smoke due to fire in the image captured by the surveillance camera, and to make a reliable fire judgment.

(Effect of candidate region extraction)
Further, the candidate area extracting means includes a difference image generating means for generating a difference image by subtracting the moving average image from the image input by the preprocessing means, and a pixel brightness of the difference image generated by the difference image generating means. Since the binarized image generation means is provided for binarizing smoke pixels when the threshold value is equal to or exceeds the threshold value, and binarizes the background pixels otherwise, the background image without movement in the captured surveillance image is generated. By generating the removed difference image and further binarizing the pixels of the difference image with a predetermined brightness or higher as smoke pixels, it is possible to extract the smoke candidate region from which the background component is reliably removed.

(Effect of candidate region extraction)
Further, the differential image generating means obtains the moving average image by multiplying the value obtained by multiplying the current input image by a predetermined update rate, and multiplying the previous moving average image by the value obtained by subtracting the update rate from 1. Since the difference image is generated by subtracting the past moving average luminance from the latest pixel luminance for each pixel, the difference image is generated. Then, the difference image can be generated with high accuracy.

(Effect of determination of effective optical flow)
In addition, when the optical flow of the candidate point is generated, the optical flow generation means determines that the optical flow is effective when the optical flow exists in the predetermined smoke determination area, and outputs the optical flow to the clustering processing means. With respect to the two-dimensional coordinates of the monitoring screen, the smoke judgment area where the optical flow of smoke exists is known in advance with respect to the area where the optical flow of people and stationary objects exists. By judging that it exists in the judgment region and performing the next clustering, it is possible to eliminate the need for clustering for optical flows that are clearly not smoke, reduce the processing load, and increase the accuracy of identifying smoke regions by clustering. .

(Effect of clustering method)
Also. The clustering processing means applies the optical flow to a predetermined variational mixture Gaussian distribution method (VBGMM method), a clustering method using a Gaussian mixture model, a k-means method (k-means method), or a mean shift method (MeanShift method). Clusters are generated by clustering using the method used, so highly reliable algorithms with open libraries can be easily obtained and used.

(Effect of smoke estimation area)
In addition, the smoke determination means defines, on a two-dimensional coordinate plane, the smoke estimation area lower limit line that rises with a predetermined slope starting from a predetermined minimum value of the average velocity near the origin and then becomes a constant slope. , and the smoke estimation region upper limit line rising with a predetermined slope larger than the smoke estimation region lower limit line starting from a predetermined minimum value of degree variation near the origin. By setting by performing optical flow generation and clustering using images, steam images, images of people working and living, etc., smoke with smoke clusters that do not include steam optical flow clusters By setting the estimated area with high accuracy and comparing it with the cluster, it is possible to determine whether or not it is a smoke area with high accuracy.

(Effect of cluster weight setting)
Further, the smoke determination means includes a weight setting means for setting a predetermined weight to a predetermined number of generated clusters, and a value obtained by multiplying the number of optical flows included in the cluster by the weight for each cluster and averaging the total sum. and a smoke area determination means for determining a cluster as a smoke area when the determination value is equal to or greater than a predetermined determination threshold value or exceeds the determination threshold value. and the estimated smoke area can be quantified to reliably determine whether or not it is a smoke area.

(Effect of cluster weight setting 1)
In addition, the weight setting means sets the weight according to the distance from the lower limit of the smoke estimation region of the cluster position plotted on the two-dimensional coordinate plane by the average speed and the speed variation amount. If the distance between the plotted cluster position and the lower limit of the smoke estimation region is short, a small weight is set, and if the distance is long, a large weight is set. You can increase your chances of being judged.

(Effect of cluster weight setting 2)
Further, the weight setting means sets a weight of 1 if the position of the cluster plotted by the average speed and the amount of speed variation on the two-dimensional coordinate plane is in the smoke estimation region, and sets a weight of 0 if it is not in the smoke estimation region. , the setting of the weight is simple, and the processing load of fire determination in the smoke candidate area can be reduced.

(Effect of cluster weight setting 3)
Further, the weight setting means inputs the distance to the lower limit of the smoke estimation region of the cluster position plotted by the average speed and the speed variation on the two-dimensional coordinate plane into a predetermined evaluation function including a tanh function or a sigmoid function. Since the obtained values are set as weights, it is easy to set weights to determine whether or not they are in the smoke estimation region. By using the value obtained by inputting the function or the evaluation function such as the sigmoid function as the weight, it prevents overestimation of the data that is far from the fire source in the smoke estimation area and adversely affects the judgment. It is possible to improve the accuracy of determining whether or not it is a smoke area based on the weight setting.

(Effect of cluster weight setting 4)
In addition, the weight setting means sets the two-dimensional probability density function of the smoke obtained in advance corresponding to the average velocity and the amount of velocity variation to two weights in the range of the predetermined average velocity and the amount of velocity variation around the position of the cluster. Since the probability of being smoke calculated by integration of the dimensional probability density function is set as the weight of the cluster, the smoke estimation region set on the two-dimensional coordinate plane by the average speed and the amount of speed variation is calculated in advance in two dimensions. When the probability density function is obtained, the probability of being smoke is obtained from the generated cluster, and by using this as a weight, it is possible to judge whether or not the area is a smoke area with higher accuracy.

(Effect of cluster weight setting 5)
Further, the weight setting means has a lookup table in which the two-dimensional probability density function of smoke obtained in advance corresponding to the average speed and the amount of speed variation is set, and is obtained by referring to the lookup table according to the position of the cluster. Since the output of the two-dimensional probability density function is set as the weight of the cluster, the two-dimensional probability density function of the smoke estimation area set on the two-dimensional coordinate plane by the average speed and the amount of speed variation is obtained in advance. If a cluster is generated by preparing a lookup table in which the values of the two-dimensional probability density function corresponding to the average speed and the amount of speed variation are set, on the two-dimensional coordinate plane By obtaining the output of the value of the two-dimensional probability density function read by referring to the lookup table based on the average speed and the amount of speed variation in a predetermined range centered on the position of , the probability of being smoke is easily obtained and the weight is set. can do.

(Smoke identification method)
Another aspect of the present invention is a smoke identification method, in which images of a monitoring area are sequentially captured by an imaging means, smoke candidate areas are extracted for each image output from the imaging means by a candidate area extraction means, and optical The flow generation means sets a plurality of candidate points in the smoke candidate region, generates an optical flow based on the vector indicating the temporal movement for each complementary point, and clusters the optical flow by the movement amount x On a coordinate plane having two-dimensional coordinates of the component and the y component, clustering is performed into one or more clusters, the average velocity and the amount of variation are obtained for each cluster, and the x-axis of the average velocity and the y-axis of the amount of velocity variation are determined by the smoke judgment means. On the two-dimensional coordinate plane of , the average speed and speed variation of the preset smoke estimation area and the cluster are compared and the comparison result is output. be able to.

Explanatory drawing showing through the monitoring area where the smoke detection device is installed Block diagram showing an outline of the functional configuration of the smoke detection device Block diagram showing the functional configuration of the candidate region extraction unit Explanatory diagram showing multiple optical flows generated from candidate regions and their grouping Explanatory diagram showing smoke estimation area in cluster coordinates Explanatory diagram showing detection areas for smoke, steam, people, etc. Explanatory diagram showing the process of setting weights according to the distance from the lower limit of the smoke estimation region to multiple clusters generated in time series. Flowchart showing processing control of the smoke detector

[Outline of smoke detector]
FIG. 1 is an explanatory view showing a monitoring area in which a smoke detection device is installed. As shown in FIG. 1, a monitoring camera 10 functioning as an imaging means is installed in a monitoring area 16, and the monitoring area 16 is captured by the monitoring camera 10. As shown in FIG.

The surveillance camera 10 is installed, for example, in an upper corner of a surveillance area (monitoring space) 16 partitioned vertically, horizontally, and forward and backward, and can image the surveillance area 16 as a whole by arranging its imaging optical axis obliquely downward. and

The monitoring camera 10 captures a moving image of the monitoring area 16. The image capturing size is, for example, 1280×720 pixels, and outputs a moving image consisting of a series of color frame images at a frame rate of 30 frames per second or 60 frames per second.

The combustible material of the fire source 18 placed in the monitoring area 16 is in a state of fire for some reason, and smoke 20 is rising from the fire source 18 . Also, steam 24 is rising from the pot 22 placed in the monitoring area 16 .

A moving image captured by the surveillance camera 10 is transmitted through a transmission line to a smoke detector 12 installed in a janitor's room or the like. First, the smoke 20 rising from the fire source 18 is detected, and a smoke detection signal is output to the fire alarm equipment 14 to output a fire alarm.

[Outline of smoke detector]
FIG. 2 is a block diagram showing an outline of the functional configuration of the smoke detection device. As shown in FIG. 2, the smoke detection device 12 is composed of a CPU, a memory, a computer circuit having various input/output ports, etc. as its hardware. A preprocessing unit 26 functioning as processing means, a candidate region extraction unit 28 functioning as candidate region extraction means, a feature amount extraction unit 30 functioning as feature amount extraction means, a smoke determination unit 32 and control unit 34 functioning as smoke determination means. is provided.

The feature quantity extraction unit 30 is provided with an optical flow generation unit 36 functioning as optical flow generation means and a clustering processing unit 38 functioning as clustering processing means. Further, the smoke determination section 32 is provided with a weight setting section 40 functioning as weight setting means, a determination value calculation section 42 functioning as determination value calculation means, and a determination section 44 functioning as determination means. A detailed description will be given below.

[Pretreatment part]
The preprocessing unit 26 grayscales the frame images of the color moving image from the monitoring camera 10 and outputs them to the candidate area extracting unit 28 . Here, the size of the color frame image captured by the surveillance camera 10 is, for example, a resolution of 1280×720 pixels, and processing can be performed even at this resolution. For the input frame image, create multiple scale images using, for example, a Gaussian pyramid, convert to grayscale by focusing on the scale where changes in smoke are extracted, and scale the input frame image to 1/4 to 640 x 360 pixels, for example. A 1/4 gray scale image with a resolution of 1/4 may be generated and output to the candidate area extraction unit 28 .

By reducing the number of pixels per frame by generating a 1/4 grayscale image by the preprocessing unit 26, the arithmetic processing time of the smoke detection device 12 using a computer circuit is shortened, and the surveillance camera 10 A fire can be determined by generating a smoke ridge line image by real-time processing for a captured moving image input.

[Candidate region extraction unit]
FIG. 3 is a block diagram showing the functional configuration of the candidate area extraction section. The candidate area extraction unit 28 extracts a smoke candidate area for each image input from the preprocessing unit 26. As shown in FIG. It is composed of a moving average image generating section 46 functioning as average image generating means and a binary image generating section 48 functioning as binary image generating means.

The difference image generation unit 45 generates a difference image by subtracting the moving average image output from the moving average image generation unit 46 at that time from the latest image each time the latest image is input from the preprocessing unit 26 . output.

Every time the latest image is input from the preprocessing unit 26, the moving average image generating unit 46 outputs the held moving average image to the difference image generating unit 45, then generates and holds the moving average image. Update the moving average image. The moving average image is generated by the moving average image generator 46 by multiplying the value obtained by multiplying the current input image by a predetermined update rate, and multiplying the previous moving average image by the value obtained by subtracting the update rate from 1. Generate by adding values. That is, the moving average image is generated by the following equation.
(moving average image) =
(current input image) x (update rate) + (previous moving average image) x (1 - update rate)
This moving average image is generated for each pixel. For example, since the brightness value of a pixel has a gradation value of 0 to 256, for example, if the update rate is 0.3, the current pixel value is 70, and the previous moving average pixel value is 60, the moving average pixel value is 63 is sought.

The binarized image generation unit 48 sets smoke pixel 1 (black pixel) when the pixel luminance of the difference image output from the difference image generation unit 45 is equal to or higher than a predetermined threshold value, and otherwise sets it to smoke pixel 1 (black pixel). The background pixel 0 (white pixel) is binarized. Here, the binarization threshold value is three times the standard deviation of the luminance distribution of the difference image. For example, when the standard deviation is 3.33, 10, which is three times the standard deviation, is set as the brightness threshold. By such binarization, it is possible to extract, as a smoke candidate area, a portion having a large luminance value in the differential image representing a moving portion.

[Optical flow generator]
The optical flow generation unit 36 provided in the feature amount extraction unit 30 in FIG. 2 sets a plurality of candidate points by dividing the smoke candidate region output from the candidate region extraction unit 28 into, for example, a mesh shape, and calculates each candidate point. An optical flow based on a vector indicating temporal movement is generated at the beginning and output to the clustering processing unit 38 .

That is, the optical flow generation unit 36 generates optical flows corresponding to the number of candidate points based on the luminance distribution relationship between the latest frame image and the previous frame image around a plurality of set candidate points, and performs clustering. Output to the processing unit 38 .

A plurality of optical flows generated by the optical flow generation unit 36 have two values, the amount of movement in the x direction and the amount of movement in the y direction, for each flow. Pairing these two values and plotting them on the xy coordinate plane yields FIG.

[Clustering]
FIG. 4 is an explanatory diagram showing a plurality of optical flows generated from candidate regions and their grouping.

The clustering processing unit 38 groups the optical flows of the smoke candidate regions generated by the optical flow generating unit 36 into clusters distributed in close positions.

In the screen coordinates of FIG. 4(A), the optical flow is plotted at three locations, i.e., the first quadrant, the second quadrant, and the vicinity of the origin. For a plurality of optical flows generated in this way, in this embodiment, as indicated by the dotted line enclosure in FIG. It is intended that the clustering processing unit 38 performs the processing of classifying into 50-3.

As a result of the clustering, by obtaining the central coordinates and the length of the long axis of the ellipse surrounded by the dotted line in FIG. The quantity can be obtained as a numerical value. Here, the length of the long axis is used as the amount of speed variation, but the length at which the Mahalanobis distance in the long axis direction is 1 when the cluster is applied to a two-dimensional normal distribution may be used. In the following description, the amount of speed variation is described as "speed variation".

FIG. 5 is an explanatory diagram showing a smoke estimation area in cluster coordinates. As shown in FIG. 5, the cluster coordinate plane has two-dimensional coordinates with the average speed Vave on the x-axis and the speed variation Vσ on the y-axis. .

In FIG. 5, a smoke estimation region 60 can be set for the cluster coordinate plane based on the average speed and speed variation. The estimated smoke area 60 of this embodiment is set as an area between an estimated smoke area lower limit line 62 and an estimated smoke area upper limit line 64 .

The estimated smoke area lower limit line 62 rises with a predetermined slope from a predetermined minimum value of the average velocity Vave near the origin as a starting point P1, and then becomes a constant slope at a point P2. The estimated smoke area lower limit line 62 is, for example, an inclined line 62-1 connecting a coordinate point P2 with an average velocity of 5 cm/s and a velocity variation of 0 cm/s as a starting point P1 and a coordinate point P2 with an average velocity of 17.5 cm and a velocity variation of 12.5 cm. It is composed of a horizontal line 62-2 starting from the point P2 and increasing only in average speed.

The estimated smoke area upper limit line 64 is an inclined line that rises with a predetermined inclination larger than the inclined line 62-1 of the estimated smoke area lower limit line 62, starting from a predetermined minimum value Q1 of the velocity variation near the origin. For example, the estimated smoke area upper limit line 64 is composed of an inclined line with an average velocity of 0 cm/s and a velocity variation of 5 cm/s as a starting point Q1, where the average velocity and the velocity variation increase at a slope of about 1.5 to 1.6, for example. be.

An estimated steam area 66 is below the estimated smoke area lower limit line 62 , and an estimated human area 68 is above the estimated smoke area upper limit line 64 . Note that the steam estimation area 66 includes moving objects other than steam, and the person estimation area 68 includes moving objects other than people.

FIG. 6 is an explanatory diagram showing determination areas such as smoke, steam (water vapor), and people. As shown in FIG. 6, the estimated smoke area 60 includes a smoldering smoke area 74-1 representing smoke due to smoldering fire, a smoke area 74-2 due to normal fire, and a smoke area 74-2 due to normal fire, depending on the situation of the fire. It can be divided into a smoke region 74-3 that burns with the machine, a smoke region 74-4 that blows out of the machine, and so on. Therefore, it is possible to determine the type of smoke caused by the fire from the area where the cluster exists.

[Smoke judgment part]
The smoke determination unit 32 shown in FIG. 2 compares the estimated smoke area 60 preset on the cluster coordinate plane shown in FIG. and output the judgment result. The smoke determination section 32 is provided with a weight setting section 40 , a determination value calculation section 42 and a determination section 44 .

The weight setting unit 40 sets a predetermined weight Wi for each of k clusters generated in time series. The determination value calculation unit 42 obtains a value obtained by multiplying the number n of optical flows included in the cluster by a weight Wi for each cluster, and calculates a determination value D by averaging the sum totals. Namely
D=Σ(Wi·n)/k
calculated as

If the determination value D is equal to or greater than a predetermined determination threshold value Dth or exceeds the determination threshold value Dth, the determination unit 44 determines that the cluster is a smoke area, and outputs a determination result.

(First embodiment of weight setting)
The weight setting unit 40 of this embodiment sets the weight according to the distance of the cluster position from the lower limit line of the smoke estimation region.

FIG. 7 is an explanatory diagram showing a process of setting weights according to distances from the lower limit of the smoke estimation region to a plurality of clusters generated in time series. FIG. 7A shows clusters 70-1 to 70-5 of smoke optical flow superimposed in chronological order, and distances L1 to L5 of the clusters 70-1 to 70-5 with respect to the estimated smoke region lower limit line 62 are shown. Then, weights W1 to W5 that increase in proportion to the distance are set in the clusters 70-1 to 70-5, and the judgment value D is calculated. Here, since the center-of-gravity positions G1 to G5 are above the estimated smoke area lower limit line 62, all of them have positive values.

FIG. 7B shows steam optical flow clusters 72-1 to 72-5 superimposed in time series. Then, weights that increase in proportion to the distance are set in the clusters 72-1 to 72-5, and the determination value D is calculated.

Here, since the clusters 72-1 to 72-4 are below the estimated smoke area lower limit line 62, they have negative values, the weights W1 to W4 also have negative values, and the weights -W1 to -W4 and W5 The determination value D calculated based on this is a negative value and certainly falls below the determination threshold value Dth, so that it is not determined to be a smoke area.

(Second embodiment of weight setting)
The weight setting unit 40 of this embodiment sets a weight of 1 if the position of the cluster is in the smoke estimation region, and sets a weight of 0 if it is not in the smoke estimation region. Therefore, in the case of FIG. 7(A), weight 1 is set to all clusters 70-1 to 70-5, and in the case of FIG. 7(B), weight 0 is set to clusters 72-1 to 72-4. A weight of 1 is set for the cluster 72-5. In this embodiment, the setting of weights is simple, and the processing load of cluster fire determination can be reduced.

(Third embodiment of weight setting)
The weight setting unit 40 of this embodiment assigns a weight of 1 if the position of the cluster is in the smoke estimation region, and assigns a weight of 0 if the cluster is not in the smoke estimation region. A value obtained by inputting a predetermined evaluation function including is set as a weight.

In the above-described second embodiment of weight setting, the weight is set based on whether or not the cluster is in the smoke estimation area, so the weight setting is simple. , the weights 1 and 0 set in the first stage are input to an evaluation function such as a tanh function or a sigmoid function, and the weights are set in the second stage. It is possible to improve the accuracy of determination as to whether or not there is a smoke area.

(Fourth embodiment of weight setting)
The weight setting unit 40 of the present embodiment applies a predetermined average velocity and a range of velocity variations around the position of the cluster to the two-dimensional probability density function of smoke obtained in advance corresponding to the average velocity and velocity variations. The probability of being smoke calculated by integration of the two-dimensional probability density function is set as the weight of the cluster.

In the present embodiment, a two-dimensional probability density function given by a curved surface of a two-dimensional histogram distribution indicating the frequency that the optical flow is smoke, centering on the smoke estimation area set on the cluster coordinate plane by the average speed and the speed variation, is prepared in advance. On the premise that it is obtained, the probability that it is smoke is obtained by integrating (volume) the two-dimensional probability density function of the average speed and the speed variation in a predetermined range centering on the position of the cluster generated from the optical flow. By using the weight, it is possible to judge whether or not the area is a smoke area with higher accuracy.

(Fifth embodiment of weight setting)
The weight setting unit 40 of this embodiment is characterized in that the weight setting of the fourth embodiment is performed using a lookup table.

That is, on the premise that the two-dimensional probability density function of smoke on the cluster coordinate plane is obtained in advance, the weight setting unit 40 of the present embodiment calculates the two-dimensional probability density function of smoke corresponding to the average speed and the speed variation. Equipped with a lookup table in which functions are set, the sum of two-dimensional probability density functions obtained by referencing the lookup table based on the average speed and speed variation in a predetermined range centered on the cluster position is set as the smoke weight. do.

In this way, by obtaining the sum of the values of the two-dimensional probability density function read out by referring to the lookup table based on the cluster average speed and speed variation, the probability of being smoke can be easily obtained and set as a weight. .

[Processing control of smoke detector]
FIG. 8 is a flow chart showing processing control of the smoke detection device, which is a control operation by the control section 34 shown in FIG.

As shown in FIG. 8, the control unit 34 reads a color frame image output from the monitoring camera 10 in step S1, then instructs the preprocessing unit 26 in step S2 to read the frame output from the monitoring camera 10. The image is grayscaled to reduce the number of pixels, and further subjected to a predetermined enhancement process as necessary, and then output to the candidate area extraction unit 28 .

Subsequently, in step S3, the control unit 34 instructs the candidate area extraction unit 28 to extract a smoke candidate area for each input image. Specifically, each time the latest image is input from the preprocessing unit 26, the moving average image being generated at that time is subtracted from the latest image to generate a difference image. are smoke pixels and the others are background pixels, and an image portion with motion is extracted as a smoke candidate region.

Subsequently, in step S4, the control unit 34 instructs the feature quantity extraction unit 30 to extract a feature quantity for determining a smoke region from the smoke candidate region. In this embodiment, the control unit 34 instructs the optical flow generation unit 36 provided in the feature amount extraction unit 30 to set candidate points by dividing the smoke candidate region into, for example, a mesh, and to calculate the movement of each candidate point. Generate the optical flow from the vector shown. Subsequently, the control unit 34 instructs the clustering processing unit 38 to cluster the generated optical flows by, for example, the difference mixture Gaussian distribution method to generate clusters.

Subsequently, in step S5, the control unit 34 instructs the smoke determination unit 32 to determine whether or not there is a smoke area based on the relationship between the cluster generated as the feature amount and the smoke estimation area preset on the cluster coordinate plane. For example, for multiple clusters that are continuous in time series, weights are set according to the distance relationship with the smoke estimation area, the weight is multiplied by the number of optical flows for each cluster, and the average value for the entire cluster is the judgment value. Ask as

Subsequently, the control unit 34 compares the determination value with a predetermined determination threshold in step S6, and returns to step S1 if the determination value is less than the predetermined determination threshold or equal to or less than the determination threshold. Alternatively, if the determination threshold is exceeded, the cluster is determined to be a smoke area, and the process proceeds to step S7 to output a smoke detection signal to the fire alarm equipment 14 to output a fire alarm or fire preliminary alarm (pre-alarm). Further, the output of the smoke detection signal in step S7 may be such that the smoke detection signal is output when the determination result of the smoke region continues for a predetermined number of times. Further, when the cluster is determined to be a non-smoke area, a non-smoke detection signal may be output as necessary.

[Modification of the present invention]
(Clustering)
Although the above embodiments take the optical flow clustering method by the variational mixture Gaussian distribution method (VBGMM method) as an example, the clustering method by the Gaussian mixture model, the k-means method (k-means method), or the mean shift method Clusters may be generated by clustering using a widely used method such as (MeanShift method).

In addition, in the method described in the above embodiment, clustering was performed using a set of optical flows obtained in a single frame, but a set of all optical flows obtained in a plurality of frames (for example, 10 frames) is used. Discrimination may be performed by clustering with

(Surveillance camera)
In the above-described embodiment, since a monitoring camera that outputs color moving images is used, the preprocessing unit 26 converts color frame images into grayscale images. , grayscaling at the image input section is not required.

(others)
Moreover, the present invention is not limited to the above-described embodiments, includes appropriate modifications that do not impair the objects and advantages thereof, and is not limited by the numerical values shown in the above-described embodiments.

10: Surveillance camera 12: Smoke detection device 14: Fire alarm equipment 16: Monitoring area 18: Fire source 20: Smoke 22: Pot 24: Steam 26: Preprocessing unit 28: Candidate area extraction unit 30: Feature amount extraction unit 32: Smoke determination unit 34: control unit 36: optical flow generation unit 38: clustering processing unit 40: weight setting unit 42: determination value calculation unit 44: determination unit 45: difference image generation unit 46: moving average image generation unit 48: binary Converted image generators 56-1, 56-2, 70-1 to 70-5, 72-1 to 72-5: cluster 60: estimated smoke area 62: estimated smoke area lower limit line 62-1: inclined line 62-2 : horizontal line 64: smoke estimation area upper limit line 66: steam estimation area 68: person estimation area

Claims (13)

imaging means for outputting images obtained by sequentially imaging a monitoring area;
candidate area extracting means for extracting a smoke candidate area for each of the images output from the imaging means;
optical flow generation means for setting a plurality of candidate points in the smoke candidate region and generating an optical flow based on a vector indicating temporal motion for each of the candidate points;
Clustering processing means for clustering the optical flow into one or more clusters on a coordinate plane having two-dimensional coordinates of the x component and the y component of the movement amount, and obtaining an average velocity and an amount of variation in velocity for each cluster;
Smoke judgment for comparing the average speed and the speed variation of a preset smoke estimation region and the cluster on a two-dimensional coordinate plane of the x-axis of the average speed and the y-axis of the speed variation, and outputting the comparison result. A smoke detection device, characterized in that means are provided.
2. The smoke detection apparatus according to claim 1, wherein said candidate area extracting means comprises:
a difference image generating means for generating a difference image by subtracting a moving average image from the image output from the imaging means;
binarized image generating means for binarizing the pixel luminance of the differential image generated by the differential image generating means into smoke pixels when the pixel luminance is equal to or greater than a predetermined threshold value or exceeding the threshold value, and into background pixels otherwise; ,
A smoke detection device, characterized in that a
3. The smoke detection apparatus according to claim 2, wherein said difference image generating means updates said moving average image to the previous moving average image by multiplying the current input image by a predetermined update rate. A smoke detection device characterized in that it is generated by adding a value obtained by multiplying a value obtained by subtracting a rate from 1.
4. A smoke detection apparatus according to claim 3, wherein said optical flow generating means, when generating an optical flow for said candidate point, determines that the optical flow is valid if it exists in a predetermined smoke determination region, and performs said clustering. A smoke detection device, characterized by outputting to a processing means.
4. The smoke detection apparatus according to claim 3, wherein the clustering processing means clusters the optical flow by a predetermined variational mixture Gaussian distribution method, a clustering method using a Gaussian mixture model, a k-mean method, or a mean shift method. A smoke detection device, wherein the one or more clusters are generated.
2. The smoke detection device according to claim 1, wherein the smoke determination means defines the estimated smoke region on the two-dimensional coordinate plane at a predetermined gradient starting from a predetermined minimum value of the average velocity near the origin. and an estimated smoke area lower limit line that rises with a predetermined slope that is greater than the estimated smoke area lower limit line with a predetermined minimum value of the speed variation amount near the origin as a starting point. A smoke detection device characterized by being set between lines.
The smoke detection device of claim 1, wherein
The smoke determination means includes:
weight setting means for setting a predetermined weight to the generated predetermined number of clusters;
a judgment value calculating means for obtaining a value obtained by multiplying the number of the optical flows included in the cluster by the weight for each cluster, averaging the total sum, and calculating a judgment value;
smoke area determination means for determining the cluster as a smoke area when the determination value is equal to or greater than a predetermined determination threshold or exceeds the determination threshold;
A smoke detection device, characterized in that a
The smoke detection device of claim 7, wherein
The weight setting means
The smoke detection apparatus, wherein the weight is set according to the distance of the position of the cluster plotted from the average speed and the speed variation amount on the two-dimensional coordinate plane to the lower limit of the smoke estimation region.
The smoke detection device of claim 7, wherein
The weight setting means
If the position of the cluster plotted on the two-dimensional coordinate plane by the average velocity and the velocity variation is in the smoke estimation area, a weight of 1 is set, and if it is not in the smoke estimation area, a weight of 0 is set. A smoke detection device characterized by:
The smoke detection device of claim 7, wherein
The weight setting means
Obtained by inputting the distance of the position of the cluster plotted on the two-dimensional coordinate plane by the average velocity and the amount of velocity variation to the lower limit of the smoke estimation area into a predetermined evaluation function including a tanh function or a sigmoid function. A smoke detection device characterized by setting a value as a weight.
The smoke detection device of claim 7, wherein
The weight setting means is configured to set the predetermined average velocity and the velocity variation around the position of the cluster with respect to the two-dimensional probability density function of smoke obtained in advance corresponding to the average velocity and the velocity variation amount. A smoke detection apparatus, wherein a probability of being smoke calculated by integration of the two-dimensional probability density function of the range is set as the weight of the cluster.
The smoke detection device of claim 7, wherein
The weight setting means includes a lookup table in which a two-dimensional probability density function of smoke obtained in advance corresponding to the average velocity and the amount of velocity variation is set, and the lookup table is referenced by the position of the cluster. A smoke detection apparatus, wherein the output of the two-dimensional probability density function obtained in 1. is set as the weight of the cluster.
The imaging means sequentially captures images of the monitored area,
extracting a smoke candidate region for each image output from the imaging means by candidate region extraction means;
setting a plurality of candidate points in the smoke candidate region by an optical flow generation means, and generating an optical flow based on a vector indicating temporal movement for each of the candidate points;
Clustering processing means clusters the optical flow into one or more clusters on a coordinate plane having two-dimensional coordinates of the x component and the y component of the movement amount, and obtains the average velocity and the amount of variation in velocity for each cluster. The means compares the average speed and the speed variation of the cluster with the preset smoke estimation region on a two-dimensional coordinate plane of the x-axis of the average speed and the y-axis of the speed variation, and outputs the comparison result. do,
A smoke identification method characterized by:

Images