JP6457728B2 - Laminar smoke detection device and laminar smoke detection method - Google Patents

Description

  The present invention relates to a technique for detecting the generation of smoke by performing image processing on an image captured by a surveillance camera, and in particular, smoke that flows in layers due to a fire (hereinafter referred to as laminar smoke). ), And a laminar smoke detection method suitable for detection.

  From the viewpoint of initial fire extinguishment in the event of a fire or prevention of escape delay in a fire accident, early detection of fire or smoke is very important. Thus, in the field of smoke detection devices, research has been conducted on early detection of smoke by performing image processing on an image captured by a surveillance camera.

  As an example, there is a conventional smoke detection device that detects smoke by installing a camera in a tunnel or the like and performing image processing on an image captured by the camera. In image processing for detecting smoke, generally, a reference image (reference image) is stored in advance, a difference image between the latest captured image and the reference image is calculated, and a region where a change has occurred is calculated. Extraction detects smoke (for example, see Patent Document 1).

  In addition, the reference image is regularly updated in order to cope with the temporal change of the reference image due to the influence of sunlight.

Thus, by performing image processing on the image captured by the camera and performing smoke detection, the following two merits can be obtained.
1) By visually confirming the image of the surveillance camera, it is possible to grasp the smoke detection status in a remote place.
2) It is possible to divert already installed surveillance cameras and construct efficient equipment.

Japanese Patent No. 3909665

However, the prior art has the following problems.
It is preferable that the image of the surveillance camera is always taken in an environment where the illumination conditions and the visual field are stable. However, depending on the monitoring location, not only under such favorable conditions, but within the monitoring range, there are places where moving objects such as people come and go or places where sunshine conditions etc. change over time. It becomes a source of misinformation.

  In the prior art, the luminance difference result is obtained even though there is no smoke even with respect to the luminance change partially generated in the area not suitable for monitoring (that is, the area where the false alarm source exists). May cause false detection of smoke from In addition, it is conceivable that an area that is not suitable for monitoring and has a false alarm source may be masked so as not to detect smoke, but this results in a limited monitoring range.

  Furthermore, the color of the laminar smoke that is the detection target of the present invention is transmissive and has a feature that a landscape (background) on the other side of the laminar smoke can be seen from the camera side. Therefore, the simple detection method based on the difference image between the captured image and the reference image results in over-detection or false detection, and establishment of an image processing method suitable for detecting laminar smoke is desired.

  The present invention has been made to solve the above-described problems, and provides a laminar smoke detection device and a laminar smoke detection method that suppresses the influence of a false alarm source and improves detection accuracy. Objective.

A laminar smoke detection device according to the present invention is a laminar smoke detection device that detects the occurrence of laminar smoke by performing image processing on an image captured by a surveillance camera, and is detected at the time of preparation in advance. Stores multiple laminar smoke images when the target laminar smoke actually occurs, multiple false source images where there are actually false alarm sources, and a reference image where both laminar smoke and false alarm sources do not exist , At the time of monitoring, the image memory for storing the monitoring target image captured by the monitoring camera, and for each image stored in the image memory, the target pixel and the surrounding pixels of the target pixel are sequentially scanned, and for each area, calculating a local binary pattern data against the brightness of the pixel of interest, a plurality of laminar flow smoke image, first calculated plurality of false alarm sources images, and any one of a monitored image as the target image b And Cal binary pattern data, comparing the second local binary pattern data calculated with respect to the reference image, to calculate the similarity by counting the number of matched bits, corresponding to the pixel of interest in an image for the target image A feature amount extraction unit that calculates an LBP feature amount by calculating an average value of similarities, and a plurality of laminar smokes for the LBP feature amount calculated by the feature amount extraction unit using a plurality of laminar smoke images as target images A feature that creates a laminar smoke feature amount histogram relating to an image and also creates an error source feature amount histogram relating to a plurality of misreport source images for LBP feature amounts calculated by a feature amount extraction unit using a plurality of misreport source images as target images. the amount histogram creation unit, and the LBP feature amount calculated by the feature amount extraction unit monitored image as the target image, the feature histogram Laminar smoke judgment that determines whether or not laminar smoke is generated in the monitoring target image captured at the time of monitoring from the correlation between the laminar smoke feature histogram and the false alarm source feature histogram created by the creation unit Part.

Further, the laminar smoke detection method according to the present invention performs laminar smoke executed in a laminar smoke detection device that detects generation of laminar smoke by performing image processing on an image captured by a surveillance camera. In the detection method , at the time of advance preparation, a plurality of laminar smoke images when a laminar smoke that is a detection target actually occurs, a plurality of false alarm images where a false alarm source actually exists, and laminar smoke and misinformation A reference image in which no source is present is stored in the image memory, and at the time of monitoring, a monitoring step in which the monitoring target image captured by the monitoring camera is stored in the image memory, and a target pixel for each image stored in the image memory And the surrounding pixels of the pixel of interest are sequentially scanned, and local binary pattern data for the luminance of the pixel of interest is calculated for each region, and a plurality of laminar smoke images, a plurality of false alarm images, and The first local binary pattern data calculated for any one of the monitoring target images as the target image is compared with the second local binary pattern data calculated for the reference image, and the number of matched bits is counted. A feature amount extraction step of calculating an LBP feature amount by calculating an average value of the similarity degree corresponding to the target pixel in the image for the target image, and a plurality of laminar smoke images as the target image For the LBP feature amount calculated by the feature amount extraction step, a laminar smoke feature amount histogram relating to a plurality of laminar smoke images is created, and the LBP calculated by the feature amount extraction step using a plurality of false alarm source images as target images A feature amount histogram creating step for creating a misinformation source feature amount histogram relating to a plurality of misinformation source images for the feature amount, Captured at the time of monitoring from the correlation between the LBP feature quantity calculated by the feature quantity extraction step using the visual target image as the target image, and the laminar smoke feature quantity histogram and the misreport source feature quantity histogram created by the feature quantity histogram creation step A laminar smoke determination step for determining whether or not laminar smoke is generated in the monitored image.

  According to the present invention, for the LBP feature quantity that is an index of the similarity relationship between the pixel of interest and surrounding pixels, the frequency distribution of laminar smoke and the frequency distribution of false alarm sources are calculated in the preliminary preparation stage, By determining the presence or absence of laminar smoke based on the correlation between each frequency distribution and the LBP feature value calculated for the image at the time of monitoring, the influence of false alarm sources can be suppressed and detection accuracy can be improved. The intended laminar smoke detection device and laminar smoke detection method can be obtained.

It is a block diagram of the laminar flow smoke detection apparatus in Embodiment 1 of this invention. It is an illustration figure which shows the feature-value histogram calculated with respect to each of a laminar-flow smoke and a misreport source with respect to several images in the feature-value histogram creation part of the laminar smoke detection apparatus in Embodiment 1 of this invention. It is a block diagram of the laminar flow smoke detection apparatus in Embodiment 2 of this invention. It is explanatory drawing regarding the local binary pattern (LBP) which is the feature-value in Embodiment 2 of this invention. It is explanatory drawing regarding the calculation process of the feature-value based on LBP data in Embodiment 2 of this invention. It is an illustration figure which shows the feature-value histogram calculated with respect to the several image about each of a laminar-flow smoke and a false alarm source in the feature-value histogram preparation part of the laminar-flow smoke detection apparatus in Embodiment 2 of this invention. It is the figure which showed an example of the sample based on two feature-values in Embodiment 3 of this invention. It is the figure which described the distribution state of A group, B group, center position, and an unknown point in Embodiment 3 of this invention as the feature-value 1 as the X-axis and the feature-value 2 as the Y-axis. FIG. 8 is a view different from FIG. 8 in which the distribution state, center position, and unknown point of the A group and the B group in Embodiment 3 of the present invention are described with the feature amount 1 as the X axis and the feature amount 2 as the Y axis is there.

  Hereinafter, preferred embodiments of the laminar smoke detection device and the laminar smoke detection method of the present invention will be described with reference to the drawings.

Embodiment 1 FIG.
In the first embodiment, a method for improving the detection accuracy of laminar smoke will be described based on texture analysis using a luminance histogram (shading histogram).

  The texture means a pattern represented by regular shading change. And the laminar smoke that is the object of detection has a unique texture with fine particles. Therefore, in the first embodiment, it is determined whether or not the captured image is laminar smoke by detecting and analyzing a texture as a feature amount.

  In determining whether it is laminar smoke, some quantification is required to characterize the texture, which is called a texture feature. Texture features can be obtained by measuring a density histogram, a Fourier power spectrum, a run length matrix, a co-occurrence matrix, and the like. In the first embodiment, attention is paid to the texture feature measured from the density histogram.

  FIG. 1 is a configuration diagram of a laminar smoke detector according to Embodiment 1 of the present invention. The laminar smoke detection device according to the first embodiment includes an image memory 10, a feature amount extraction unit 20, and a laminar smoke generation detection unit 30. The image memory 10 is configured as an image memory for a plurality of frames so that an image captured by the camera 1 can be stored. Specifically, a plurality of laminar smoke images when laminar smoke that is a detection target actually occurs, a plurality of false alarm images in which a false alarm source actually exists, and a monitoring target image captured during monitoring, The image memory 10 can be stored, and details of each image will be described later.

  The feature amount extraction unit 20 includes a luminance histogram creation unit 21 and a texture feature amount calculation unit 22. The feature amount extraction unit 20 has a function of calculating a plurality of texture feature amounts with respect to an image captured by the camera 1 and stored in the image memory 10.

  Then, in the preliminary preparation stage, the feature amount extraction unit 20 applies to a plurality of images (a plurality of laminar smoke images and a plurality of false alarm source images) captured for each of the actual laminar smoke and the actual false alarm sources. A plurality of texture feature amounts are calculated. Further, when monitoring laminar smoke, a plurality of texture feature amounts are calculated for an actual image (monitoring target image) captured in the monitoring space.

  The laminar smoke generation detection unit 30 includes a feature amount histogram creation unit 31 and a laminar smoke determination unit 32. Then, the laminar smoke generation detection unit 30 creates a feature amount histogram for each texture feature amount of the laminar smoke and the false alarm source calculated for a plurality of images by the feature amount extraction unit 20 in the preliminary preparation stage.

  Furthermore, the laminar smoke generation detection unit 30 detects that the plurality of texture feature amounts generated by the feature amount extraction unit 20 at the time of monitoring is more laminar than the feature amount histogram related to the false alarm source created in the preliminary preparation stage. When the correlation with the feature amount histogram is high and a predetermined condition is satisfied, it is determined that there is a high possibility that laminar smoke has occurred.

  Therefore, a series of processing procedures for detecting laminar smoke including the preparatory stage and actual monitoring based on the configuration of FIG. 1 will be described in detail below.

Procedure 1: Calculation of density histogram The brightness histogram creation section 21 in the feature amount extraction section 20 performs a density histogram H (l) (l) for the image area of the image stored in the image memory 10 where the texture feature is to be measured. = 0, 1, 2, ..., L-1). Here, l corresponds to a light and shade level, and in the case of 256 gradations, L = 256, l = 0, 1,.

Procedure 2: Normalization of density histogram Further, the luminance histogram creation unit 21 divides the frequency H (l) of each density level by the number of pixels in the image area, normalizes the total number of pixels to 1.0, and P (L). Here, normalization is performed so that the texture feature does not depend on the area of the region to be measured.

Procedure 3: Calculation of texture feature amount The texture feature amount calculation unit 22 in the feature amount extraction unit 20 calculates the following five texture feature amounts based on the frequency P (l) normalized by the luminance histogram creation unit 21. calculate.

[Texture Feature Amount 1] Average The average of the light and shade levels is calculated as the texture feature amount 1 by the following equation (1). This average value corresponds to a simple average of the light and shade levels in the image area.

[Texture Feature Amount 2] Contrast A gray level contrast is calculated as the texture feature amount 2 by the following equation (2). This contrast value takes a large value if the distribution is biased to a high level.

[Texture Feature Amount 3] Dispersion The variance of the shading level is calculated as the texture feature amount 3 by the following equation (3). This variance value takes a large value if there are many pixels away from the average value.

[Texture Feature Amount 4] Energy The density level energy is calculated as the texture feature amount 4 by the following equation (4). This energy value takes a large value and approaches 1 when pixels are present at a specific gray level.

[Texture Feature Amount 5] Entropy The density level entropy is calculated as the texture feature amount 5 by the following equation (5). This entropy value takes a large value when there are many pixels of many light and dark levels.

Procedure 4: Collecting data in advance regarding each feature quantity The feature quantity extraction unit 20 images each of the actual laminar smoke and the actual misinformation source by repeating steps 1 to 3 in the preliminary preparation stage. A plurality of texture feature amounts are calculated for a plurality of images.

  On the other hand, the feature amount histogram creation unit 31 in the laminar smoke generation detection unit 30 is configured to obtain each texture feature amount of laminar smoke and false alarm sources calculated for a plurality of images by the feature amount extraction unit 20 in the preliminary preparation stage. A feature amount histogram is created for. That is, for example, when the texture feature amount 1 is taken as an example, the feature amount histogram creating unit 31 calculates the texture feature amount 1 calculated for each of a plurality of images (laminar smoke image) in which laminar smoke actually occurs. A histogram (laminar smoke characteristic amount histogram) is created for the values (average values).

  Similarly, the feature amount histogram creation unit 31 also applies a histogram (misinformation source) to the value (average value) of the texture feature amount 1 calculated for each of a plurality of images (misinformation source images) in which a misreport source has actually occurred. A feature amount histogram). Then, the feature amount histogram creating unit 31 individually creates such a histogram for each of the texture feature amount 1 to the texture feature amount 5 in the preliminary preparation stage.

  FIG. 2 is an exemplary diagram showing a feature amount histogram calculated for a plurality of images for each of the laminar smoke and the false alarm source in the feature amount histogram creation unit 31 of the laminar smoke detection device according to Embodiment 1 of the present invention. It is. More specifically, the frequency distribution of laminar smoke and misinformation sources for the average feature value of texture feature 1, the variance of texture feature value 3, and the entropy of texture feature value 5 are summarized. Is.

(A)-(f) in FIG. 2 has shown the histogram of the following contents, respectively.
Fig. 2 (a): Frequency distribution data related to the calculation result of texture feature 1 (average) in a plurality of captured images when laminar smoke actually occurs Fig. 2 (b): When a false alarm source actually occurs Frequency distribution data regarding the calculation result of texture feature amount 1 (average) in a plurality of captured images of FIG. 2C: Texture feature amount 3 (dispersion) in a plurality of captured images when laminar smoke actually occurs FIG. 2D: Frequency distribution data regarding the calculation result of the texture feature amount 3 (dispersion) in a plurality of captured images when an erroneous report source actually occurs FIG. 2E: Frequency distribution data related to the calculation result of texture feature amount 5 (entropy) in a plurality of captured images when laminar smoke is generated. FIG. 2 (f): a plurality of captured images when a false alarm source is actually generated. Kicking, frequency distribution data relating to the calculation result of the texture feature 5 (entropy)

  From these data, it can be seen that the frequency distribution of laminar smoke is coherent, whereas the frequency distribution of false alarm sources tends to vary compared to the frequency distribution of laminar smoke. In particular, when looking at the average frequency distribution shown in FIG. 2 (b) and the entropy frequency distribution shown in FIG. 2 (f) regarding the false alarm source, frequency peaks exist at a plurality of locations.

Procedure 5: Judgment of the presence or absence of laminar smoke during monitoring Based on the data collection results in the preliminary preparation steps of Procedures 1 to 4 described above, the following Procedure 5 (1) and Procedure 5 (2) are performed during actual monitoring. To determine whether laminar smoke is generated.

Procedure 5 (1): Calculation of texture feature quantity in image at the time of monitoring The feature quantity extraction unit 20 performs the texture feature quantity 1 to the texture feature quantity according to the above-described procedure 1 to procedure 3 for the captured image obtained at the actual monitoring time. 5 is calculated.

Procedure 5 (2): Determination of the correlation between texture feature values obtained in Procedure 5 (1) The laminar smoke judgment unit 32 in the laminar smoke generation detection unit 30 determines the image at the time of monitoring in Procedure 5 (1). For each of the plurality of texture feature amounts obtained by the feature amount extraction unit 20, a laminar smoke feature amount histogram (frequency distribution) and a false alarm source feature amount histogram (frequency) relating to each texture feature amount collected in step 4 Distribution).

  Then, the laminar smoke determination unit 32 has a correlation between the plurality of texture feature amounts related to the image at the time of monitoring and the feature amount histogram related to the laminar smoke rather than the feature amount histogram related to the false alarm source created in the preliminary preparation stage. If the predetermined condition is high, it is determined that there is a high possibility that laminar smoke has been generated.

  For example, the laminar smoke determination unit 32 has a range in which the plurality of texture feature values obtained in step 5 (1) are indicated by circles of the three distributions of (a), (c), and (e) in FIG. If it is all inside, it can be determined that laminar smoke has occurred.

  That is, the laminar smoke determination unit 32 may have generated laminar smoke for each texture feature amount from the laminar smoke feature amount histogram created for each of the plurality of texture feature amounts by the feature amount histogram creation unit 31. A high feature amount range can be defined.

  And the laminar smoke determination part 32 is N (N is 2 or more and M or less) among M (M is an integer greater than or equal to 2) which is a plurality of texture feature amounts calculated for the monitoring target image. If the texture feature amount is equal to or greater than the specified feature amount range, the laminar smoke is generated in the monitoring target image captured during monitoring. It can be determined that

  In addition, the laminar smoke determination unit 32 can make a determination based on the Mahalanobis distance as another quantitative determination method considering the correlation, and this determination method will be described in detail in the third embodiment. .

  As described above, according to the first embodiment, based on a plurality of images in which laminar smoke actually exists and a plurality of images in which false alarm sources actually exist, a plurality of texture feature amounts in the preliminary preparation stage. Each feature amount histogram is created for. At the time of monitoring, a plurality of texture feature values are calculated from the monitoring image, and the presence or absence of laminar smoke is determined based on the correlation between the feature histograms related to laminar smoke and the feature histograms related to false alarm sources. Judging.

  In other words, the laminar smoke detection device according to Embodiment 1 calculates the frequency distribution of laminar smoke and the frequency distribution of false alarm sources based on the actual data at the preliminary preparation stage for a plurality of texture feature amounts, Whether or not laminar smoke is generated is determined based on the correlation between the frequency distribution and a plurality of texture feature quantities calculated at the time of monitoring. As a result, it is possible to realize laminar smoke detection that suppresses the influence of false alarm sources and improves detection accuracy by judgment based on statistical data.

Embodiment 2. FIG.
In the first embodiment, the case where the texture feature amount based on the luminance histogram is employed as the feature amount for detecting laminar smoke has been described. On the other hand, in the second embodiment, attention is paid to the luminance of the target pixel and surrounding pixels, and an index relating to the similarity between the reference image acquired in advance and the acquired image at the time of monitoring is expressed as laminar smoke. A case where it is adopted as a feature amount for detection will be described.

  The color of laminar smoke is transmissive and has a characteristic that a landscape (background) beyond the laminar smoke can be seen from the camera side. That is, it is conceivable that the relationship between the luminance of the surrounding pixels and the luminance of the pixel of interest has a strong correlation between when there is laminar smoke and when there is no laminar smoke. Therefore, in the second embodiment, a feature quantity based on a local binary pattern (LBP) that serves as an index of such correlation is used, which will be described in detail below.

  FIG. 3 is a configuration diagram of the laminar smoke detector according to Embodiment 2 of the present invention. The laminar smoke detection device according to the second embodiment includes an image memory 10, a feature amount extraction unit 20a, and a laminar smoke generation detection unit 30. Compared with the configuration of FIG. 1 in the first embodiment, the function of the feature amount extraction unit 20a is different. The image memory 10 is configured as an image memory for a plurality of frames so that an image captured by the camera 1 can be stored. Specifically, multiple laminar smoke images when the laminar smoke that is the detection target actually occurs, multiple false source images where the false alarm source actually exists, reference image where there is no laminar smoke or false alarm source The monitoring target image captured at the time of monitoring can be stored in the image memory 10.

  The feature quantity extraction unit 20 a includes an LBP data creation unit 23 and an LBP feature quantity calculation unit 24. The feature quantity extraction unit 20 has a function of calculating a feature quantity (LBP feature quantity) based on the local banari pattern for the image captured by the camera 1 and stored in the image memory 10.

  Then, in the preliminary preparation stage, the feature amount extraction unit 20a calculates an LBP feature amount based on a plurality of images and reference images captured for each of the actual laminar smoke and the actual false alarm source (for details, see FIG. , Described later). When monitoring laminar smoke, the LBP feature value is calculated based on the actual image and the reference image captured in the monitoring space.

  The laminar smoke generation detection unit 30 includes a feature amount histogram creation unit 31 and a laminar smoke determination unit 32. Then, in the preliminary preparation stage, the laminar smoke generation detection unit 30 creates a feature amount histogram for the laminar smoke and the LBP feature amount of the false alarm source calculated for the plurality of images by the feature amount extraction unit 20a.

  Furthermore, the laminar smoke generation detection unit 30 is characterized in that the LBP feature value generated by the feature value extraction unit 20a is more characteristic than the feature amount histogram related to the false alarm source created in the preliminary preparation stage. When the correlation with the quantity histogram is high and a predetermined condition is satisfied, it is determined that there is a high possibility that laminar smoke has occurred.

  Therefore, details of a series of processing procedures including the preliminary preparation stage and actual monitoring based on the configuration of FIG. 3 will be described below.

Procedure 1: Creation of LBP Data FIG. 4 is an explanatory diagram regarding a local binary pattern (LBP) which is a feature amount according to Embodiment 2 of the present invention. FIG. 4A shows the luminance values (light / dark values) in 256 gradations for the pixel of interest in the center and the surrounding pixels (1) to (8) that are the eight surrounding pixels.

  The LBP data creation unit 23 in the feature amount extraction unit 20a sequentially scans a 3 × 3 region including the target pixel and the surrounding pixels (1) to (8) in the image region where the feature amount is desired to be measured. Then, LBP data is created for each pixel of interest as follows.

  In the specific example shown in FIG. 4, first, the LBP data generation unit 23 compares the luminance value “80” of the target pixel with the luminance value “90” of the surrounding pixel (1) located in the upper left. To do. In this case, since the luminance value of the surrounding pixel (1) is larger than the luminance value of the target pixel, 1 is stored in the corresponding bit.

  Next, the LBP data creation unit 23 compares the luminance value “40” of the surrounding pixel (2) located above the target pixel with the luminance value “80” of the target pixel. In this case, since the luminance value of the surrounding pixel (2) is smaller than the luminance value of the target pixel, 0 is stored in the corresponding bit.

  Bit setting based on such comparison processing is also performed for the remaining surrounding pixels (3) to (8), so that 8 created from luminance data in the vicinity of 8 as shown in FIG. Bit data is obtained as LBP data.

Procedure 2: Calculation of Feature Amount Based on LBP Data FIG. 5 is an explanatory diagram relating to a feature amount calculation process based on LBP data according to the second embodiment of the present invention. The LBP data creation unit 23 creates the 8-bit LBP data shown in FIG. 4 (b) and has been obtained in advance with the acquisition data composed of 3 × 3 pixels sequentially scanned with the monitoring target image at the time of monitoring. The reference data consisting of 3 × 3 pixels sequentially scanned in the reference image is sequentially performed.

  Then, the LBP feature value calculation unit 24 calculates the correlation data by obtaining the exclusive OR XOR of the corresponding bits of both the acquired data and the reference data of the same region composed of 3 × 3 pixels. Further, the LBP feature quantity calculation unit 24 obtains the number of bits that are “0” in the correlation data as NUM [0]. In the example of FIG. 5, it is obtained as NUM [0] = 6.

  Further, the LBP feature value calculation unit 24 calculates, as an LBP correlation coefficient, a value obtained by dividing the average value of NUM [0] in all the target pixels in the image region whose feature value is desired to be measured by 8. This LBP correlation coefficient is a feature amount (LBP feature amount) based on local binary data.

Procedure 3: Collecting Prior Data on LBP Feature Amount In the advance preparation stage, the feature amount extraction unit 20a captures an image of each of the actual laminar smoke and the actual misinformation source by repeating Step 1 and Step 2. LBP feature values are calculated for a plurality of images.

  On the other hand, the feature amount histogram creation unit 31 in the laminar smoke generation detection unit 30 uses the feature amount extraction unit 20a to calculate the LBP feature amount of the laminar smoke and the false alarm source calculated for a plurality of images in the preliminary preparation stage. A feature amount histogram is created.

  FIG. 6 shows a plurality of images (a plurality of laminar smoke images and a plurality of false alarms) for each of the laminar smoke and the false alarm source in the feature amount histogram creation unit 31 of the laminar smoke detector according to Embodiment 2 of the present invention. It is an illustration figure which shows the feature-value histogram (Laminar smoke feature-value histogram and false report source feature-value histogram) calculated with respect to the source image). More specifically, the frequency distributions of laminar smoke and false alarm sources for LBP feature values are summarized.

6A and 6B show histograms having the following contents, respectively.
FIG. 6A: Frequency distribution data related to the calculation result of the LBP feature value in a plurality of captured images when laminar smoke actually occurs FIG. 6B: A plurality of images when a false alarm source actually occurs Frequency distribution data related to LBP feature value calculation results in images

  From these data, it can be seen that the LBP correlation coefficient of laminar smoke is distributed in the range of 40 to 55% centering around 50%. On the other hand, it can be seen that the LBP correlation coefficient of the false alarm source is widely distributed in the vicinity of 30 to 50%.

Procedure 4: Judgment of the presence or absence of laminar smoke during monitoring Based on the data collection results in the preliminary preparation steps of Procedures 1 to 3 described above, the following Procedure 4 (1) and Procedure 4 (2) are performed during actual monitoring. To determine whether laminar smoke is generated.

Procedure 4 (1): Calculation of LBP feature value in image at the time of monitoring The feature value extraction unit 20a calculates an LBP feature value for the captured image obtained at the time of actual monitoring according to the above-described procedure 1 and procedure 2.

Procedure 4 (2): Judgment of the correlation between the LBP feature values obtained in Procedure 4 (1) The laminar smoke judgment unit 32 in the laminar smoke generation detection unit 30 determines the image at the time of monitoring in Procedure 4 (1). The correlation between the LBP feature quantity obtained by the feature quantity extraction unit 20a and the laminar smoke feature quantity histogram (frequency distribution) related to the LBP feature quantity collected in step 3 and the misreport source feature quantity histogram (frequency distribution) Ask for.

  The laminar smoke determination unit 32 has a higher correlation between the LBP feature value related to the image at the time of monitoring and the feature value histogram related to the laminar smoke than the feature amount histogram related to the false alarm source created in the preliminary preparation stage, When the predetermined condition is satisfied, it is determined that there is a high possibility that laminar smoke has been generated.

  For example, taking the case of FIG. 6 as an example, the laminar smoke determination unit 32 may have generated laminar smoke if the LBP feature value obtained in step 4 (1) exceeds 50%. It can be judged that it is expensive. Moreover, it can also be judged that the possibility that laminar smoke has occurred is high by continuing the state exceeding 50% a predetermined number of times. Further, when the LBP feature value obtained in the procedure 4 (1) is less than 40%, it can be determined that there is a high possibility that laminar smoke is not generated.

  That is, the laminar smoke determination unit 32 defines the first feature amount range based on the frequency distribution of laminar smoke from the laminar smoke feature amount histogram created by the feature amount histogram creation unit 31 using the LBP feature amount. it can. Similarly, the laminar smoke determination unit 32 can define the second feature amount range based on the frequency distribution of the false alarm source from the false alarm source feature amount histogram created by using the feature quantity histogram creation unit 31 using the LBP feature amount. .

  Then, the laminar smoke determination unit 32 determines that the LBP feature value calculated for the monitoring target image belongs to the first feature value range and does not belong to the second feature value range. It can be determined that laminar smoke is generated in the monitoring target image captured during monitoring.

  Further, the presence / absence of laminar smoke can also be determined by combining the determination result with the plurality of texture feature amounts used in the first embodiment and the determination result with the LBP feature amount in the second embodiment. .

  In addition, the laminar smoke determination unit 32 can make a determination based on the Mahalanobis distance as another quantitative determination method considering the correlation, and this determination method will be described in detail in the third embodiment. .

  As described above, according to the second embodiment, based on the plurality of images in which laminar smoke actually exists, the plurality of images in which false alarm sources actually exist, and the reference image, in the preliminary preparation stage, A feature amount histogram is created for each feature amount. At the time of monitoring, the LBP feature amount is calculated from the monitoring image and the reference image, and from the correlation with the feature amount histogram relating to the laminar smoke and the feature amount histogram relating to the false alarm source prepared in advance, Judgment is made.

  In other words, the laminar smoke detection device according to the second embodiment calculates the frequency distribution of laminar smoke and the frequency distribution of false alarm sources based on the actual data at the preliminary preparation stage for the LBP feature quantity, and each frequency distribution. And whether or not laminar smoke is generated is determined based on the correlation between the LBP feature value calculated at the time of monitoring. As a result, it is possible to realize laminar smoke detection that suppresses the influence of false alarm sources and improves detection accuracy by judgment based on statistical data.

  In addition, as the surrounding pixels described based on FIG. 4, the surrounding pixels (1) to (8) that are eight pixels around the pixel of interest at the center are adopted. However, the number of surrounding pixels in the present invention is as follows. It is not limited to eight. Focusing on the four neighboring pixels above, below, left, and right of the target pixel, adopt the four peripheral pixels consisting of the peripheral pixels (2), (4), (6), (8), or the upper left of the target pixel, It is also possible to adopt four surrounding pixels composed of surrounding pixels (1), (3), (5), and (7) by paying attention to the four neighboring pixels in the upper right, lower left, and lower right. In this case, the LBP data is configured as 4-bit data instead of 8-bit, but the basic processing is the same as in the 8-bit case.

Embodiment 3 FIG.
In the third embodiment, a method for quantitatively evaluating the degree of correlation of feature amounts obtained at the time of monitoring with respect to one or a plurality of feature amount histograms using the “Mahalanobis distance” will be described. That is, a method for quantitatively evaluating the correlation between the feature amount calculated based on the monitoring target image, the laminar smoke feature amount histogram, and the misinformation source feature histogram using the “Mahalanobis distance” will be described in detail. .

  Mahalanobis distance is a kind of distance used in statistics. It is based on the correlation between multivariables and is used for multivariate analysis. Then, for a new sample (corresponding to the feature amount calculated at the time of monitoring in this application), a known sample (in this application, the laminar smoke feature amount histogram created in the preliminary preparation stage, and the false alarm source feature amount) This is useful for clarifying the relationship to the histogram. It differs from the normal Euclidean distance defined in Euclidean space in that it considers the correlation of data and does not depend on the scale level.

で、共分散行列（各変数間の共分散を配列した行列）がΣであるような下式（７）の多変数ベクトル

で表される一群の値に対するマハラノビス距離は、下式（８）のように定義される。 Formally, the average is the following formula (6)

And the multivariable vector of the following equation (7) in which the covariance matrix (matrix in which the covariances between the variables are arranged) is Σ

The Mahalanobis distance with respect to a group of values represented by is defined as the following equation (8).

  The Mahalanobis distance can also be defined as the following expression (9) as an index of dissimilarity between two vectors whose covariance matrix is Σ and follows the same probability distribution.

  The above is the definition related to the Mahalanobis distance. Next, the difference between the “Mahalanobis distance” and the “Euclidean distance” will be described using a specific calculation example regarding two variables (that is, two feature amounts).

  FIG. 7 is a diagram showing an example of a sample based on two feature amounts according to Embodiment 3 of the present invention. Specifically, the results of calculating two feature values 1 and 2 from eight laminar smoke images in which laminar smoke was actually generated are shown as group A, and a false alarm source has actually occurred. The results of calculating two feature amounts 1 and 2 from eight false report source images are shown as group B.

  FIG. 8 is a diagram in which the distribution amount, center position, and unknown point of the A group and the B group according to the third embodiment of the present invention are described with the feature amount 1 as the X axis and the feature amount 2 as the Y axis. Here, the unknown point is calculated when the feature amount 1 and the feature amount 2 are both 5.5 at the time of monitoring, and corresponds to data in which it is unknown whether the correlation is stronger in the A group or the B group. .

At this time, according to the above equation (9), the Mahalanobis distance from the center of the A group to the unknown point is calculated as MA, and the Mahalanobis distance from the center of the B group to the unknown point is calculated as MB. Become.
MA = 3.6458
MB = 22.3125

On the other hand, when the Euclidean distance from the center of the A group to the unknown point is calculated as UA, and the Euclidean distance from the center of the B group to the unknown point is calculated as UB, the following is obtained.
UA = 3.5355
UB = 2.213

  As can be seen from this result, when the Euclidean distance is used as the determination index, the unknown point is determined to be close to the B group (higher correlation with the B group), but the Mahalanobis distance is used as the determination index. In the case, on the contrary, it is determined to be close to the A group (the correlation with the A group is higher).

  That is, as is clear from FIG. 8, the unknown point is close to the center of the B group as the Euclidean distance, but it can be determined that the correlation with the distribution of the A group is high by using the Mahalanobis distance. it can. Therefore, by adopting such Mahalanobis distance as a judgment index when judging the presence or absence of laminar smoke, it is possible to make a judgment considering the correlation with the distribution based on statistical data, suppress the influence of false alarm sources, and detect accuracy Thus, a laminar smoke detector that improves the above can be realized.

  FIG. 9 shows the distribution state, center position, and unknown point of group A and group B according to the third embodiment of the present invention, with feature quantity 1 as the X axis and feature quantity 2 as the Y axis. Is a different figure. The example of FIG. 9 shows a state where an unknown point defined by the feature amount obtained at the time of monitoring belongs to the right end of the distribution of the A group that is laminar smoke. Even in such a case, by using the Mahalanobis distance as a determination index, an unknown point close to group B as the Euclidean distance can be determined as having a high correlation with group A.

  In the configuration of FIG. 1 of the previous embodiment 1 or FIG. 3 of the previous embodiment 2, the laminar smoke determination unit 32 makes a determination based on this Mahalanobis distance. In determining the presence or absence of laminar smoke, the laminar smoke determination unit 32 does not determine only whether the distribution of laminar smoke or the distribution of false alarm sources is closer to the actual distance obtained as the Mahalanobis distance. It can be determined from the numerical values that laminar smoke has occurred when the distance is shorter than a predetermined threshold distance. In addition, even in a case where a part of regions are overlapped in a plurality of distributions, the similarity of correlation can be quantitatively evaluated.

  In the description using FIGS. 7 to 9, the case where the Mahalanobis distance is obtained from two feature amounts has been described for the sake of simplicity, but the present invention is not limited to this. The Mahalanobis distance can also be obtained for one feature quantity or three or more feature quantities. It is also possible to determine whether laminar smoke has been generated based on the calculation result of Mahalanobis distance in each pair by changing two feature quantity pairs and using a plurality of pairs. . Furthermore, the feature quantities used for calculating the Mahalanobis distance are not limited to the plurality of texture feature quantities in the first embodiment and the LBP feature quantities in the second embodiment, and other feature quantities related to the generation of smoke are necessary. It is also possible to consider depending on.

  As described above, according to the third embodiment, the Mahalanobis distance is used as an index when performing quantitative determination of correlation using statistical data, thereby determining whether or not laminar smoke is generated. Yes. Specifically, the Mahalanobis distance calculated for the distribution of the laminar smoke feature amount histogram with respect to the correlation between the feature amounts calculated for the monitoring target image has a value within the first predetermined range ( In other words, the Mahalanobis distance calculated for the distribution of the misinformation source feature histogram has a value outside the second predetermined range (that is, the misinformation source feature If the correlation is weak), it can be determined that laminar smoke is generated in the monitoring target image. As a result, it is possible to realize a laminar smoke detector that suppresses the influence of a false alarm source and improves detection accuracy.

  DESCRIPTION OF SYMBOLS 1 Camera, 10 Image memory, 20, 20a Feature-value extraction part, 21 Luminance histogram creation part, 22 Texture feature-value calculation part, 23 Data creation part, 24 Feature-value calculation part, 30 Laminar smoke generation detection part, 31 Feature-value Histogram creation part, 32 laminar smoke judgment part.

Claims (4)

A laminar smoke detection device that detects the occurrence of laminar smoke by performing image processing on an image captured by a surveillance camera,
At the time of advance preparation, multiple laminar smoke images when the laminar smoke that is the detection target actually occurs, multiple misinformation sources images where the false alarm source actually exists, and both the laminar smoke and the false alarm source exist An image memory that stores a reference image that is not to be stored, and stores a monitoring target image captured by the monitoring camera at the time of monitoring;
For each image stored in the image memory, the target pixel and surrounding pixels of the target pixel are sequentially scanned, and local binary pattern data for the luminance of the target pixel is calculated for each region, and the plurality of layers First local binary pattern data calculated using any one of a smoke image, the plurality of false alarm source images, and the monitoring target image as a target image, and a second local calculated for the reference image Comparing binary pattern data, calculating the degree of similarity by counting the number of matched bits, and calculating the average value of the degree of similarity corresponding to the target pixel in the image for the target image A feature quantity extraction unit,
For the LBP feature value calculated by the feature value extraction unit using the plurality of laminar smoke images as the target image, a laminar smoke feature amount histogram relating to the plurality of laminar smoke images is created, and the plurality of false reports A feature amount histogram creating unit for creating a false report source feature amount histogram relating to the plurality of false report source images for the LBP feature amount calculated by the feature amount extraction unit using a source image as the target image;
The LBP feature amount calculated by the feature amount extraction unit using the monitoring target image as the target image, and the laminar smoke feature amount histogram and the false alarm source feature amount histogram created by the feature amount histogram creation unit A laminar smoke detection device comprising: a laminar smoke determining unit that determines whether or not laminar smoke is generated in the monitoring target image captured at the time of monitoring from a correlation. The laminar smoke detector according to claim 1,
When calculating the local binary pattern data, the feature amount extraction unit includes four neighboring pixels in the upper, lower, left, and right sides of the target pixel, or four neighboring areas in the upper left, upper right, lower left, and lower right of the target pixel. Either local binary pattern data consisting of 4 bits corresponding to the pixel is calculated , or local area consisting of 8 bits corresponding to surrounding pixels in the vicinity of the upper, lower, left, right, upper left, upper right, lower left, and lower right of the pixel of interest. A laminar smoke detector that calculates binary pattern data. The laminar smoke detector according to claim 1 or 2,
The laminar smoke determination unit defines a feature amount range where the possibility that laminar smoke has occurred is high from the laminar smoke feature amount histogram created by the feature amount histogram creation unit, and the image memory at the time of monitoring When the LBP feature quantity calculated by the feature quantity extraction unit with respect to the monitoring target image stored in the group belongs to the feature quantity range, a layer is included in the monitoring target image captured at the time of monitoring. A laminar smoke detector that judges that smoke is flowing. A laminar smoke detection method executed in a laminar smoke detection device that detects generation of laminar smoke by performing image processing on an image captured by a surveillance camera,
At the time of advance preparation, multiple laminar smoke images when the laminar smoke that is the detection target actually occurs, multiple misinformation sources images where the false alarm source actually exists, and both the laminar smoke and the false alarm source exist Storing a reference image not to be stored in the image memory, and storing a monitoring target image captured by the monitoring camera in the image memory at the time of monitoring;
For each image stored in the image memory, the target pixel and surrounding pixels of the target pixel are sequentially scanned, and local binary pattern data for the luminance of the target pixel is calculated for each region, and the plurality of layers First local binary pattern data calculated using any one of a smoke image, the plurality of false alarm source images, and the monitoring target image as a target image, and a second local calculated for the reference image Comparing binary pattern data, calculating the degree of similarity by counting the number of matched bits, and calculating the average value of the degree of similarity corresponding to the target pixel in the image for the target image A feature extraction step to perform,
For the LBP feature value calculated by the feature value extraction step using the plurality of laminar smoke images as the target image, a laminar smoke feature value histogram relating to the plurality of laminar smoke images is created, and the plurality of misinformation A feature amount histogram creating step for creating a false report source feature amount histogram relating to the plurality of false report source images for the LBP feature amount calculated by the feature amount extraction step using a source image as the target image;
The LBP feature amount calculated by the feature amount extraction step using the monitoring target image as the target image, and the laminar smoke feature amount histogram and the false alarm source feature amount histogram created by the feature amount histogram creation step. A laminar smoke detection method comprising: judging from the correlation whether or not laminar smoke is generated in the monitoring target image captured during the monitoring.

Images