KR102233402B1 - Method for estimating concentration of fine dust and apparatus for executing the method - Google Patents

Abstract

Disclosed are a method for estimating the concentration of fine dust to easily estimate the concentration of fine dust and an apparatus for executing the same. According to one embodiment of the present invention, the method for estimating the concentration of fine dust is executed by a computing device including at least one processor and at least one program executed by the at least one processor. The method comprises the following steps: acquiring an image captured at a predetermined place; loss-compressing the captured image and storing the compressed image by a preset unit time length; generating a fine dust removal image from which fine dust is removed from the compressed image; generating a residual image through a difference between the compressed image and the fine dust removal image; and estimating the concentration of fine dust in the captured image on the basis of the residual image.

Description

A method for estimating the concentration of fine dust and an apparatus for performing it {METHOD FOR ESTIMATING CONCENTRATION OF FINE DUST AND APPARATUS FOR EXECUTING THE METHOD}

An embodiment of the present invention relates to a technique for estimating the concentration of fine dust.

Recently, fine dust has become a social issue, and various media such as news provide the concentration of fine dust for each region. Existing methods of measuring fine dust include a method of measuring indirectly using the physical properties of radiation or light (beta ray absorption method or light scattering method, etc.) and a method of measuring the mass of fine dust with a balance (weight concentration method, etc.). However, there is a problem that expensive fine dust measuring equipment is required.

In the past, fine dust was measured for each administrative area unit (eg, ward unit), but in reality, even in the same administrative area, the measured value of the fine dust differs depending on topographic characteristics and building density. Therefore, there is a need for a method capable of measuring the concentration of fine dust according to each location without expensive fine dust measuring equipment.

Korean Patent Publication No. 10-2018-0003750 (2018.01.10)

An embodiment of the present invention is to provide a method for estimating the concentration of fine dust capable of easily estimating the concentration of the fine dust and an apparatus for performing the same.

The method for estimating the concentration of fine dust according to the disclosed embodiment is a method performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, at a predetermined location. Obtaining a photographed photographed image; Performing lossy compression processing on the captured image, and storing the captured image by dividing it into a predetermined unit time length; Generating a fine dust removal image from which fine dust has been removed from the compressed image based on the compressed image; Generating a residual image through a difference between the compressed image and the fine dust removal image; And estimating the concentration of fine dust in the captured image based on the residual image.

The generating of the fine dust removal image may include extracting RGB data from the compressed image to generate an R-channel compressed image, a G-channel compressed image, and a B-channel compressed image; Generating a dark channel image based on the R channel compressed image, the G channel compressed image, and the B channel compressed image; Generating a transmission amount map by calculating a transmission amount at the location of each pixel based on the dark channel image; And generating the fine dust removal image based on the delivery amount map and the compressed image.

The step of generating the fine dust removal image is performed by a convolution neural network (CNN), and the convolutional neural network includes the R channel compressed image, the G channel compressed image, and the B channel compressed image A feature extraction network learned to generate the transfer amount map when is input; And a reconstruction network trained to output the fine dust removal image when the transmission amount map and the compressed image are input.

The estimating of the fine dust concentration may include calculating a statistic for the residual image; And estimating a fine dust concentration value corresponding to the calculated statistic as a fine dust concentration value in the captured image.

The calculating of the statistic may include calculating an entropy of the residual image; And estimating a fine dust concentration value corresponding to the calculated entropy from a previously stored entropy and fine dust concentration relationship graph as a fine dust concentration value in the captured image.

A computing device according to the disclosed embodiment includes: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include: an instruction for obtaining a photographed image photographed at a predetermined location; A command for lossy compression processing the captured image, dividing and storing the captured image by a preset unit time length; A command for generating a fine dust removal image from which fine dust is removed from the compressed image based on the compressed image; A command for generating a residual image through a difference between the compressed image and the fine dust removal image; And a command for estimating the concentration of fine dust in the captured image based on the residual image.

A command for extracting RGB data from the compressed image to generate an R-channel compressed image, a G-channel compressed image, and a B-channel compressed image; A command for generating a dark channel image based on the R channel compressed image, the G channel compressed image, and the B channel compressed image; A command for generating a transmission amount map by calculating a transmission amount at the location of each pixel based on the dark channel image; And a command for generating the fine dust removal image based on the delivery amount map and the compressed image.

The command for generating the fine dust removal image is performed by a convolution neural network (CNN), and the convolutional neural network includes the R channel compressed image, the G channel compressed image, and the B channel compressed A feature extraction network learned to generate the transfer amount map when an image is input; And a reconstruction network trained to output the fine dust removal image when the transmission amount map and the compressed image are input.

The command for estimating the fine dust concentration may include: a command for calculating a statistic for the residual image; And a command for estimating a fine dust concentration value corresponding to the calculated statistic as a fine dust concentration value in the captured image.

The instruction for calculating the statistic may include an instruction for calculating an entropy of the residual image; And a command for estimating a fine dust concentration value corresponding to the calculated entropy from a previously stored entropy and fine dust concentration relationship graph as a fine dust concentration value in the captured image.

According to the disclosed embodiment, since it is possible to check the concentration of fine dust in a corresponding region only with a photographed image, expensive fine dust measuring equipment is not required, and the concentration of fine dust in each region in which a user is located can be easily checked. That is, since only an image photographed around a place for knowing the fine dust concentration is required, it is possible to easily check the fine dust concentration at any place regardless of the place or region.

1 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments.
2 is a flow chart for explaining a fine dust reading method according to an embodiment of the present invention
3 is a diagram showing a state in which a fine dust removal image is generated from a compressed image using a convolutional neural network in an embodiment of the present invention
4 is a diagram showing a state in which fine dust is removed using a convolutional neural network (CNN) in an embodiment of the present invention
5 is a diagram showing a residual image as a gray scale image in an embodiment of the present invention
FIG. 6 is a graph created by correlating a statistic (variance or average value) calculated based on an image taken at a location where the actual fine dust concentration was measured and the actual fine dust concentration measurement value
Fig. 7 is a graph created by correlating a statistic (entropy) calculated based on an image taken at a location where the actual fine dust concentration is measured and the actual fine dust concentration measurement value

Hereinafter, a specific embodiment of the present invention will be described with reference to the drawings. The following detailed description is provided to aid in a comprehensive understanding of the methods, devices, and/or systems described herein. However, this is only an example and the present invention is not limited thereto.

In describing the embodiments of the present invention, when it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of users or operators. Therefore, the definition should be made based on the contents throughout the present specification. The terms used in the detailed description are only for describing embodiments of the present invention, and should not be limiting. Unless explicitly used otherwise, expressions in the singular form include the meaning of the plural form. In the present description, expressions such as "comprising" or "feature" are intended to refer to certain features, numbers, steps, actions, elements, some or combination thereof, and one or more It should not be construed to exclude the presence or possibility of other features, numbers, steps, actions, elements, any part or combination thereof.

In the following description, "transmission", "communication", "transmission", "reception" of signals or information, and other terms having a similar meaning are not only directly transmitted signals or information from one component to another component. It includes what is passed through other components. In particular, "transmitting" or "transmitting" a signal or information to a component indicates the final destination of the signal or information and does not mean a direct destination. The same is true for "reception" of signals or information. In addition, in the present specification, when two or more pieces of data or information are "related", it means that when one data (or information) is obtained, at least a part of other data (or information) can be obtained based thereon.

In addition, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms may be used for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

1 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12. In an embodiment, the computing device 12 may be a device for measuring the concentration of fine dust based on a captured image.

The computing device 12 includes at least one processor 14, a computer-readable storage medium 16 and a communication bus 18. The processor 14 may cause the computing device 12 to operate in accordance with the aforementioned exemplary embodiments. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16. The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions are configured to cause the computing device 12 to perform operations according to an exemplary embodiment when executed by the processor 14 Can be.

The computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer-readable storage medium 16 includes a set of instructions executable by the processor 14. In one embodiment, the computer-readable storage medium 16 includes memory (volatile memory such as random access memory, nonvolatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other types of storage media that can be accessed by the computing device 12 and store desired information, or a suitable combination thereof.

The communication bus 18 interconnects the various other components of the computing device 12, including the processor 14 and computer readable storage medium 16.

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24. The input/output interface 22 and the network communication interface 26 are connected to the communication bus 18. The input/output device 24 may be connected to other components of the computing device 12 through the input/output interface 22. The exemplary input/output device 24 includes a pointing device (mouse or track pad, etc.), a keyboard, a touch input device (touch pad or touch screen, etc.), a voice or sound input device, various types of sensor devices, and/or photographing devices. Input devices, and/or output devices such as display devices, printers, speakers, and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12, and may be connected to the computing device 12 as a separate device distinct from the computing device 12. May be.

2 is a flowchart illustrating a method of reading fine dust according to an embodiment of the present invention. As described above, the method 200 for analyzing fine dust according to an embodiment of the present invention includes a computing device 12 having one or more processors and a memory for storing one or more programs executed by the one or more processors. ) Can be performed. To this end, the fine dust analysis method 200 may be implemented in the form of a program or software including one or more computer executable instructions and stored in the memory.

In addition, in the illustrated flowchart, the method is described by dividing the method into a plurality of steps, but at least some of the steps are performed in a different order, combined with other steps, performed together, omitted, divided into detailed steps, or shown. One or more steps that have not been performed may be added and performed.

In step 202, the computing device 12 may acquire a photographed video captured at a predetermined location (a location for checking the concentration of fine dust).

In an exemplary embodiment, the computing device 12 may capture a corresponding location through a camera provided in the computing device 12 to obtain a captured video. For example, the computing device 12 may be a mobile device such as a smart phone or a tablet PC. In this case, the user can take a picture of a place where he wants to know the concentration of fine dust (ie, a place where he is located) using his or her mobile device. However, the present invention is not limited thereto, and the computing device 12 may receive and acquire a captured video from another device (eg, CCTV installed in a specific place).

In step 204, the computing device 12 may perform lossy compression processing and store the acquired captured video. That is, when storing the acquired captured video as an internal file, the computing device 12 may perform lossy compression processing and store it in order to reduce a data capacity.

Here, the computing device 12 may divide and store the captured video by a preset unit time length (eg, 1 second or 2 seconds, or a frame unit, etc.). In addition, the computing device 12 may each extract RGB data from a captured video having a predetermined unit time length to obtain compressed images of the R (Red) channel, G (Green) channel, and B (Blue) channel, respectively. have.

In operation 206, the computing device 12 may generate an image from which fine dust has been removed from the compressed image. That is, the computing device 12 may generate an image (fine dust removal image) from which fine dust has been removed based on the R-channel compressed image, the G-channel compressed image, and the B-channel compressed image.

Specifically, the computing device 12 may estimate a transmission amount of a pixel including fine dust information based on the R-channel compressed image, the G-channel compressed image, and the B-channel compressed image.

That is, the process of lossy compression processing the captured video in step 204 is a compression process in a way that considerably loses light information from the captured video by taking advantage of the fact that the human vision has a relatively low sensitivity to light compared to color. It is done. Accordingly, the lossy compression-processed video does not seem to have a significant difference to the human eye, but in reality, information about light is lost in a large part compared to the original video.

Accordingly, the computing device 12 may restore the fine dust information by estimating a transmission amount of a pixel including the fine dust information based on the R-channel compressed image, the G-channel compressed image, and the B-channel compressed image. Here, the amount of transmission may represent a degree obtained by the camera without scattering the color of the object. In an exemplary embodiment, the computing device 12 may generate a dark channel image based on an R-channel compressed image, a G-channel compressed image, and a B-channel compressed image to estimate a transmission amount of a pixel.

The computing device 12 may extract a minimum value of pixels in the window as a dark channel value while moving a window of a preset size from the R channel compressed image, the G channel compressed image, and the B channel compressed image by a preset interval. . The computing device 12 may generate a dark channel image by synthesizing an image having a minimum value in an R-channel compressed image, an image having a minimum value in a G-channel compressed image, and an image having a minimum value in a B-channel compressed image.

In addition, the computing device 12 may generate a transmission amount map by calculating a transmission amount at the location of each pixel based on the dark channel image. For example, the computing device 12 may calculate a transmission amount at the location of the corresponding pixel by applying a preset weight to a pixel value having the highest value among dark channel values of the corresponding window area.

Next, the computing device 12 may generate an image from which fine dust has been removed based on the delivery amount map and the compressed image. That is, since the delivery amount map includes fine dust information, if fine dust information included in the delivery amount map is removed from the compressed image, an image from which fine dust has been removed (fine dust removal image) can be generated.

In an exemplary embodiment, the computing device 12 may perform a process of generating a fine dust removal image from the compressed image by a machine learning technique. For example, a convolution neural network (CNN) may be used as a machine learning technique, but is not limited thereto.

3 is a diagram showing a state in which a fine dust removal image is generated from a compressed image using a convolutional neural network in an embodiment of the present invention.

Referring to FIG. 3, a convolutional neural network (CNN) includes a feature extraction network (N1) and a reconstruction network (N2). Here, the feature extraction network N1 may be a network that has been trained to generate a delivery amount map based on the input R channel compressed image, G channel compressed image, and B channel compressed image.

When the reconstruction network (N2) receives a transmission amount map and a compressed image (an image in which the R channel compressed image, the G channel compressed image, and the B channel compressed image are combined) as inputs, an image from which fine dust has been removed (fine dust removed image) It may be a network that has been trained to output.

4 is a diagram showing a state in which fine dust is removed using a convolutional neural network (CNN) in an embodiment of the present invention.

Referring back to FIG. 2, in step 208, the computing device 12 may generate a residual image through a difference between the compressed image (ie, the compressed image before fine dust removal) and the fine dust removal image. . That is, the computing device 12 may generate a residual image by calculating a difference in pixel values between pixels of the compressed image and the fine dust removal image. In this case, the residual image reflects the amount of change in the image due to fine dust based on the compressed image.

Here, as illustrated in FIG. 5, the computing device 12 may convert the residual image into a gray scale image and visualize it (ie, visually express the fine dust concentration). Referring to FIG. 5, it can be seen that the higher the fine dust concentration, the darker the gray scale image (that is, the larger the gray scale value).

In operation 210, the computing device 12 may calculate a statistic capable of predicting the fine dust concentration based on the residual image. In an exemplary embodiment, the computing device 12 may calculate a variance value or an average value for each pixel of the residual image as a statistic for predicting the fine dust concentration.

In addition, the computing device 12 may calculate the entropy of the residual image as a statistic for predicting the concentration of fine dust. The computing device 12 may calculate the entropy (H) of the residual image through Equation 1 below.

(Equation 1)

Here, P(x) may mean a probability density at which a pixel value x appears in the residual image.

의 픽셀값을 갖는 경우, P(x=1) = 3(x=1인 픽셀의 개수)/10(잔차 이미지의 총 픽셀 개수) = 0.3이고, P(x=2) = 2(x=2인 픽셀의 개수)/10(잔차 이미지의 총 픽셀 개수) = 0.2이며, P(x=3) = 5(x=3인 픽셀의 개수)/10(잔차 이미지의 총 픽셀 개수) = 0.5이다. 그러면, 잔차 이미지의 엔트로피(H)는 -(0.3×log0.3 + 0.2×log0.2 + 0.5×log0.5) = 1.029653이 되게 된다.For example, the residual image is a 5×2 image,

For a pixel value of P(x=1) = 3 (number of pixels with x=1)/10 (total number of pixels in the residual image) = 0.3, and P(x=2) = 2(x=2 The number of in pixels)/10 (the total number of pixels in the residual image) = 0.2, and P(x=3) = 5 (the number of pixels where x=3)/10 (the total number of pixels in the residual image) = 0.5. Then, the entropy (H) of the residual image becomes -(0.3×log0.3 + 0.2×log0.2 + 0.5×log0.5) = 1.029653.

In operation 212, the computing device 12 may estimate the concentration of fine dust in the captured image based on the statistics calculated from the residual image. Here, the statistic may be provided to correspond to the fine dust concentration one-to-one.

For example, the computing device 12 may estimate the fine dust concentration using a graph between the statistic shown in FIG. 6 and the fine dust concentration. FIG. 6A is a graph showing the relationship between the variance value of the residual image and the fine dust concentration, and FIG. 6B is a graph showing the relationship between the average value of the residual image and the fine dust concentration.

6 is a graph created by correlating a statistic (variance or average value) calculated based on an image taken at a location where the actual fine dust concentration is measured and an actual fine dust concentration measurement value. FIG. 6A is a graph showing a relationship between dispersion and a concentration of fine dust, and FIG. 6B is a graph showing a relationship between an average value and a concentration of fine dust.

로 측정된 장소에서 촬영된 영상을 기반으로 산출된 통계량을 미세먼지 농도값 75

에 대응시키고, 미세먼지 농도가 100

로 측정된 장소에서 촬영된 영상을 기반으로 산출된 통계량을 미세먼지 농도값 100

에 대응시키며, 미세먼지 농도가 125

로 측정된 장소에서 촬영된 영상을 기반으로 산출된 통계량을 미세먼지 농도값 125

에 대응시키는 방식으로 각 측정된 미세먼지 농도에 대해 통계량을 일대일로 대응시켜 도 6과 같은 그래프를 생성할 수 있다. For example, the fine dust concentration is 75

The statistic calculated based on the image taken at the place measured with the fine dust concentration value is 75

And the fine dust concentration is 100

The statistic calculated based on the image taken at the place measured with the fine dust concentration value is 100

And the fine dust concentration is 125

The statistic calculated based on the image taken at the place measured by the fine dust concentration value is 125

A graph as shown in FIG. 6 may be generated by matching statistics for each measured fine dust concentration in a manner corresponding to.

In addition, FIG. 7 is a graph (entropy and fine dust concentration relationship graph) created by correlating a statistic (entropy) calculated based on an image taken at a place where the actual fine dust concentration was measured and an actual fine dust concentration measurement value. According to the entropy, it is possible to more accurately estimate the concentration of fine dust by minimizing the error of the change in statistics according to the change in the concentration of fine dust.

When the statistic for the residual image is calculated in operation 210, the computing device 12 may estimate a fine dust concentration value corresponding to the calculated statistic as the fine dust concentration in the captured image.

According to the disclosed embodiment, since it is possible to check the concentration of fine dust in a corresponding region only with a photographed image, expensive fine dust measuring equipment is not required, and the concentration of fine dust in each region in which a user is located can be easily checked. That is, since only an image photographed around a place for knowing the fine dust concentration is required, it is possible to easily check the fine dust concentration at any place regardless of the place or region.

Although the exemplary embodiments of the present invention have been described in detail above, those of ordinary skill in the art to which the present invention pertains will understand that various modifications can be made to the above-described embodiments without departing from the scope of the present invention. . Therefore, the scope of the present invention is limited to the described embodiments and should not be determined, and should not be determined by the claims to be described later, but also by those equivalents to the claims.

10: computing environment
12: computing device
14: processor
16: computer readable storage medium
18: communication bus
20: program
22: input/output interface
24: input/output device
26: network communication interface

Claims (10)

One or more processors, and
A method performed in a computing device having a memory storing one or more programs executed by the one or more processors,
Obtaining a photographed image photographed at a predetermined location;
Performing lossy compression processing on the captured image;
Generating a fine dust removal image from which fine dust has been removed from the compressed image based on the compressed image;
Generating a residual image through a difference between the compressed image and the fine dust removal image; And
Including the step of estimating the concentration of fine dust in the photographed image based on the residual image,
Generating the fine dust removal image,
Extracting RGB data from the compressed image to generate an R-channel compressed image, a G-channel compressed image, and a B-channel compressed image;
Generating a dark channel image based on the R channel compressed image, the G channel compressed image, and the B channel compressed image;
Generating a transmission amount map by calculating a transmission amount at the location of each pixel based on the dark channel image; And
And generating the fine dust removal image based on the delivery amount map and the compressed image.
delete The method according to claim 1,
The step of generating the fine dust removal image is performed by a convolution neural network (CNN),
The convolutional neural network,
A feature extraction network learned to generate the transfer amount map when the R-channel compressed image, the G-channel compressed image, and the B-channel compressed image are input; And
And a reconstruction network trained to output the fine dust removal image when the transmission amount map and the compressed image are input.
The method according to claim 1,
The step of estimating the fine dust concentration,
Calculating statistics for the residual image; And
And estimating a fine dust concentration value corresponding to the calculated statistic as a fine dust concentration value in the photographed image.
One or more processors, and
A method performed in a computing device having a memory storing one or more programs executed by the one or more processors,
Obtaining a photographed image photographed at a predetermined location;
Performing lossy compression processing on the captured image;
Generating a fine dust removal image from which fine dust has been removed from the compressed image based on the compressed image;
Generating a residual image through a difference between the compressed image and the fine dust removal image; And
Including the step of estimating the concentration of fine dust in the photographed image based on the residual image,
The step of estimating the fine dust concentration,
Calculating an entropy of the residual image; And
And estimating a fine dust concentration value corresponding to the calculated entropy from a previously stored entropy and fine dust concentration relationship graph as a fine dust concentration value in the photographed image.
One or more processors;
Memory; And
Contains one or more programs,
The one or more programs are stored in the memory and configured to be executed by the one or more processors,
The one or more programs,
A command for obtaining a photographed image photographed at a predetermined location;
A command for lossy compression processing the captured image;
A command for generating a fine dust removal image from which fine dust is removed from the compressed image based on the compressed image;
A command for generating a residual image through a difference between the compressed image and the fine dust removal image; And
Including a command for estimating the concentration of fine dust in the captured image based on the residual image,
The command for generating the fine dust removal image,
A command for extracting RGB data from the compressed image to generate an R-channel compressed image, a G-channel compressed image, and a B-channel compressed image;
A command for generating a dark channel image based on the R channel compressed image, the G channel compressed image, and the B channel compressed image;
A command for generating a transmission amount map by calculating a transmission amount at the location of each pixel based on the dark channel image; And
And a command for generating the fine dust removal image based on the delivery amount map and the compressed image.
delete The method of claim 6,
The command for generating the fine dust removal image is performed by a convolution neural network (CNN),
The convolutional neural network,
A feature extraction network learned to generate the transfer amount map when the R-channel compressed image, the G-channel compressed image, and the B-channel compressed image are input; And
And a reconstruction network trained to output the fine dust removal image when the transmission amount map and the compressed image are input.
The method of claim 6,
The command for estimating the fine dust concentration,
An instruction for calculating a statistic for the residual image; And
And a command for estimating a fine dust concentration value corresponding to the calculated statistic as a fine dust concentration value in the captured image.
One or more processors;
Memory; And
Contains one or more programs,
The one or more programs are stored in the memory and configured to be executed by the one or more processors,
The one or more programs,
A command for obtaining a photographed image photographed at a predetermined location;
A command for lossy compression processing the captured image;
A command for generating a fine dust removal image from which fine dust is removed from the compressed image based on the compressed image;
A command for generating a residual image through a difference between the compressed image and the fine dust removal image; And
Including a command for estimating the concentration of fine dust in the captured image based on the residual image,
The command for estimating the fine dust concentration,
An instruction for calculating an entropy of the residual image; And
A computing device comprising a command for estimating a fine dust concentration value corresponding to the calculated entropy from a previously stored entropy and fine dust concentration relationship graph as a fine dust concentration value in the captured image.

Images