JP2014048131A - Image processing device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of estimating a state of the atmosphere.SOLUTION: An image processing device includes an acquisition section, a detector, an estimation section, and an output section. The acquisition section acquires an image of the sky. The detector detects the distribution of cloud by using a pixel value included in the image. The estimation section estimates a state of the atmosphere at a predetermined place from a pickup point of the image and the distribution. The output section outputs the information on the state of the atmosphere.

Description

  Embodiments described herein relate generally to an image processing apparatus, a method, and a program.

A technique for determining a sky pattern from a captured image is disclosed. Saturation and brightness are calculated using the image data of the captured image, and a sky pattern is determined.

JP 2012-8097 A

Srinivasa G. Narasimhan, Shree K. Nayar “Shedding Light on the Weather” (Computer Vision and Pattern Recognition 2003/6/16, p.665-672)

It is difficult not only to determine the sky pattern depending on the cloudy state, but also to estimate the atmospheric state without obtaining more detailed information about the cloud state.

The problem to be solved by the present invention is to provide an image processing apparatus capable of estimating the atmospheric state in more detail.

The image processing apparatus according to the embodiment includes an acquisition unit, a detection unit, an estimation unit, and an output unit. The acquisition unit acquires an empty image. The detection unit detects a cloud distribution using pixel values included in the image. The estimation unit estimates the state of the atmosphere at a predetermined location based on the image capturing point and distribution. The output unit outputs information related to atmospheric conditions.

The block diagram which shows the example of the apparatus of 1st Embodiment. The flowchart which shows the example of the apparatus of 1st Embodiment. The block diagram which shows the example of the apparatus of 2nd Embodiment. The figure which shows the example which matched the kind and pattern of the cloud. The flowchart which shows the example of the apparatus of 2nd Embodiment. The block diagram which shows the example of the apparatus of 3rd Embodiment. The flowchart which shows the example of the apparatus of 3rd Embodiment. The block diagram which shows an example of the hardware constitutions of an image processing apparatus.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
The image processing apparatus 1 according to the first embodiment measures the atmosphere using a sky image taken by a camera.

The image processing apparatus 1 acquires a sky image taken by a camera as an input, and detects a cloud distribution in the image. The state of the atmosphere in the direction connecting the camera and the cloud, for example, the amount of cloud water or the amount of water vapor related to the amount of rain water is estimated from the presence or absence of the cloud.

FIG. 1 is a block diagram showing an image processing apparatus 1 of the present embodiment. The image processing apparatus 1 includes an acquisition unit 11, a detection unit 12, an estimation unit 13, and an output unit 14.

The acquisition unit 11 acquires an image. For example, an image obtained by photographing the sky with a camera installed on the ground or a building is used. Alternatively, the whole sky may be photographed using a fisheye lens or the like.

The detection unit 12 detects a cloud from the acquired image. By detecting clouds in the image, areas with clouds and areas without clouds are extracted.

The estimation unit 13 estimates the state of the atmosphere in the sky from the position of the cloud in the detected image, the position and direction of the installed camera. From the presence or absence of a cloud of each pixel in the image, for example, it is estimated that a region with a cloud has a large amount of water vapor, and a region without a cloud has a small amount of water vapor.

  The output unit 14 outputs the estimated atmospheric state.

  The configuration of the image processing apparatus 1 has been described above.

FIG. 2 is a flowchart showing processing of the image processing apparatus 1. The process shown in the flowchart of FIG. 2 is performed every time the acquisition unit 11 acquires an image.

In step S11, the acquisition unit 11 acquires an image (S11). The acquisition unit 11 supplies the acquired image to the detection unit 12.

In step S <b> 12, the detection unit 12 detects a cloud from the image, and supplies the cloud region and the cloudless region in the image to the estimation unit 13.

Cloud detection is performed based on the pixel value for each pixel of the image. When the pixel is within the predetermined threshold range, the pixel is determined as a cloud, and when the pixel is out of the range, it is determined that there is no cloud. More specifically, the pixel value of each pixel of a clear image, which is an image taken when clear, is used. For example, the camera position and the shooting range may be measured in advance, and the pixel value of the clear image may be calculated based on the light diffusion model from the camera position, the shooting range, and the sun position. Here, the light diffusion model described in Non-Patent Document 1 may be used.

In addition, an image captured in the past and the position of the sun at that time are stored in advance, and there is no cloud in the pixel whose luminance is to be calculated in the past image area close to the acquired position of the sun. A region may be selected, and the luminance value of the current clear image may be calculated using the luminance of the pixel. here,
The area where there is no cloud may be an area where the luminance value has not changed when there are a plurality of images taken in the past. Alternatively, when there are a plurality of pixels for which the luminance is to be calculated without a cloud, the luminance value in the clear image may be calculated from the average value.

Further, the output may be a continuous value from 0 in a state where there is no cloud to 1 in a state where there is no cloud. For the determination, brightness or saturation may be used instead of luminance. Alternatively, the color of each pixel may be regarded as a point in the color space, and the output of a function on the color space preset by learning or the like may be used as the determination result.

The data of the cloud region and the cloudless region output from the detection unit 12 to the estimation unit 13 may be a mask image assigned 1 if there is a cloud in each pixel of the image and 0 if there is no cloud. If the judgment value is a continuous value, that value is entered.

Further, cloud determination may be performed not for each pixel of the image but for each region such as a rectangle. The determination is made using an average of pixel values, brightness, and saturation of the pixels in the region. Also distributed,
A maximum value or a minimum value may be used. The data to be output is a set of coordinates of the upper right and lower left vertices of the rectangle of the cloud region and a cloud judgment value. In this case, the outside of the rectangular area is regarded as a cloudless area.

In step S <b> 13, the estimation unit 13 estimates the atmospheric state of the sky from the camera installation position, direction, cloud region, and cloudless region, and outputs the estimation result to the output unit 14. As the installation position of the camera, the latitude and longitude when the image is taken are used. From the position and the imaging direction, the sky direction corresponding to each pixel of the image is calculated. It is assumed that the amount of water vapor in the direction is large when there is a cloud, and the amount of water vapor is small when there is no cloud, using the cloud judgment value of the pixel. In the case of a continuous value from 0 to 1, the amount of water vapor in that direction is estimated according to the value. When performing cloud judgment by region,
The above processing is performed for each region.

The data of the atmosphere in the sky output from the estimation unit 13 to the output unit 14 is a set of the position and direction of the camera and the amount of water vapor estimated with respect to that direction. Alternatively, the atmosphere may be divided into a three-dimensional mesh, and a set of coordinates of each point and an estimated water vapor amount may be used.

  In step S14, the output unit 14 outputs the estimated state of the atmosphere in the sky.

According to the present embodiment, it is possible to estimate the state of the atmosphere in more detail. In addition, it is possible to estimate the state of the atmosphere in each direction around the camera by using a sky image captured by the camera.

(Second Embodiment)
The image processing apparatus 2 according to the second embodiment uses the fact that the height of the cloud (cloud bottom, etc.) is known by the type of cloud, and recognizes the type of cloud in addition to the detection of the cloud reflected in the image. Is different from the first embodiment. Moreover, the point which also estimates the height of a cloud from the kind of cloud differs from 1st Embodiment. Hereinafter, a different part from 1st Embodiment is demonstrated.

FIG. 3 is a block diagram showing the image processing apparatus 2. The image processing device 2 is the image processing device 1
As compared with the above, the detection unit 12 is replaced with the extraction unit 22, and the estimation unit 13 is replaced with the estimation unit 23. The extraction unit 22 includes a detection unit 12, an identification unit 221, and a pattern storage unit 222.

The extraction unit 22 detects a cloud from the image and identifies the type of the cloud. The cloud type is classified according to the shape or the like. For example, it is a rough classification of stratiform clouds and convective clouds (cumulonimbus clouds, etc.). It is even better if the classification of the cloud is more detailed by combining the shape with the cloud base (altitude, etc.), the upper cloud (e.g., scale cloud), the lower cloud (e.g., rain cloud), and the thunder cloud (e.g., turbulent cloud). Since the types of clouds vary depending on the region, these tables may be switched according to the location where the camera is installed. Alternatively, a set of the area and cloud appearance probability may be added to the table, and the appearance probability of the area where the camera is installed may be used when identifying the cloud type.

The identifying unit 221 cuts out the detected cloud region from the image, compares the detected cloud region with the pattern stored in the pattern storage unit 222, identifies the cloud type, and outputs the corresponding cloud height.

The pattern storage unit 223 stores a set of cloud type, cloud height, and cloud pattern. FIG.
Is an example of a table of cloud types, cloud heights and cloud patterns. The cloud pattern is held as a vector in which pixel values, brightness, saturation, and the like of the region are arranged.

The estimation unit 23 estimates the state of the atmosphere in the sky from the position of the cloud, the height of the cloud, and the position and direction of the installed camera. A direction is estimated from the presence or absence of a cloud of each pixel in the image, and the depth of the cloud in the direction is estimated from the height of the cloud, and the amount of water vapor in the region is estimated from the presence or absence of the cloud or a determination value.

Heretofore, the configuration of the image processing apparatus 2 has been described for the parts that are different from the first embodiment.

FIG. 5 is a flowchart showing processing of the image processing apparatus 2. The process shown in the flowchart of FIG. 5 is performed every time the acquisition unit 11 acquires an image. Hereinafter, a different part from 1st Embodiment is demonstrated.

In step S22, the type of cloud is identified based on the cloud distribution detected by the detection unit 12.

First, the cloud region is cut out based on the detected cloud distribution. Next, the extracted cloud region is normalized to a predetermined size and matched with the cloud pattern stored in the pattern storage unit 222. The cloud having the highest matching score exceeding a predetermined threshold is defined as the cloud type.
If there is nothing exceeding the threshold, the identification is impossible. When there is an appearance probability of a cloud type derived from the camera installation location, a score obtained by multiplying each cloud type matching score by the appearance probability may be used. From the identified cloud type, the height of the cloud is estimated using a table stored in the pattern storage unit 222. The identification unit 221 is an estimation unit 2
3 is supplied with the judgment value of the cloud in the image and the height of each cloud.

About step S23, the estimation part 23 outputs an estimation result to the output part 14 from the position of a camera, a direction, and the cloud area | region and its depth. The direction of the sky corresponding to each pixel of the image is calculated from the installation position and direction of the camera, and the depth in that direction is calculated from the height of the cloud.

From the cloud judgment value of the pixel, it is assumed that when there is a cloud, the amount of water vapor at that position is large, and when there is no cloud, the amount of water vapor is small. In the case of a continuous value from 0 to 1, the amount of water vapor at that position is estimated according to the value. When the cloud determination is performed in units of regions, the above process is performed for each region.

The data of the atmosphere in the sky output from the estimation unit 23 to the output unit 14 is a set of the position and direction of the camera and the amount of water vapor estimated with respect to that direction. Alternatively, the atmosphere may be divided into a three-dimensional mesh, and a set of coordinates of each point and an estimated water vapor amount may be used.

As above, the processing of the image processing apparatus 2 has been described for the parts different from the first embodiment.

According to the present embodiment, it is possible to estimate the state of the atmosphere in more detail. In addition, the atmospheric state can be estimated from the sky image taken by the camera.

(Third embodiment)
The image processing apparatus 3 according to the third embodiment uses a plurality of time-series images and detects the cloud movement to estimate the wind direction of the cloud position in the first and second embodiments. And different. Hereinafter, a different part from 2nd Embodiment is demonstrated.

The image processing apparatus 3 receives a plurality of time-sequential image sequences as an input as an input.
Process one by one. For example, a moving image may be used.

FIG. 6 is a block diagram showing the image processing apparatus 3. The image processing device 3 is a face image processing device 2.
As compared with the above, the extraction unit 22 is replaced with the extraction unit 32, and the estimation unit 23 is replaced with the estimation unit 33. The extraction unit 32 includes a detection unit 12, an identification unit 221, a pattern storage unit 222, a tracking unit 321, and a cloud storage unit 322.

The extraction unit 32 detects a cloud from the image, identifies the type of the cloud, and tracks the cloud in comparison with past images. Also, the detected image and cloud position are stored.

The tracking unit 321 takes a correspondence from the cloud position detected by the detection unit 12 and the past image stored in the cloud storage unit 322 and the cloud position, and estimates the movement of the cloud. The correspondence of the cloud position is performed by cutting out a cloud region from a past image and matching it with the image. Outputs the movement of each cloud that has a response.

The cloud storage unit 322 stores the image acquired by the acquisition unit 11 and the cloud position detected by the detection unit 12. Instead of the acquired image, a clipped cloud image may be held based on the cloud position.

The estimation unit 33 estimates the state of the atmosphere in the sky from the position of the cloud, the height of the cloud, the movement of the cloud, and the position and direction of the installed camera. Estimate the direction from the presence or absence of clouds at each pixel in the image, estimate the depth from the height of the clouds, estimate the wind direction at that point from the movement of the clouds, and estimate the amount of water vapor in the area by the presence or absence of clouds or the judgment value To do.

As above, the configuration of the image processing apparatus 3 has been described with respect to the differences from the second embodiment.

FIG. 7 is a flowchart showing processing of the image processing apparatus 3. The process shown in the flowchart of FIG. 7 is performed every time the acquisition unit 11 acquires an image. Hereinafter, a different part from 2nd Embodiment is demonstrated.

In step S321, the current cloud and the past cloud are matched, and the movement of the cloud is detected. First, when stored in the cloud storage unit 322, a cloud region is cut out from the past image and the cloud position to generate a past cloud image. Next, a current cloud image is generated from the position and image of the cloud detected by the detection unit 12, and the past cloud image and the current cloud image are matched. For the cloud images that have been matched, the movement is extracted from the past cloud position and the current cloud position, and is supplied to the estimation unit 33.

In step S322, the image and the cloud position detected by the detection unit 12 are stored in the cloud storage unit 322.

In step S33, the type of cloud is identified based on the cloud position detected by the cloud extraction unit 22. First, a cloud region is cut out based on the detected cloud position. Next, the extracted cloud region is normalized to a predetermined size, and matching with the cloud pattern stored in the pattern storage unit 23 is performed. The cloud having the highest matching score exceeding a predetermined threshold is defined as the cloud type. If there is nothing exceeding the threshold, the identification is impossible. From the identified cloud type, the height of the cloud is estimated using a table stored in the pattern storage unit 23. The extraction unit 22 supplies the estimation value of the cloud in the image and the height of each cloud to the estimation unit 24.

As above, the processing of the image processing apparatus 3 has been described for the parts different from the second embodiment.

According to the present embodiment, it is possible to estimate the state of the atmosphere in more detail. In addition, it is possible to estimate the state of the atmosphere in each direction around the camera by using a sky image captured by the camera.

FIG. 8 is a block diagram illustrating an example of a hardware configuration of the image processing apparatuses 1, 2, and 3.
As illustrated in FIG. 8, the image processing apparatuses 1, 2, and 3 have a CPU 801, a ROM 802 that stores a recognition program that recognizes a face and an object from an image, a RAM 803, and an HDD 804 that stores a recognition table and the like. , I / F 805 which is an interface with HDD 804
An I / F 806 that is an interface for image input, an input device 807 such as a mouse and a keyboard, an I / F 806 that is an interface with the input device 807, a display device 808 such as a display, and a display device 808 I / F 810 and a bus 88 are provided, and a hardware configuration using a normal computer is provided.
CPU 801, ROM 802, RAM 803, I / F 805, I / F 806, I / F
816 and I / F 810 are connected to each other via a bus 811.

In the image processing apparatuses 1, 2, and 3, the CPU 801 loads a program from the ROM 802 to the RAM 8.
The above-described units (detection unit, detection unit, estimation unit, etc.) are realized on a computer by reading them onto 03, and using the collation data stored in the HDD 804, the I / F 8
The image is processed from 06.

Note that the program may be stored in the HDD 804. The program is a file in an installable or executable format, and is provided by being stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk (FD). You may make it do. Further, the program may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. The program may be provided or distributed via a network such as the Internet. In addition, the table of FIG.
It may be stored in M802. Also, the image is stored in the HDD 804 and the I / F 8
An image may be input from 05.

As described above, in the image processing apparatuses 1, 2, and 3, it is possible to detect a cloud from an image obtained by shooting the sky, and to estimate a weather condition such as the amount of water vapor in the sky from the result and the installation position and direction of the camera. It becomes. As a result, a weather observation device that is easy to install can be realized without the need for a device that radiates electromagnetic waves like a radar. Further, according to the second embodiment, by identifying the cloud type by matching, the three-dimensional position of the cloud is estimated from the relationship between the cloud type and the cloud height, and the weather condition in the sky is determined. It can be estimated three-dimensionally. Further, according to the third embodiment, the movement of the cloud is detected by matching with the past image, and in addition to the water vapor amount, the wind direction is 3
It can be estimated dimensionally.

As described above, according to the embodiment, it is possible to estimate the atmospheric state in more detail.

DESCRIPTION OF SYMBOLS 11 ... Acquisition part, 12 ... Detection part, 13 ... Estimation part, 14 ... Output part, 221 *
..Identification unit, 222... Pattern storage unit, 321... Tracking unit, 322.

Claims (7)

An acquisition unit for acquiring a sky image, a detection unit for detecting a cloud distribution using pixel values included in the image, and estimating an atmospheric state at a predetermined location based on the image capturing point and the distribution An image processing apparatus comprising: an estimation unit that performs an output; and an output unit that outputs information related to the atmospheric state. The cloud distribution and the pattern stored in advance are collated, further comprising an identification unit for identifying the cloud type,
The image processing apparatus according to claim 1, wherein the estimation unit includes an estimation unit that estimates an atmospheric state from a geographical position of the cloud and the type of the cloud. The detection unit includes a storage unit that stores a past sky image and a cloud distribution;
A tracking unit that detects the movement of the cloud by comparing the detected cloud with the past cloud, and further comprising:
The image processing apparatus according to claim 2, wherein the estimation unit estimates an amount of water vapor at the location and a wind direction from the movement of the cloud. The image processing apparatus according to claim 2, wherein the estimation unit estimates an atmospheric state by further using the height of the cloud. A storage unit for storing the pattern;
The image processing apparatus according to claim 2, wherein the pattern is a plurality of images corresponding to cloud types. Get an empty image,
Detecting a cloud distribution using pixel values included in the image;
Estimating the state of the atmosphere in a predetermined place from the imaging point of the image and the distribution,
An image processing method for outputting information relating to the atmospheric state. An acquisition means for acquiring an empty image from the computer;
Detecting means for detecting a cloud distribution using pixel values included in the image;
Estimating means for estimating the state of the atmosphere in a predetermined place from the imaging point of the image and the distribution;
An image processing program that functions as output means for outputting information relating to the atmospheric state.

Images