KR102457470B1 - Apparatus and Method for Artificial Intelligence Based Precipitation Determination Using Image Analysis - Google Patents

Abstract

Disclosed are a device and a method for determining artificial intelligence-based precipitation using image analysis. According to one aspect of an embodiment of the present invention, provided are the device and the method for determining precipitation, which determine and guide whether there is the precipitation in a specific (local) region checked by a user by using image analysis based on artificial intelligence.

Description

Apparatus and Method for Artificial Intelligence Based Precipitation Determination Using Image Analysis

The present invention relates to an apparatus and method for determining whether precipitation is present in a specific (local) area using artificial intelligence-based image analysis.

The content described in this section merely provides background information for the present embodiment and does not constitute the prior art.

Conventionally, the detection of the weather has mainly depended on the weather forecast by the Korea Meteorological Administration. Weather forecasting involves observation of weather elements such as wind, temperature, and atmospheric pressure, analysis of observation data, and weather estimation. These observations include observations from observatories that exist only on land, observations by meteorological buoys or ships that supplement ocean data, and observations by artificial satellites. In the past, radar has been mainly used for estimating the weather.

Radar is a device that estimates the amount of precipitation by sending microwave signals to the clouds and using the signals that are reflected back by raindrops. As such, it is a very important precipitation monitoring system used by disaster-related organizations.

Since the conventional weather forecast is for a wide area, there is a limit in predicting the actual weather in a local area that the user wants to see, and there is a problem in that it is not possible to effectively prepare for localized heavy rain or heavy snow.

In addition, only a rough estimation of the weather is possible, and it is impossible to obtain real-time weather information in a time zone desired by the user, thereby causing inconvenience to the user.

An embodiment of the present invention has an object to provide a precipitation determination apparatus and method for guiding by determining whether precipitation is in a specific (local) area that a user wants to check using artificial intelligence-based image analysis.

According to one aspect of the present invention, in the precipitation determination device for determining the weather and precipitation in the confirmed area that the user wants to check, the information on the confirmed area from the user terminal and the CCTV image or image of the confirmed area from the CCTV A communication unit that receives each and transmits the weather and precipitation determination results to the user terminal, a first learning model for classifying weather from CCTV images or images, and a second learning model for detecting precipitation components in CCTV images or images Classifying the weather of the confirmed area from the CCTV image or image using the memory unit and the first learning model, and determining how much precipitation component in the CCTV image or image is captured using the second learning model It provides a precipitation determination device comprising a control unit and a control unit for controlling the operation of the communication unit, the memory unit and the determination unit.

According to one aspect of the present invention, the first learning model is characterized in that it classifies whether it is snowing or raining weather or not, from CCTV images or images.

According to an aspect of the present invention, the determination unit is characterized in that when the weather is raining in the confirmed area, the CCTV image or precipitation component in the image is captured by using the second learning model.

According to one aspect of the present invention, the precipitation determining device receives input data for learning each of the first learning model and the second learning model and output data for each input data from the outside, and the first learning model and a learning unit for learning the second learning model.

According to one aspect of the present invention, the learning unit uses convolutional neural networks to learn from input data for each of the first learning model and the second learning model and from output data for each input data. Characterized in learning the first learning model and the second learning model.

According to one aspect of the present invention, a user terminal that receives information of a confirmation area in which the user wants to check the weather and precipitation from the user and delivers it to the outside, and a CCTV which is disposed in a preset place and shoots an image around the place Precipitation determination system comprising a precipitation determination device that analyzes the CCTV image of the confirmed area requested from the user terminal, determines the weather and precipitation in the confirmed area, and transmits the determination result to the user terminal to provide.

According to an aspect of the present invention, the precipitation determination device receives the information of the confirmed area from the user terminal, and receives the CCTV image or image of the confirmed area from the CCTV, respectively, and transmits the weather and precipitation determination results to the user terminal. A CCTV image or image using the first learning model and a memory unit that stores a first learning model for classifying weather from a communication unit and a CCTV image or image and a second learning model for detecting a precipitation component in the CCTV image or image Classifying the weather of the confirmed area from the judging unit and the communication unit, the memory unit and the determination unit that determine how much precipitation component is captured in the CCTV image or image using the second learning model. It is characterized in that it includes a control unit.

According to one aspect of the present invention, the first learning model is learned by a convolutional neural network using a plurality of images having weather of each situation as input data and date information displayed in each image as output data. .

According to one aspect of the present invention, the second learning model is characterized in that it is learned by a convolutional neural network using an image of a raining situation as input data and a precipitation component in the input data as output data.

According to an aspect of the present invention, the second learning model is characterized in that the precipitation component is detected based on whether or not distortion in the image has occurred.

According to an aspect of the present invention, in a method for a precipitation determining apparatus to determine the weather and precipitation of a confirmed area that a user wants to check, a first reception process of receiving information on the confirmed area from a user terminal and the confirmed area Using a second receiving process of receiving a CCTV image or image from a CCTV existing in and a first learning model of classifying the weather from the CCTV image or image, Using a classification process for classifying the weather and a second learning model for detecting a precipitation component in a CCTV image or image, a judgment process to determine how much precipitation component is captured in the CCTV image or image received in the second reception process and transmitting the weather of the confirmed area classified in the classification process and a result of the determination of whether precipitation is determined in the determination process to the user terminal.

As described above, according to one aspect of the present invention, since precipitation in a specific (local) area that the user wants to check is determined in real time and provided to the user using artificial intelligence-based image analysis, the user This has the advantage of being able to accurately check the real-time weather in the area you want to check.

1 is a diagram illustrating a precipitation determination system according to an embodiment of the present invention.
2 is a diagram illustrating a configuration of an apparatus for determining precipitation according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating data that the precipitation determination apparatus learns for weather classification according to an embodiment of the present invention.
4 is a diagram illustrating data that the precipitation determination apparatus learns to detect a precipitation component according to an embodiment of the present invention.
5 is a flowchart illustrating a method of determining whether precipitation is present in an image by the apparatus for determining precipitation according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to a specific embodiment, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. It should be understood that terms such as “comprise” or “have” in the present application do not preclude the possibility of addition or existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification in advance. .

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, each configuration, process, process or method included in each embodiment of the present invention may be shared within a range that does not technically contradict each other.

1 is a diagram illustrating a precipitation determination system according to an embodiment of the present invention.

Referring to FIG. 1 , a precipitation determination system 100 according to an embodiment of the present invention includes a CCTV 110 , a precipitation determination device 120 , and a user terminal 130 .

The CCTV 110 is arranged in a specific place, and shoots an image around the place. The CCTV 110 is installed alone or attached to a specific structure, rotates within a preset angle range, and continuously takes images of the surroundings. The CCTV 110 stores the captured image for a preset period, and destroys it after the preset period. The CCTV 110 transmits an image captured in response to the request of the precipitation determination device 120 or an image previously recorded and stored to the precipitation determination device 120 . Although it is shown in FIG. 1 that the CCTV is disposed on the golf course, it is not necessarily limited thereto, and any area may be used as long as the CCTV can exist.

The precipitation determination device 120 analyzes the CCTV image of the area requested for determination from the user terminal 130 to determine the weather and precipitation in the area. The precipitation determination device 120 communicates with the CCTV 110 in various regions, and may receive an image captured by the CCTV 110 or previously recorded and stored. Accordingly, the precipitation determination device 120 receives an image from the CCTV of the area in which the precipitation determination request is received from the user terminal 130 . Since the precipitation determination device 120 determines whether precipitation is present from an image of a specific local area having real-time characteristics, accurate information in time or location can be delivered to the user.

The precipitation determination device 120 stores a first learning model for classifying weather by receiving an image of the CCTV 110 and a second learning model for detecting a precipitation component in the image. The precipitation determination device 120 analyzes the presence of precipitation and the degree of precipitation in the corresponding area using each learning model. The precipitation determination device 120 transmits the analyzed information to the user terminal 130 so that the user can grasp the weather, precipitation presence, and precipitation degree in the corresponding area. At this time, the precipitation determination device 120 also transmits the CCTV image received from the CCTV 110 together with the corresponding information, so that the user can also check the CCTV image of the corresponding area. Specific details of the precipitation determination device 120 will be described later with reference to FIGS. 2 to 4 .

The user terminal 130 receives an input from the user for an area (hereinafter, abbreviated as 'confirmation area') in which the user wants to check whether precipitation is present, and transmits it to the precipitation determination device 120, and determines whether precipitation is precipitation in the precipitation determination device 120 ) and output to the user. The user inputs an area to check whether rain or snow is coming to the user terminal 130 , and the user terminal 130 transmits the received area information to the precipitation determination device 120 . The user terminal 130 receives, from the precipitation determining device 120, whether precipitation is in the area input by the user, and outputs it so that the user can check whether precipitation is in the area in real time. The user terminal 130 may be implemented as any device capable of communicating with an external device and outputting specific information, and may be implemented as a smart phone as a representative example.

2 is a diagram illustrating a configuration of an apparatus for determining precipitation according to an embodiment of the present invention.

Referring to FIG. 2 , the precipitation determination apparatus 120 according to an embodiment of the present invention includes a communication unit 210 , a memory unit 220 , a determination unit 230 , and a control unit 240 . Furthermore, the precipitation determination device 120 may further include a learning unit 250 .

The communication unit 210 wirelessly communicates with the CCTV 110 , and receives an image photographed or previously recorded and stored from the CCTV 110 . The video captured by the CCTV is stored in a server or database that is electrically connected to the CCTV or connected wirelessly. The communication unit 210 communicates with a corresponding server or database in each region, and may receive an image photographed or previously photographed and stored from the CCTV 110 . The communication unit 210 receives an image while communicating with the CCTV 110 closest to a specific area, more specifically, a confirmation area (point) under the control of the control unit 240 .

The communication unit 210 wirelessly communicates with the user terminal 130 , receives information about the confirmed area from the user terminal 130 , and transmits the weather of the confirmed area and whether precipitation in the confirmed area to the user terminal 130 . . The communication unit 210 receives information about the confirmation area from the user terminal 130 . The information on the confirmed area may be implemented as any means for recognizing the same point between the names 120 and 130, such as a place name, address, or coordinates of the corresponding area. The communication unit 210 transmits the received information to the determination unit 230 . In addition, the communication unit 210 may transmit the CCTV image of the confirmed area received from the CCTV 110 together with the weather of the confirmed area and the presence of precipitation in the confirmed area. As the CCTV images are transmitted together, the user can use the user terminal 130 to check all information about the weather and precipitation together with the CCTV images of the area.

The memory unit 220 stores a first learning model for classifying weather from the CCTV image received by the communication unit 210 and a second learning model for detecting a precipitation component in the image.

The first learning model is a model for classifying weather in an image by receiving an image using a convolutional neural network (CNN). The first learning model receives as input data an image of each weather situation as shown in FIG. 3 .

FIG. 3 is a diagram illustrating data that the precipitation determination apparatus learns for weather classification according to an embodiment of the present invention.

Fig. 3 (a) is an image with sunny weather without precipitation, Fig. 3 (b) is an image with raining or falling weather, and Fig. 3 (c) is an image with snowing or falling weather. .

The first learning model learns with a convolutional neural network by using various images having the weather of each situation as input values and weather information (sunny, raining, snowing, etc.) displayed in the images as output values. More specifically, the weather information, which is an output value, may be classified by a difference in brightness, chromaticity, or chroma of an image. An image for a sunny day has a relatively high brightness, chromaticity or saturation, whereas an image for a rain or snowy day has a relatively low brightness, chromaticity or saturation. In addition, images of rainy days and snowy days also have different brightness, chromaticity, or saturation according to each weather feature. Using this, when the first learning model receives an image as an input value, it can classify the weather in the image by analyzing the brightness, chromaticity, or saturation in the image. For example, when the first learning model receives the image of FIG. 3A , it may output that the weather in the corresponding area is clear.

In this case, in order to increase the accuracy of classification, the first learning model may learn images that are successive images in time series as input data. By learning continuous images in time-series, it is possible to distinguish between cloudy weather and blurriness with the CCTV image itself. Although the weather itself is not cloudy, there may be cases where the image quality of the CCTV itself is low and the image is blurred, or the image is blurred because foreign substances are attached to the recording part of the CCTV. If such an image is not filtered, an inaccurate judgment result may be provided to the user. To this end, the first learning model uses time-series continuous images as input data. If the weather itself is cloudy, changes are highly likely to occur due to movement of objects or people in the images in time-series continuous images. On the other hand, if the video itself is blurry due to the physical reasons of the CCTV, there is no change in the blurry part or the whole part regardless of whether there is a substantial change in the place. Using this point, the first learning model learns continuous images, classifies the weather as a result, and at the same time can distinguish situations in which it is difficult to grasp images due to external factors of CCTV.

In addition, the first learning model may learn both images of each other according to seasons or temperatures, and images of each other according to time of day. When learning is conducted using only images of the same season or time zone (dawn, morning, afternoon, sunset, evening, etc.) uniformly, the first learning model does not classify the correct result value according to the change of season or time zone. There is a possibility that it cannot. In order to prevent this problem, the first learning model receives continuous images, images in different seasons (temperatures), or images at different times when receiving images about each weather as input data to improve classification accuracy. improve

Referring back to FIG. 2 , the second learning model is a model that similarly receives an image (image) using a convolutional neural network and detects a precipitation component in the image. The second learning model uses a convolutional neural network to use an image of precipitation, particularly a raining situation, as input data, and learns by using a precipitation component in the input data as output data. The data learned by the second learning model is illustrated in FIG. 4 .

4 is a diagram illustrating data that the precipitation determination apparatus learns to detect a precipitation component according to an embodiment of the present invention.

Referring to FIG. 4 , an image of a raining situation is learned as input data of the second learning model. If it is raining, raindrops continuously form on the outside of the CCTV, and the raindrops operate like a fisheye lens and refract light to generate distortion 420 in the image. It is self-evident that the degree of occurrence of the distortion 420 is proportional to the amount of precipitation. Using this point, the second learning model receives an image as input data, learns distortions 420 generated in the image as output data, and determines how much distortion exists in the input image. As described above, since the number of distortions is proportional to the amount of precipitation, the second learning model determines the degree of precipitation by detecting a precipitation component, especially in a raining situation.

Referring back to FIG. 2 , the memory unit 220 stores each learning model so that the determination unit 230 can analyze each result value from the image received by the communication unit 210 .

The determination unit 230 uses the learning models stored in the memory unit 220 to determine whether precipitation occurs in the corresponding area from the image received by the communication unit 210 .

The determination unit 230 classifies the weather of the corresponding area from the CCTV image (video) of the corresponding area using the first learning model stored in the memory unit 220 . The determination unit 230 inputs the CCTV image of the corresponding area as the first learning model, and classifies the weather of the corresponding area. In particular, the determination unit 230 can distinguish whether the weather in the corresponding area itself is cloudy or whether the image is blurred due to the physical reason of the CCTV using the first learning model, thereby increasing the accuracy of classification.

When it is classified as raining in the corresponding area, the determination unit 230 determines the degree of precipitation in the corresponding area using the second learning model stored in the memory unit 220 . The determination unit 230 inputs the CCTV image of the corresponding area as the second learning model, and determines how many precipitation components are captured in the image. The determination unit 230 determines the degree of precipitation according to the degree of the captured precipitation component. The determination unit 230 may classify the degree of precipitation into grades according to the degree of the captured precipitation component, or may calculate a rough amount of precipitation.

The control unit 240 controls the operation of each component in the precipitation determination device 120 .

When the communication unit 210 receives the confirmation area information from the user terminal 130, the control unit 240 determines the CCTV closest to the confirmation area, and controls the communication unit 210 to receive an image from the CCTV.

When the communication unit 210 receives an image (video) from the CCTV 130 , the control unit 240 controls the determination unit 230 to analyze the weather, precipitation, and degree of precipitation in the confirmed area based on the image.

When the determination unit 230 analyzes the weather, precipitation, and degree of precipitation in the confirmed area, the control unit 240 controls the communication unit 210 to transmit the analysis result to the user terminal 130 . In this case, the controller 240 may control the communication unit 210 to transmit the image (video) received from the CCTV 130 together with the analysis result.

According to the control in this way, the control unit 240 determines whether each component in the precipitation determining device 120 determines whether precipitation is in the confirmed area and notifies the user terminal 130 .

The precipitation determination device 120 may further include a learning unit 250 .

The learning unit 250 learns the first learning model and the second learning model stored in the memory unit 220 . The learning unit 250 receives input data and output data for each of the first learning model and the second learning model from the outside. The learning unit 250 learns the first learning model and the second learning model by using a convolutional neural network based on each received input data and output data. As the precipitation determination device 120 includes the learning unit 250 , it can directly learn and determine necessary learning models.

Accordingly, the precipitation determination apparatus 120 may analyze the real-time weather and precipitation degree of a specific area and provide it to the user by including each of the above-described components. The user can check the local (accurate) weather only in the area he wants to check in real time.

5 is a flowchart illustrating a method of determining whether precipitation is present in an image by the apparatus for determining precipitation according to an embodiment of the present invention.

The communication unit 210 receives area information for which the user wants to check whether precipitation is present from the user terminal 130 (S510),

The communication unit 210 receives the CCTV image of the corresponding area from the CCTV 110 (S520).

The analysis unit 230 analyzes the received CCTV image and classifies the weather in the corresponding area (S530). The analysis unit 230 classifies the weather in the corresponding area from the received CCTV image using the first learning model stored in the memory unit 220 .

The analysis unit 230 detects a precipitation component in the image and analyzes the degree of precipitation in the corresponding area ( S540 ). The analysis unit 230 analyzes the degree of precipitation by detecting a precipitation component in the image when it is raining in the corresponding area by using the second learning model stored in the memory unit 220 .

The communication unit 210 notifies the classified or analyzed result to the user terminal 130 (S550).

Although it is described that each process is sequentially executed in FIG. 5, this is merely illustrative of the technical idea of an embodiment of the present invention. In other words, a person of ordinary skill in the art to which an embodiment of the present invention pertains may change the order described in each drawing within a range that does not depart from the essential characteristics of an embodiment of the present invention, or perform one or more of each process. Since various modifications and variations can be applied by executing in parallel, FIG. 5 is not limited to a time-series order.

Meanwhile, the processes illustrated in FIG. 5 can be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored. That is, the computer-readable recording medium includes a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, etc.) and an optical readable medium (eg, a CD-ROM, a DVD, etc.). In addition, the computer-readable recording medium is distributed in a network-connected computer system so that the computer-readable code can be stored and executed in a distributed manner.

The above description is merely illustrative of the technical idea of this embodiment, and various modifications and variations will be possible without departing from the essential characteristics of the present embodiment by those of ordinary skill in the art to which this embodiment belongs. Accordingly, the present embodiments are intended to explain rather than limit the technical spirit of the present embodiment, and the scope of the technical spirit of the present embodiment is not limited by these embodiments. The protection scope of this embodiment should be interpreted by the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

100: precipitation judgment system
110: CCTV
120: precipitation judgment device
130: user terminal
210: communication unit
220: memory unit
230: judgment unit
240: control unit
250: study department
420: distortion

Claims (11)

In the precipitation determination device for determining the weather and precipitation in the confirmation area that the user wants to check,
a communication unit for receiving the information of the confirmed area from the user terminal and receiving CCTV images or images of the confirmed area from the CCTV, respectively, and transmitting the weather and precipitation determination results to the user terminal;
a memory unit for storing a first learning model for classifying weather from CCTV images or images and a second learning model for detecting a precipitation component in CCTV images or images;
a determination unit that classifies the weather of the confirmed area from the CCTV image or image using the first learning model, and determines how much precipitation component is captured in the CCTV image or image using the second learning model; and
a control unit for controlling the operation of the communication unit, the memory unit, and the determination unit;
The first learning model is a model learned by using images having each weather as an input value and weather information appearing in the images as an output value, and classifies the weather by differences in brightness, chromaticity or saturation of the images;
The first learning model learns by receiving and learning time-series continuous images as input data, and distinguishes whether the image is blurred due to the physical reason of the CCTV or is due to the weather,
The second learning model receives an image as input data, and determines the degree of precipitation by determining how much distortion due to raindrops exists in the input image. According to claim 1,
The first learning model,
Precipitation determination device, characterized in that it classifies whether it is snowing or raining weather or not, from CCTV images or images. 3. The method of claim 2,
The judging unit,
Precipitation determination apparatus, characterized in that when it is raining weather in the confirmed area, capturing a precipitation component in a CCTV image or image using the second learning model. According to claim 1,
Further comprising a learning unit for receiving input data for each of the first learning model and the second learning model and output data for each input data from the outside to learn the first learning model and the second learning model Precipitation judgment device, characterized in that. 5. The method of claim 4,
The learning unit,
Using convolutional neural networks, the first learning model and the second learning model are obtained from the input data for each learning of the first and second learning models and the output data for each input data. Precipitation judgment device, characterized in that learning. a user terminal for receiving information on a confirmed area in which the user wants to check weather and precipitation from the user and transmitting the information to the outside;
CCTV placed in a preset place to shoot an image around the place; and
The precipitation determination device of any one of claims 1 to 5
Precipitation judgment system comprising a. delete delete delete delete In the method for the precipitation determination device to determine the weather and precipitation in the confirmation area that the user wants to check,
a first receiving process of receiving information on the confirmed area from a user terminal;
a second receiving process of receiving a CCTV image or image from the CCTV existing in the confirmation area;
a classification process of classifying the weather of the confirmed area from the CCTV images or images received in the second reception process using a first learning model for classifying weather from CCTV images or images;
a determination process of determining how much precipitation component is captured in the CCTV image or image received in the second reception process using a second learning model for detecting a precipitation component in the CCTV image or image; and
A transmission process of transmitting the weather of the confirmed area classified in the classification process and the precipitation determination result determined in the determination process to the user terminal,
The first learning model is a model learned by using images having each weather as an input value and weather information appearing in the images as an output value, and classifies the weather by differences in brightness, chromaticity or saturation of the images;
The first learning model learns by receiving and learning time-series continuous images as input data, and distinguishes whether the image is blurred due to the physical reason of the CCTV or is due to the weather,
The second learning model receives an image as input data, and determines the degree of precipitation by determining how much distortion due to raindrops exists in the input image.

Images