KR102676906B1 - Method for monitoring wind power generator based on artificial intelligence using sound, vibration information and drond shooting photo - Google Patents

Abstract

According to one embodiment, an artificial intelligence-based wind power generator monitoring method using sound, vibration information, and drone photos performed by a device includes the steps of receiving sound information from a first sensor attached to a first generator; Receiving vibration information from a second sensor attached to the first generator; Receiving a photograph from a first drone that photographed the first generator; Analyzing the state of the first generator based on the sound information, the vibration information, and the captured photo; And based on the analysis results, an artificial intelligence-based wind power generator monitoring method using sound, vibration information, and drone photos is provided, including the step of monitoring the first generator.

Description

Artificial intelligence-based wind power generator monitoring method using sound, vibration information, and drone photos {METHOD FOR MONITORING WIND POWER GENERATOR BASED ON ARTIFICIAL INTELLIGENCE USING SOUND, VIBRATION INFORMATION AND DROND SHOOTING PHOTO}

The examples below relate to technology for monitoring wind power generators based on artificial intelligence using sound, vibration information, and drone photos.

A wind power generator is a generator that produces electrical energy from rotational energy caused by the wind. It includes a rotor with a hub to which a plurality of blades rotated by the wind are connected, and a nacelle (nacelle) connected to the rotor. It includes a nacelle cover that supports and protects the nacelle, and a tower that supports the blades, rotor, nacelle, and nacelle cover.

The blade uses an aerodynamically designed shape to generate useful aerodynamic torque from wind energy, and uses the aerodynamic torque to rotate a generator to generate electricity.

Blades, which are considered the most important part of a wind power generator, are often damaged because they always rotate in response to the wind and are exposed to the outside.

If a blade is damaged, wind power generation stops, so a process to detect and prevent blade failure in advance is necessary.

Therefore, there is a need for research and development on technology that can monitor the blades of wind power generators.

Korean Patent No. 10-2437275 Korean Patent No. 10-2322693 Korean Patent No. 10-1764535 Korean Patent No. 10-1345598

According to one embodiment, the purpose is to monitor wind power generators based on artificial intelligence using sound, vibration information, and drone photos.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned can be clearly understood from the description below.

According to one embodiment, an artificial intelligence-based wind power generator monitoring method using sound, vibration information, and drone photos performed by a device includes the steps of receiving sound information from a first sensor attached to a first generator; Receiving vibration information from a second sensor attached to the first generator; Receiving a photograph from a first drone that photographed the first generator; Analyzing the state of the first generator based on the sound information, the vibration information, and the captured photo; And based on the analysis results, an artificial intelligence-based wind power generator monitoring method using sound, vibration information, and drone photos is provided, including the step of monitoring the first generator.

Receiving sound information from the first sensor may include receiving sound information from the first sensor at preset reference periods; Generating a first input signal based on sound information received from the first sensor at a first time; Inputting the first input signal to a first artificial intelligence model learned to predict the probability of an abnormality of the generator through sound analysis; When the abnormality probability of the first generator is predicted as a first probability through the input of the first input signal, obtaining a first output signal indicating the first probability from the first artificial intelligence model; And based on the first output signal, it may include confirming that the abnormality probability of the first generator analyzed through the sound at the first time is the first probability.

Receiving vibration information from the second sensor may include receiving vibration information from the second sensor every reference period; generating a second input signal based on vibration information received from the second sensor at the first time point; Inputting the second input signal to a second artificial intelligence model learned to predict the probability of an abnormality of the generator through vibration analysis; When the abnormality probability of the first generator is predicted as a second probability through the input of the second input signal, obtaining a second output signal indicating the second probability from the second artificial intelligence model; And based on the second output signal, it may include confirming that the abnormal probability of the first generator analyzed through vibration at the first time point is the second probability.

Receiving a photograph from the first drone may include calculating an average value of the first probability and the second probability as a third probability; checking whether the third probability is outside a preset reference range; If it is confirmed that the third probability is outside the reference range, determining a drone to photograph the first generator from among a plurality of drones; When the drone to photograph the first generator is determined to be the first drone, transmitting a movement command to the first drone to move to a first point where the first generator is installed; And when the first drone moves to the first point and photographs the first generator at a second time point, it may include receiving a photograph taken at the second time point from the first drone.

Analyzing the state of the first generator includes generating a third input signal based on sound information received from the first sensor at a third time point after the second time point; Inputting the third input signal to the first artificial intelligence model; When the abnormality probability of the first generator is predicted as a fourth probability through the input of the third input signal, obtaining a third output signal indicating the fourth probability from the first artificial intelligence model; Based on the third output signal, confirming that the abnormality probability of the first generator analyzed through the sound at the third time point is the fourth probability; generating a fourth input signal based on vibration information received from the second sensor at the third time point; Inputting the fourth input signal to the second artificial intelligence model; When the abnormality probability of the first generator is predicted as a fifth probability through the input of the fourth input signal, obtaining a fourth output signal indicating the fifth probability from the second artificial intelligence model; Based on the fourth output signal, confirming that the abnormality probability of the first generator analyzed through vibration at the third time point is the fifth probability; generating a fifth input signal based on the photograph received from the first drone; Inputting the fifth input signal to a third artificial intelligence model learned to predict the probability of an abnormality of the generator through image analysis; When the abnormality probability of the first generator is predicted as a sixth probability through the input of the fifth input signal, obtaining a fifth output signal indicating the sixth probability from the third artificial intelligence model; Based on the fifth output signal, confirming that the abnormality probability of the first generator analyzed through the image at the second viewpoint is the sixth probability; calculating an average of the fourth probability, the fifth probability, and the sixth probability as a seventh probability; If the seventh probability is confirmed to be lower than a preset first reference rate, classifying the first generator as a steady state; If it is confirmed that the seventh probability is not lower than the first reference rate but is lower than a preset second reference rate, classifying the first generator into a caution state; If it is confirmed that the seventh probability is not lower than the second reference rate but is lower than a preset third reference rate, classifying the first generator into a warning state; If the seventh probability is confirmed to be not lower than the third reference rate, classifying the first generator into a critical state; If the first generator is classified as a caution state, determining that remote inspection of the first generator is necessary; When the first generator is classified as a warning state, determining that a visit inspection of the first generator is necessary; And when the first generator is classified as being in a critical state, it may include determining that shutdown of the first generator is necessary.

According to one embodiment, by monitoring wind power generators based on artificial intelligence using sound, vibration information, and drone photos, failure of blades installed on wind power generators can be detected in advance to increase the efficiency of wind power generation. It works.

Meanwhile, the effects according to the embodiments are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below.

Figure 1 is a diagram schematically showing the configuration of a system according to an embodiment.
Figure 2 is a flow chart to explain the process of monitoring a wind power generator using sound, vibration information, and drone photos according to an embodiment.
FIG. 3 is a flowchart illustrating a process for predicting an abnormality probability of a generator based on sound information according to an embodiment.
FIG. 4 is a flowchart illustrating a process for predicting an abnormality probability of a generator based on vibration information according to an embodiment.
Figure 5 is a flowchart for explaining a process of receiving a photograph from a drone according to an embodiment.
6 to 7 are flowcharts for explaining a process for checking an abnormal state of a generator according to an embodiment.
Figure 8 is a flowchart for explaining the process of setting a reference range according to an embodiment.
9 to 10 are flowcharts for explaining the process of determining a drone to photograph a generator according to an embodiment.
11 is an exemplary diagram of the configuration of a device according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, various changes can be made to the embodiments, so the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents, or substitutes for the embodiments are included in the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

The terms used in the examples are for descriptive purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the embodiments belong. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

In addition, when describing with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiments, if it is determined that detailed descriptions of related known technologies may unnecessarily obscure the gist of the embodiments, the detailed descriptions are omitted.

Embodiments may be implemented in various types of products such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices.

In an embodiment, an artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike existing rule-based smart systems, it is a system in which machines learn and make decisions on their own. As artificial intelligence systems are used, recognition rates improve and sellers' tastes can be more accurately understood, and existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning and element technologies using machine learning. Machine learning is an algorithmic technology that classifies/learns the characteristics of input data on its own, and elemental technology is a technology that mimics the functions of the human brain such as cognition and judgment by utilizing machine learning algorithms such as deep learning, including linguistic understanding and visual It consists of technical areas such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology that recognizes and applies/processes human language/characters and includes natural language processing, machine translation, conversation systems, question and answer, and voice recognition/synthesis. Visual understanding is a technology that recognizes and processes objects like human vision, and includes object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and image improvement. Inferential prediction is a technology that judges information to make logical inferences and predictions, and includes knowledge/probability-based inference, optimization prediction, preference-based planning, and recommendation. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data creation/classification) and knowledge management (data utilization). Motion control is a technology that controls the autonomous driving of vehicles and the movement of robots, and includes motion control (navigation, collision, driving), operation control (behavior control), etc.

Generally, in order to apply machine learning algorithms to real life, learning is performed using a trial and error method due to the nature of the basic methodology of machine learning. In particular, deep learning requires hundreds of thousands of iterations. It is impossible to execute this in an actual physical external environment, so instead, the actual physical external environment is virtually implemented on a computer and learning is performed through simulation.

Figure 1 is a diagram schematically showing the configuration of a system according to an embodiment.

Referring to FIG. 1, a system according to an embodiment may include a first sensor 110, a second sensor 120, a plurality of drones 200, and a device 300 that can communicate with each other through a communication network.

First, a communication network can be configured regardless of the communication mode, such as wired or wireless, and can be implemented in various forms to enable communication between servers and between servers and terminals.

The first sensor 110 is a sensor that detects sounds occurring in the surrounding area and measures the volume of the detected sounds, and may be implemented as a sound sensor.

The first sensor 110 is attached to the first generator and detects the sound generated by the rotation of the blade installed on the first generator when the first generator is operated, measures the volume of the detected sound, and provides information on the volume of the sound. Sound information can be generated from the measured values. Here, the first generator is a wind power generator, and the first sensor 110 may be attached to an area adjacent to the blade in order to detect sound generated as the blade installed on the first generator rotates.

The first sensor 110 includes a communication module, can communicate with an external device, is connected to the device 300 through a network, and can operate by a control signal received from the device 300.

The second sensor 120 is a sensor that detects vibration occurring in the surroundings and measures the magnitude of the detected vibration, and may be implemented as a vibration sensor.

The second sensor 120 is attached to the first generator, detects vibration generated by the rotation of the blade installed on the first generator when the first generator is operated, measures the magnitude of the detected vibration, and provides information about the magnitude of the vibration. Vibration information can be generated from the measured values. Here, the first generator is a wind power generator, and the second sensor 120 may be attached to an area adjacent to the blade in order to detect vibration that occurs as the blade installed on the first generator rotates.

The second sensor 120 includes a communication module, can communicate with an external device, is connected to the device 300 through a network, and can operate by a control signal received from the device 300.

The first sensor 110 and the second sensor 120 may be implemented as individually separate sensors, or may be implemented as one integrated sensor.

Each of the plurality of drones 200 may be connected to the device 300 through wireless communication and may operate according to control signals received from the device 300.

Each of the plurality of drones 200 is equipped with a camera, and a photograph or video captured through the camera can be transmitted to the device 300.

Each of the plurality of drones 200 is equipped with a 3-axis acceleration sensor, gyroscope, magnetometer, ultrasonic sensor, pressure gauge, etc., and can generate movement information such as current location, altitude, movement speed, and direction of movement, and the generated movement Information can be transmitted to the device 300.

The device 300 may be its own server owned by a person or organization that provides services using the device 300, a cloud server, or a p2p (peer-to-peer) set of distributed nodes. It may be possible. The device 300 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer. The device 300 may be equipped with at least one artificial intelligence model that performs an inference function.

The device 300 may be configured to communicate wired or wirelessly with the first sensor 110 and may control the overall operation of the first sensor 110.

The device 300 may be configured to communicate wired or wirelessly with the second sensor 120 and may control the overall operation of the second sensor 120.

The device 300 may be configured to communicate wirelessly with a plurality of drones 200 and can control the overall operation of each of the plurality of drones 200.

Meanwhile, for convenience of explanation, only the first drone 210 and the second drone 220 are shown in FIG. 1 among the plurality of drones 200, but the number of drones may vary depending on the embodiment. As long as the processing capacity of the device 300 allows, the number of drones is not particularly limited.

According to one embodiment, sound sensors, vibration sensors, drones, etc. may be used to inspect wind power generators to determine whether there is a problem with the blades of the wind power generator.

That is, the device 300 can collect various information such as sound, vibration, and captured photos to check abnormal conditions of the wind power generator, and a detailed description regarding this will be described later with reference to FIG. 2.

According to one embodiment, the device 300 can predict the probability of an abnormality of the generator based on artificial intelligence. Here, the abnormality probability of the generator may mean the probability that there is a problem with the blades of the wind generator.

In the present invention, artificial intelligence (AI) refers to technology that imitates human learning ability, reasoning ability, perception ability, etc. and implements this with a computer, and includes concepts such as machine learning and symbolic logic. may include. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the results of the analysis, and makes judgments or predictions based on the results of the learning. Additionally, technologies that mimic the functions of the human brain, such as cognition and judgment, using machine learning algorithms can also be understood as the category of artificial intelligence. For example, technical fields such as verbal understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control may be included.

Machine learning can refer to the process of training a neural network model using experience processing data. Machine learning can mean that computer software improves its own data processing capabilities. A neural network model is built by modeling the correlation between data, and the correlation can be expressed by a plurality of parameters. A neural network model extracts and analyzes features from given data to derive correlations between data. Repeating this process to optimize the parameters of the neural network model can be called machine learning. For example, a neural network model can learn the mapping (correlation) between input and output for data given as input-output pairs. Alternatively, a neural network model may learn the relationships by deriving regularities between given data even when only input data is given.

An artificial intelligence learning model or neural network model may be designed to implement the human brain structure on a computer, and may include a plurality of network nodes with weights that simulate neurons of a human neural network. A plurality of network nodes may have a connection relationship with each other by simulating the synaptic activity of neurons in which neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes are located in layers of different depths and can exchange data according to convolutional connection relationships. The artificial intelligence learning model may be, for example, an artificial neural network model (Artificial Neural Network), a convolution neural network (CNN) model, etc. As an example, an artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, and Ada-boost. , Perceptron, Genetic Programming, Clustering, etc. can be used.

Among them, CNN is a type of multilayer perceptrons designed to use minimal preprocessing. CNN consists of one or several convolution layers and general artificial neural network layers on top of them, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize input data with a two-dimensional structure. Compared to other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained via standard back propagation. CNNs have the advantage of being easier to train and using fewer parameters than other feedforward artificial neural network techniques.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increasing size of available training data and the availability of computational power, combined with algorithmic advances such as piecewise linear unit and dropout training, have led to significant improvements in many computer vision tasks. For extremely large data sets, such as those available for many tasks today, overfitting is not critical, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. For this purpose, distributed, scalable implementations of deep neural networks can be used.

Figure 2 is a flow chart to explain the process of monitoring a wind power generator using sound, vibration information, and drone photos according to an embodiment.

Referring to FIG. 2, first, in step S201, the device 300 may receive sound information from the first sensor 110 attached to the first generator. Here, the first sensor 110 may be implemented as a sound sensor attached to one side of the first generator, and the sound information may include a measurement value of the sound volume generated by the rotation of the blade installed in the first generator. there is.

When receiving sound information from the first sensor 110, the device 300 can predict the probability of an abnormality of the first generator based on the received sound information, and a detailed description regarding this will be described later with reference to FIG. 3. do.

In step S202, the device 300 may receive vibration information from the second sensor 120 attached to the first generator. Here, the second sensor 120 may be implemented as a vibration sensor attached to one surface of the first generator, and the vibration information may include a measurement value of the magnitude of vibration generated by the rotation of the blade installed in the first generator. there is.

When receiving vibration information from the second sensor 120, the device 300 can predict the probability of an abnormality of the first generator based on the received vibration information, and a detailed description regarding this will be described later with reference to FIG. 4. do.

In step S203, the device 300 may receive a photograph from the first drone 210 that photographed the first generator. Here, the first drone 210 may move to the point where the first generator is located and photograph the first generator, and the captured photo may refer to an image generated by photographing the first generator.

When receiving a photograph from the first drone 210, the device 300 first checks the abnormal state of the first generator through sound and vibration, and then, if it is determined that there is a certain degree of abnormality in the first generator, the device 300 first checks the abnormal state of the first generator through sound and vibration. 1 After controlling the drone 210 to move and photograph the first generator, a captured photograph can be received from the first drone 210, and a detailed description regarding this will be described later with reference to FIG. 5.

In step S204, the device 300 may analyze the state of the first generator based on sound information, vibration information, and captured photos.

When analyzing the state of the first generator, the device 300 can collect sound information, vibration information, and captured photos to check the abnormal state of the first generator. Refer to FIGS. 6 and 7 for a detailed description of this. It will be described later with reference.

In step S205, the device 300 may perform monitoring on the first generator based on the analysis results.

Specifically, the device 300 may analyze the state of the first generator based on sound information, vibration information, and captured photos at predetermined periods, and based on the results of analyzing the state of the first generator, the first generator Monitoring can be performed to determine which state the generator status is maintained or changed.

That is, the device 300 may monitor whether there is a problem with the wing portion of the first generator based on the analysis result.

FIG. 3 is a flowchart illustrating a process for predicting an abnormality probability of a generator based on sound information according to an embodiment.

Referring to FIG. 3, first, in step S301, the device 300 may receive sound information from the first sensor 110 every reference period. Here, the reference period may be set differently depending on the embodiment.

Specifically, the first sensor 110 may detect sound generated by the rotation of the blade installed in the first generator every reference period, generate sound information, and transmit the generated sound information to the device 300. 300 may receive sound information from the first sensor 110 every reference period.

In step S302, the device 300 may generate a first input signal based on sound information received from the first sensor 110 at a first time. Here, the first time may refer to the time when sound information was most recently received.

When generating the first input signal, the device 300 may generate the first input signal by performing normal information processing on the sound information in order to use the sound information as input to the artificial intelligence model.

In step S303, the device 300 may input a first input signal to the first artificial intelligence model learned in advance. Here, the first artificial intelligence model is trained to predict the probability of an abnormality in the generator through sound analysis, and may be learned through generator status information classified by sound.

According to one embodiment, the first artificial intelligence model may be an algorithm that receives an input signal generated by encoding sound information, then predicts the probability of an abnormality of the generator through the input signal and outputs it.

In other words, the first artificial intelligence model can predict and calculate the probability that there is a problem with the blades of the wind generator through sound analysis, and output an output signal indicating the calculated probability. At this time, the first artificial intelligence model can predict and calculate the probability of an abnormality of the generator within a range from 0% to 100%.

For example, based on sound information, the first artificial intelligence model can predict the probability of an abnormality of the generator as 10% if the sound level is confirmed to be 50dB and output an output signal indicating 10%. .

In step S304, the device 300 may obtain a first output signal indicating the first probability from the first artificial intelligence model.

Specifically, the first artificial intelligence model may predict and calculate the probability of an abnormality of the first generator as a first probability through the input of the first input signal, and output a first output signal indicating the first probability. At this time, when the abnormality probability of the first generator is predicted as the first probability through the input of the first input signal, the device 300 can obtain a first output signal indicating the first probability from the first artificial intelligence model. .

According to one embodiment, the first artificial intelligence model may be trained to analyze the probability of an abnormality in the generator based on sound. Through this, the first artificial intelligence model can analyze and output the expected probability that there is a problem with the blades of the wind generator, considering the volume of sound. To this end, the first artificial intelligence model may be pre-trained to analyze the probability of an abnormality in the generator based on sound.

The learning device in which the first artificial intelligence model is learned may be the same device as the device 300 for predicting the abnormality probability of the generator using the learned first artificial intelligence model, or may be a separate device. Below, the process by which the first artificial intelligence model is learned is explained.

First, the learning device can generate input based on sound information.

Specifically, the learning device may perform a preprocessing process on sound information. The pre-processed sound information can be used as an input to the first artificial intelligence model, or the input can be generated through normal processing to remove unnecessary information.

Next, the learning device can apply input to the first artificial intelligence model. The first artificial intelligence model may be an artificial neural network learned according to reinforcement learning. The first artificial intelligence model may be a Q-Network, Deep Q-Network (DQN), or relational network (RN) structure suitable for outputting abstract inference through reinforcement learning.

The first artificial intelligence model learned according to reinforcement learning can be updated and optimized by reflecting the evaluation of various rewards.

For example, the first compensation may increase the compensation value by analyzing that the smaller the difference between the sound volume and the preset sound reference value, the lower the probability of an abnormality in the generator, and the second compensation may increase the compensation value between the sound volume and the preset sound reference value. If the larger the difference, the higher the probability of a generator abnormality, the compensation value can be higher.

Next, the learning device may obtain an output from the first artificial intelligence model. The output of the first artificial intelligence model may be information indicating the probability of an abnormality in the generator. At this time, the first artificial intelligence model can analyze the probability of abnormality of the generator through sound information and output information about the probability of abnormality of the generator.

Next, the learning device may evaluate the output of the first artificial intelligence model and pay a reward. At this time, the evaluation of the output may be divided into first compensation, second compensation, etc.

Specifically, if the learning device analyzes that the smaller the difference between the sound volume and the preset sound reference value is, the lower the probability of an abnormality of the generator is, then it awards a larger first reward, and the larger the difference between the sound volume and the preset sound reference value, the lower the generator abnormality probability. If the probability of an abnormality is analyzed to be high, a large amount of secondary compensation can be awarded.

Next, the learning device may update the first artificial intelligence model based on the evaluation.

Specifically, the learning device is placed in a specific state so that the expected value of the sum of rewards is maximized in an environment where the first artificial intelligence model analyzes the probability of an abnormality in the generator based on sound. ), the first artificial intelligence model can be updated through the process of optimizing the policy that determines the actions to be taken.

For example, if there is no problem with the result of analyzing the abnormality probability of the first generator with the first probability, the learning device generates first learning data indicating that there is no problem with the abnormality probability analysis of the first generator, A process of applying the first learning data to the first artificial intelligence model to train the first artificial intelligence model to have a value similar to the first probability when analyzing the abnormality probability of the generator based on sounds similar to the first generator. Through this, the first artificial intelligence model can be updated.

Meanwhile, the process of optimizing the policy can be accomplished through estimating the maximum expected value of the sum of rewards or the maximum value of the Q-function, or estimating the minimum value of the loss function of the Q-function. Estimation of the minimum value of the loss function can be done through stochastic gradient descent (SGD), etc. The process of optimizing the policy is not limited to this, and various optimization algorithms used in reinforcement learning can be used.

The learning device can gradually update the first artificial intelligence model by repeating the learning process of the first artificial intelligence model described above. Through this, the learning device can learn the first artificial intelligence model that predicts and outputs the abnormality probability of the generator through sound analysis.

In other words, when the learning device analyzes the probability of an abnormality in the generator based on the sound, it can learn the first artificial intelligence model by reflecting reinforcement learning through the first or second compensation and adjusting the analysis standard. .

In step S305, the device 300 may confirm that the abnormality probability of the first generator analyzed through the sound at the first time is the first probability, based on the first output signal.

FIG. 4 is a flowchart illustrating a process for predicting an abnormality probability of a generator based on vibration information according to an embodiment.

Referring to FIG. 4 , first, in step S401, the device 300 may receive vibration information from the second sensor 120 every reference period.

Specifically, the second sensor 120 may detect vibration generated by the rotation of the blade installed on the first generator every reference period, generate vibration information, and transmit the generated vibration information to the device 300. 300 may receive vibration information from the second sensor 120 every reference period.

In step S402, the device 300 may generate a second input signal based on vibration information received from the second sensor 120 at the first time point. Here, the first time may refer to the time when vibration information was most recently received.

When generating the second input signal, the device 300 may perform normal information processing on the vibration information to use the vibration information as input to the artificial intelligence model and generate the second input signal.

In step S403, the device 300 may input a second input signal to a pre-trained second artificial intelligence model. Here, the second artificial intelligence model is trained to predict the probability of an abnormality of the generator through vibration analysis, and may be learned through generator status information classified by vibration.

According to one embodiment, the second artificial intelligence model may be an algorithm that receives an input signal generated by encoding vibration information, then predicts the probability of an abnormality of the generator through the input signal and outputs it.

In other words, the second artificial intelligence model can predict and calculate the probability that there is a problem with the blade part of the wind generator through vibration analysis, and output an output signal indicating the calculated probability. At this time, the second artificial intelligence model can predict and calculate the probability of an abnormality of the generator within a range from 0% to 100%.

For example, based on the vibration information, the second artificial intelligence model can predict the probability of an abnormality of the generator as 10% if the magnitude of the vibration is confirmed to be 50Hz and output an output signal indicating 10%. .

In step S404, the device 300 may obtain a second output signal indicating the second probability from the second artificial intelligence model.

Specifically, the second artificial intelligence model may predict and calculate the abnormality probability of the first generator as a second probability through the input of the second input signal, and output a second output signal indicating the second probability. At this time, when the abnormality probability of the first generator is predicted as the second probability through the input of the second input signal, the device 300 can obtain a second output signal indicating the second probability from the second artificial intelligence model. .

According to one embodiment, the second artificial intelligence model may be trained to analyze the probability of an abnormality of the generator based on vibration. Through this, the second artificial intelligence model can analyze and output the expected probability that there is a problem with the blades of the wind generator, considering the magnitude of the vibration. To this end, the second artificial intelligence model may be pre-trained to analyze the probability of an abnormality in the generator based on vibration.

The learning device in which learning of the second artificial intelligence model is performed may be the same device as the device 300 for predicting the probability of abnormality of the generator using the learned second artificial intelligence model, or may be a separate device. Below, the process by which the second artificial intelligence model is learned is explained.

First, the learning device can generate input based on vibration information.

Specifically, the learning device may perform a preprocessing process on vibration information. The pre-processed vibration information can be used as input to the second artificial intelligence model, or the input can be generated through normal processing to remove unnecessary information.

Next, the learning device can apply input to the second artificial intelligence model. The second artificial intelligence model may be an artificial neural network learned according to reinforcement learning. The second artificial intelligence model may be a Q-Network, Deep Q-Network (DQN), or relational network (RN) structure suitable for outputting abstract inference through reinforcement learning.

The second artificial intelligence model learned according to reinforcement learning can be updated and optimized by reflecting the evaluation of various rewards.

For example, the third compensation can increase the compensation value by analyzing that the smaller the difference between the magnitude of vibration and the preset vibration standard value, the lower the probability of an abnormality in the generator, and the fourth compensation can be used to determine the difference between the magnitude of vibration and the preset vibration standard value. If the larger the difference, the higher the probability of a generator abnormality, the higher the compensation value.

Next, the learning device may obtain an output from the second artificial intelligence model. The output of the second artificial intelligence model may be information indicating the probability of an abnormality in the generator. At this time, the second artificial intelligence model can analyze the probability of abnormality of the generator through vibration information and output information about the probability of abnormality of the generator.

Next, the learning device can evaluate the output of the second artificial intelligence model and pay a reward. At this time, the evaluation of the output can be divided into third compensation, fourth compensation, etc.

Specifically, if the learning device analyzes that the smaller the difference between the magnitude of vibration and the preset vibration reference value is, the lower the probability of an abnormality of the generator, and awards more third compensation, and the larger the difference between the magnitude of vibration and the preset vibration reference value, the lower the probability of abnormality of the generator. If the probability of an abnormality is analyzed to be high, a large number of fourth rewards can be awarded.

Next, the learning device may update the second artificial intelligence model based on the evaluation.

Specifically, the learning device is in a specific state so that the expected value of the sum of rewards is maximized in an environment where the second artificial intelligence model analyzes the probability of an abnormality of the generator based on vibration. ), the second artificial intelligence model can be updated through the process of optimizing the policy that determines the actions to be taken.

For example, if there is no problem with the result of analyzing the abnormality probability of the first generator with the second probability, the learning device generates second learning data indicating that there is no problem with the abnormality probability analysis of the first generator, The process of applying the second learning data to the second artificial intelligence model to train the second artificial intelligence model to have a value similar to the second probability when analyzing the abnormality probability of the generator based on vibration similar to that of the first generator. Through this, the second artificial intelligence model can be updated.

Meanwhile, the process of optimizing the policy can be accomplished through estimating the maximum expected value of the sum of rewards or the maximum value of the Q-function, or estimating the minimum value of the loss function of the Q-function. Estimation of the minimum value of the loss function can be done through stochastic gradient descent (SGD), etc. The process of optimizing the policy is not limited to this, and various optimization algorithms used in reinforcement learning can be used.

The learning device can gradually update the second artificial intelligence model by repeating the learning process of the second artificial intelligence model described above. Through this, the learning device can learn a second artificial intelligence model that predicts and outputs the abnormality probability of the generator through vibration analysis.

In other words, when the learning device analyzes the abnormality probability of the generator based on vibration, it can learn the second artificial intelligence model by reflecting reinforcement learning through the third or fourth compensation, etc. and adjusting the analysis standard. .

In step S405, the device 300 may confirm that the abnormality probability of the first generator analyzed through vibration at the first time is the second probability, based on the second output signal.

Figure 5 is a flowchart for explaining a process of receiving a photograph from a drone according to an embodiment.

Each step shown in FIG. 5 may be performed after each step shown in FIG. 3 and each step shown in FIG. 4 are performed.

Referring to FIG. 5, first, in step S501, the device 300 may calculate the average value of the first probability and the second probability as the third probability. Here, the first probability can be confirmed through step S305, and the second probability can be confirmed through step S405.

In step S502, the device 300 may check whether the third probability is outside the reference range. Here, the reference range may be set differently depending on the embodiment, for example, 0% to 10%. A detailed description of the process of setting the reference range will be described later with reference to FIG. 8.

If it is confirmed in step S502 that the third probability is not outside the reference range, it can be confirmed that the third probability is included in the reference range, and if it is confirmed that the third probability is included in the reference range, after a certain period of time has passed Afterwards, the process may return to steps S302 and S402 and be performed again from the process of generating the first input signal and the second input signal. At this time, the device 300 may generate a first input signal based on the most recently received sound information and a second input signal based on the most recently received vibration information.

If it is confirmed that the third probability is outside the reference range in step S502, in step S503, the device 300 may determine a drone to photograph the first generator among the plurality of drones 200.

The device 300 can determine which drone to photograph the first generator among the plurality of drones 200 by considering the location, status, battery charge amount, etc. of each of the plurality of drones 200, and a detailed description related thereto is shown in FIGS. 9 to 9 This will be described later with reference to FIG. 10.

In step S504, when the device 300 determines that the first drone 210 is the drone to photograph the first generator, the device 300 may transmit a movement command to move to the first point to the first drone 210. Here, the first point is the point where the first generator is installed, and can be confirmed through first generator information, and the first generator information can be obtained from a database or an external server.

In step S505, the device 300 may receive a photograph taken at a second time point from the first drone 210.

Specifically, when the first drone 210 receives a movement command to move to the first point from the device 300, it can move to the first point based on the movement command, and while moving to the first point, When the blades of the first generator are recognized and the blades of the first generator are focused to capture the image, and the image is generated by capturing the image at a second viewpoint, the generated image can be transmitted to the device 300. You can. At this time, when the first drone 210 moves to the first point and photographs the first generator at the second time point, the device 300 can receive a photograph taken at the second time point from the first drone 210. there is.

6 to 7 are flowcharts for explaining a process for checking an abnormal state of a generator according to an embodiment.

Each step shown in FIGS. 6 to 7 may be performed after each step shown in FIG. 5 is performed.

Referring to FIGS. 6 and 7 , first, in step S601, the device 300 may generate a first input signal based on sound information received from the first sensor 110 at a third time point. Here, the third time point is a time point after the second time point, and may refer to a time point at which sound information is newly received after receiving a photograph from the first drone 210.

In step S602, the device 300 may input a third input signal to the first artificial intelligence model.

In step S603, the device 300 may obtain a third output signal indicating the fourth probability from the first artificial intelligence model.

Specifically, the first artificial intelligence model may predict and calculate the probability of an abnormality of the first generator as a fourth probability through the input of the third input signal, and output a third output signal indicating the fourth probability. At this time, when the abnormality probability of the first generator is predicted as the fourth probability through the input of the third input signal, the device 300 can obtain a third output signal indicating the fourth probability from the first artificial intelligence model. .

In step S604, the device 300 may confirm that the abnormality probability of the first generator analyzed through the sound at the third time point is the fourth probability, based on the third output signal.

In step S605, the device 300 may generate a fourth input signal based on vibration information received from the second sensor 120 at a third time point. Here, the third time point is a time point after the second time point, and may mean a time point at which vibration information is newly received after receiving the photograph from the first drone 210.

In step S606, the device 300 may input a fourth input signal to the second artificial intelligence model.

In step S607, the device 300 may obtain a fourth output signal indicating the fifth probability from the second artificial intelligence model.

Specifically, the second artificial intelligence model may predict and calculate the probability of an abnormality of the first generator as a fifth probability through the input of the fourth input signal, and output a fourth output signal indicating the fifth probability. At this time, if the abnormality probability of the first generator is predicted as the fifth probability through the input of the fourth input signal, the device 300 may obtain a fourth output signal indicating the fifth probability from the second artificial intelligence model. .

In step S608, the device 300 may confirm that the abnormality probability of the first generator analyzed through vibration at the third time point is the fifth probability, based on the fourth output signal.

In step S609, the device 300 may generate a fifth input signal based on the photograph received from the first drone 210.

When generating the fifth input signal, the device 300 may perform normal information processing on the captured photo to use the captured photo as an input to the artificial intelligence model, thereby generating the fifth input signal.

In step S610, the device 300 may input the fifth input signal to the third artificial intelligence model learned in advance. Here, the third artificial intelligence model is trained to predict the probability of a generator abnormality through image analysis, and may be learned through generator status information classified by image.

According to one embodiment, the third artificial intelligence model may be an algorithm that receives an input signal generated by encoding a photograph, then predicts the probability of an abnormality of the generator through the input signal and outputs it.

In other words, the third artificial intelligence model can predict and calculate the probability that there is a problem with the blades of the wind generator through image analysis, and output an output signal indicating the calculated probability. At this time, the third artificial intelligence model can predict and calculate the probability of an abnormality of the generator within a range from 0% to 100%.

For example, if the third artificial intelligence model determines that the proportion of foreign matter in the blades of the generator is 10% based on the captured photo, it can predict the probability of an abnormality of the generator as 10%, and output an output indicating 10%. A signal can be output.

In step S611, the device 300 may obtain a fifth output signal indicating the sixth probability from the third artificial intelligence model.

Specifically, the third artificial intelligence model may predict and calculate the probability of an abnormality of the first generator as a sixth probability through the input of the fifth input signal, and output a fifth output signal indicating the sixth probability. At this time, when the abnormality probability of the first generator is predicted as the sixth probability through the input of the fifth input signal, the device 300 can obtain a fifth output signal indicating the sixth probability from the third artificial intelligence model. .

According to one embodiment, the third artificial intelligence model can be trained to analyze the probability of an abnormality in the generator based on the image. Through this, the third artificial intelligence model can analyze and output the expected probability that there is a problem with the wing of the wind generator, considering the size of the foreign matter attached to the wing. To this end, the third artificial intelligence model may be pre-trained to analyze the probability of an abnormality in the generator based on the image.

The learning device in which learning of the third artificial intelligence model is performed may be the same device as the device 300 for predicting the probability of abnormality of the generator using the learned third artificial intelligence model, or may be a separate device. Below, the process by which the third artificial intelligence model is learned is explained.

First, the learning device can generate input based on the captured photo.

Specifically, the learning device may perform a preprocessing process on captured photos. The pre-processed photo can be used as an input to the third artificial intelligence model, or the input can be generated through normal processing to remove unnecessary information.

Next, the learning device can apply input to a third artificial intelligence model. The third artificial intelligence model may be an artificial neural network learned according to reinforcement learning. The third artificial intelligence model may be a Q-Network, DQN (Deep Q-Network), or relational network (RN) structure suitable for outputting abstract reasoning through reinforcement learning.

The third artificial intelligence model learned according to reinforcement learning can be updated and optimized by reflecting the evaluation of various rewards.

For example, the 5th compensation can increase the compensation value by analyzing that the smaller the proportion of foreign matter in the total area of the wing recognized in the image, the lower the probability of an abnormality in the generator, and the 6th compensation can be used to determine the value of the wing recognized in the image. If it is analyzed that the larger the proportion of foreign matter in the total area, the higher the probability of a generator abnormality, the higher the compensation value can be.

Next, the learning device can obtain the output from the third artificial intelligence model. The output of the third artificial intelligence model may be information indicating the probability of an abnormality in the generator. At this time, the third artificial intelligence model can analyze the probability of abnormality of the generator through the captured photo and output information about the probability of abnormality of the generator.

Next, the learning device can evaluate the output of the third artificial intelligence model and pay compensation. At this time, the evaluation of the output can be divided into fifth compensation, sixth compensation, etc.

Specifically, the learning device awards the fifth reward more if it analyzes that the smaller the proportion of foreign matter in the total area of the wing recognized in the image, the lower the probability of an abnormality in the generator, and the more foreign matter is in the total area of the wing recognized in the image. If the larger the proportion, the higher the probability of an abnormality in the generator, the more the 6th reward can be awarded.

Next, the learning device may update the third artificial intelligence model based on the evaluation.

Specifically, the learning device is in a specific state so that the expected value of the sum of rewards is maximized in an environment where the third artificial intelligence model analyzes the probability of an abnormality of the generator based on the image. ), the third artificial intelligence model can be updated through the process of optimizing the policy that determines the actions to be taken.

For example, if there is no problem with the result of analyzing the abnormality probability of the first generator with the sixth probability, the learning device generates third learning data indicating that there is no problem with the abnormality probability analysis of the first generator, The process of applying the third learning data to the third artificial intelligence model to train the third artificial intelligence model to have a value similar to the sixth probability when analyzing the abnormality probability of the generator based on images similar to the first generator. Through this, the third artificial intelligence model can be updated.

Meanwhile, the process of optimizing the policy can be accomplished through estimating the maximum expected value of the sum of rewards or the maximum value of the Q-function, or estimating the minimum value of the loss function of the Q-function. Estimation of the minimum value of the loss function can be done through stochastic gradient descent (SGD), etc. The process of optimizing the policy is not limited to this, and various optimization algorithms used in reinforcement learning can be used.

The learning device can gradually update the third artificial intelligence model by repeating the learning process of the third artificial intelligence model described above. Through this, the learning device can learn a third artificial intelligence model that predicts and outputs the probability of an abnormality in the generator through image analysis.

In other words, when the learning device analyzes the abnormality probability of the generator based on the image, it can learn the third artificial intelligence model by reflecting reinforcement learning through the fifth or sixth compensation, etc. and adjusting the analysis standard. .

In step S612, the device 300 may confirm that the abnormality probability of the first generator analyzed through the image at the second viewpoint is the sixth probability, based on the fifth output signal.

In step S613, the device 300 may calculate the average value of the fourth probability, the fifth probability, and the sixth probability as the seventh probability.

After step S613, in step S701, the device 300 may check whether the seventh probability is lower than the first reference rate. Here, the first reference ratio may be set differently depending on the embodiment.

If the seventh probability is confirmed to be lower than the first reference rate in step S701, the device 300 may classify the first generator as in a normal state in step S702.

If it is determined in step S701 that the seventh probability is not lower than the first reference rate, in step S703, the device 300 may check whether the seventh probability is lower than the second reference rate. Here, the second reference ratio may be set to a higher value than the first reference ratio.

If the seventh probability is confirmed to be lower than the second reference rate in step S703, the device 300 may classify the first generator into a caution state in step S704.

In step S705, the device 300 may determine that remote inspection of the first generator is necessary if the first generator is classified as a caution state. At this time, the device 300 may transmit a notification message indicating that remote inspection of the first generator is necessary to the manager terminal. To this end, the device 300 may be configured to communicate wired or wirelessly with an administrator terminal.

If it is determined in step S703 that the seventh probability is not lower than the second reference rate, in step S706, the device 300 may check whether the seventh probability is lower than the third reference rate. Here, the third reference ratio may be set to a higher value than the second reference ratio.

If the seventh probability is confirmed to be lower than the third reference rate in step S706, the device 300 may classify the first generator into a warning state in step S707.

In step S708, if the first generator is classified as a warning state, the device 300 may determine that a visit inspection of the first generator is necessary. At this time, the device 300 may transmit a notification message notifying that a visit inspection of the first generator is necessary to the manager terminal.

If it is determined in step S706 that the seventh probability is not lower than the third reference rate, in step S709, the device 300 may classify the first generator into a dangerous state.

In step S710, the device 300 may determine that shutdown of the first generator is necessary if the first generator is classified as being in a critical state. At this time, the device 300 may transmit a stop operation command to the control unit of the first generator and control the operation of the first generator to stop. To this end, the device 300 may be configured to communicate wired or wirelessly with the control unit of the first generator.

Figure 8 is a flowchart for explaining the process of setting a reference range according to an embodiment.

Each step shown in FIG. 8 may be performed before step S502.

Referring to FIG. 8, first, in step S801, if it is confirmed that the first point is included in the first region, the device 300 may obtain weather information of the first region. At this time, weather information for the first region may be obtained from an external server or database.

For example, if the device 300 checks the location of the first point and determines that the first point is included in Gangnam-gu, the device 300 may obtain weather information for Gangnam-gu from an external server that manages weather information.

In step S802, if it is confirmed that the first point in time is included in the first time zone, the device 300 determines that the maximum wind speed in the first time zone is the first wind speed, based on the weather information of the first area, and It can be confirmed that the maximum rainfall in time period 1 is the first rainfall.

For example, as a result of checking the time of the first time point, the device 300 determines that the first time point is 4:32, confirms that the first time point is included in the time zone from 4:00 to 5:00, and 1 Based on local weather information, it can be confirmed that the maximum wind speed is 3 m/s from 4 to 5 o'clock, and the maximum rainfall is 10 mm from 4 to 5 o'clock.

In step S803, the device 300 may set the first reference value to a higher value within the maximum allowable range as the first wind speed increases. Here, the maximum allowable range may be set differently depending on the embodiment.

For example, if the maximum allowable range is set to 20% to 30%, the device 300 may set the first reference value to 20% if the first wind speed is confirmed to be 1 m/s, and the first wind speed may be set to 20%. If it is confirmed that this is 2m/s, the first reference value can be set to 21%.

In step S804, the device 300 may set the second reference value to a lower value within the minimum allowable range as the first rainfall amount increases. Here, the minimum allowable range may be set differently depending on the embodiment.

For example, if the minimum allowable range is set to 0% to 10%, device 300 may set the second reference value to 10% if the first rainfall amount is confirmed to be 1 mm, and if the first rainfall amount is 2 mm. If it is confirmed that it is, the second standard value can be set to 9%.

In step S805, the device 300 may set a reference range based on the first reference value and the second reference value.

For example, when the first reference value is set to 21% and the second reference value is set to 9%, the device 300 may set 9% to 21% as the reference range.

9 to 10 are flowcharts for explaining the process of determining a drone to photograph a generator according to an embodiment.

Referring to Figures 9 and 10, first, in step S901, when the address location of the first generator is confirmed as the first point, the device 300 uses the first point as the center and an area within the first reference distance as reference. Can be set to area. Here, the first reference distance may be set differently depending on the embodiment.

For example, when the first reference distance is 1 km, the device 300 may set the area within a radius of 1 km around the first point as the reference area.

In step S902, the device 300 may classify drones within the reference area into a first group based on the positions of each of the plurality of drones 200. At this time, the device 300 may obtain location information from each of the plurality of drones 200 and confirm the current location of each of the plurality of drones 200 based on the acquired location information. To this end, a GPS module is installed in each of the plurality of drones 200, and location information of the drones can be obtained through GPS signals.

In step S903, the device 300 may check whether there is a drone classified into the first group.

If it is confirmed in step S903 that there is no drone classified into the first group, in step S904, the device 300 may set the area within the second reference distance around the first point as the reference area. Here, the second reference distance may be set to a value longer than the first reference distance.

For example, when the first reference distance is 1 km and the second reference distance is 2 km, the device 300 determines that there is no drone classified into the first group, and within a radius of 1 km, centered on the first point. By expanding the reference area set as the area, an area within a radius of 2 km centered on the first point can be set as the reference area.

After step S904, returning to step S902, the device 300 may re-perform the process of classifying drones within the reference area into the first group.

If it is confirmed in step S903 that there is a drone classified as the first group, in step S905, the device 300 may check whether there is only one drone classified as the first group.

If it is confirmed in step S905 that there is only one drone classified into the first group, the device 300 may confirm the drone classified into the first group as the first drone 210 in step S906.

That is, when it is confirmed that there is only one drone classified into the first group, the device 300 can confirm that only the first drone 210 is classified into the first group.

After step S906, in step S911, when it is confirmed that there is one drone classified into the first group, the device 300 generates a first generator when the drone classified into the first group is confirmed to be the first drone 210. The drone to be photographed may be determined as the first drone 210.

If it is confirmed in step S905 that there are more than one drone classified into the first group, in step S907, the device 300 classifies the drone in a standby state among the drones classified into the first group into the second group. can do. To this end, among the plurality of drones 200, drones that are moving may be set to a moving state, drones that are stationary but not moving may be set to a standby state, and the device 300 may be configured to use drones classified into the first group. By checking the status of each drone, drones confirmed to be in a standby state can be classified into the second group.

In step S908, the device 300 may check whether there is a drone classified into the second group.

If it is confirmed in step S908 that there is no drone classified into the second group, the device 300 may check the destination of the moving path for each of the drones classified into the first group in step S909. At this time, the device 300 may receive destination information from each of the drones classified into the first group and confirm the destination for each of the drones classified into the first group.

In step S910, the device 300 may identify the drone whose destination is closest to the first point among the drones classified into the first group as the first drone 210.

That is, the device 300 may compare the distance between the destination and the first point for each of the drones classified into the first group and confirm that the distance between the destination of the first drone 210 and the first point is the closest. .

After step S910, in step S911, when it is confirmed that there is no drone classified into the second group, the device 300 checks the destination of the moving path for each of the drones classified into the first group, and If, among the drones classified into group 1, the drone whose destination is closest to the first point is identified as the first drone 210, the drone to photograph the first generator may be determined as the first drone 210.

If it is confirmed in step S908 that there is a drone classified as the second group, in step S1001, the device 300 may check whether there is only one drone classified as the second group.

If it is confirmed that there is only one drone classified into the second group in step S1001, the device 300 may identify the drone classified into the second group as the first drone 210 in step S1002.

That is, when it is confirmed that there is only one drone classified into the second group, the device 300 can confirm that only the first drone 210 is classified into the second group.

After step S1002, in step S1005, when it is confirmed that there is one drone classified into the second group, the device 300 generates the first generator when the drone classified into the second group is confirmed to be the first drone 210. The drone to be photographed may be determined as the first drone 210.

If it is confirmed in step S1001 that there are more than one drone classified into the second group, in step S1003, the device 300 determines the battery charge amount for each of the drones classified into the second group. You can check it. At this time, the device 300 may receive battery information from each of the drones classified into the second group and check the battery charge amount for each of the drones classified into the second group.

In step S1004, the device 300 may identify the drone with the highest battery charge as the first drone 210 among the drones classified into the second group.

That is, the device 300 may compare the battery charge amount of each of the drones classified into the second group and confirm that the first drone 210 has the highest battery charge amount.

After step S1004, in step S1005, when it is confirmed that there are a plurality of drones classified into the second group, the device 300 checks the battery charge amount for each of the drones classified into the second group, and selects the drones classified into the second group. If the drone with the highest battery charge among the classified drones is identified as the first drone 210, the drone to photograph the first generator may be determined as the first drone 210.

Figure 11 is an exemplary diagram of the configuration of a device according to an embodiment.

Device 300 according to one embodiment includes a processor 310 and memory 320. The processor 310 may include at least one device described above with reference to FIGS. 1 to 10 or may perform at least one method described above with reference to FIGS. 1 to 10 . A person or organization using the device 300 may provide services related to some or all of the methods described above with reference to FIGS. 1 to 10 .

The memory 320 may store information related to the methods described above or store a program implementing methods described later. Memory 320 may be volatile memory or non-volatile memory.

The processor 310 can execute programs and control the device 300. The code of the program executed by the processor 310 may be stored in the memory 320. The device 300 is connected to an external device (eg, a personal computer or a network) through an input/output device (not shown) and can exchange data through wired or wireless communication.

The device 300 may be used to learn an artificial intelligence model or use a learned artificial intelligence model. Memory 320 may include a learning or learned artificial intelligence model. The processor 310 may learn or execute the artificial intelligence model algorithm stored in the memory 320. The learning device that trains the artificial intelligence model and the device 300 that uses the learned artificial intelligence model may be the same or may be separate.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

Although the embodiments have been described with limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the following claims.

Claims (3)

In an artificial intelligence-based wind power generator monitoring method using sound, vibration information, and drone photos performed by a device,
Receiving sound information from a first sensor attached to the first generator;
Receiving vibration information from a second sensor attached to the first generator;
Receiving a photograph from a first drone that photographed the first generator;
Analyzing the state of the first generator based on the sound information, the vibration information, and the captured photo; and
Based on the analysis results, performing monitoring on the first generator,
The step of receiving sound information from the first sensor is,
Receiving sound information from the first sensor every preset reference period;
Generating a first input signal based on sound information received from the first sensor at a first time;
Inputting the first input signal to a first artificial intelligence model learned to predict the probability of an abnormality of the generator through sound analysis;
When the abnormality probability of the first generator is predicted as a first probability through the input of the first input signal, obtaining a first output signal indicating the first probability from the first artificial intelligence model; and
Based on the first output signal, confirming that the probability of an abnormality of the first generator analyzed through the sound at the first time is the first probability,
The step of receiving vibration information from the second sensor is,
Receiving vibration information from the second sensor every reference period;
generating a second input signal based on vibration information received from the second sensor at the first time point;
Inputting the second input signal to a second artificial intelligence model learned to predict the probability of an abnormality of the generator through vibration analysis;
When the abnormality probability of the first generator is predicted as a second probability through the input of the second input signal, obtaining a second output signal indicating the second probability from the second artificial intelligence model; and
Based on the second output signal, confirming that the abnormality probability of the first generator analyzed through vibration at the first time is the second probability,
The step of receiving a photograph from the first drone is,
calculating an average of the first probability and the second probability as a third probability;
checking whether the third probability is outside a preset reference range;
If it is confirmed that the third probability is outside the reference range, determining a drone to photograph the first generator from among a plurality of drones;
When the drone to photograph the first generator is determined to be the first drone, transmitting a movement command to the first drone to move to a first point where the first generator is installed; and
When the first drone moves to the first point and photographs the first generator at a second viewpoint, receiving a photograph taken at the second viewpoint from the first drone,
Artificial intelligence-based wind power generator monitoring method using sound, vibration information, and drone photos. delete According to paragraph 1,
The step of analyzing the state of the first generator is,
generating a third input signal based on sound information received from the first sensor at a third time point after the second time point;
Inputting the third input signal to the first artificial intelligence model;
When the abnormality probability of the first generator is predicted as a fourth probability through the input of the third input signal, obtaining a third output signal indicating the fourth probability from the first artificial intelligence model;
Based on the third output signal, confirming that the abnormality probability of the first generator analyzed through the sound at the third time point is the fourth probability;
generating a fourth input signal based on vibration information received from the second sensor at the third time point;
Inputting the fourth input signal to the second artificial intelligence model;
When the abnormality probability of the first generator is predicted as a fifth probability through the input of the fourth input signal, obtaining a fourth output signal indicating the fifth probability from the second artificial intelligence model;
Based on the fourth output signal, confirming that the abnormality probability of the first generator analyzed through vibration at the third time point is the fifth probability;
generating a fifth input signal based on the photograph received from the first drone;
Inputting the fifth input signal to a third artificial intelligence model learned to predict the probability of an abnormality of the generator through image analysis;
When the abnormality probability of the first generator is predicted as a sixth probability through the input of the fifth input signal, obtaining a fifth output signal indicating the sixth probability from the third artificial intelligence model;
Based on the fifth output signal, confirming that the abnormality probability of the first generator analyzed through the image at the second viewpoint is the sixth probability;
calculating an average of the fourth probability, the fifth probability, and the sixth probability as a seventh probability;
If the seventh probability is confirmed to be lower than a preset first reference rate, classifying the first generator as a steady state;
If it is confirmed that the seventh probability is not lower than the first reference rate but is lower than a preset second reference rate, classifying the first generator into a caution state;
If it is confirmed that the seventh probability is not lower than the second reference rate but is lower than a preset third reference rate, classifying the first generator into a warning state;
If the seventh probability is confirmed to be not lower than the third reference rate, classifying the first generator into a critical state;
If the first generator is classified as a caution state, determining that remote inspection of the first generator is necessary;
When the first generator is classified as a warning state, determining that a visit inspection of the first generator is necessary; and
When the first generator is classified as at risk, determining that shutdown of the first generator is necessary,
Artificial intelligence-based wind power generator monitoring method using sound, vibration information, and drone photos.