JP2020125618A - Information processing method, program, water intake control system, and learned model creation method - Google Patents

Abstract

To provide an information processing method and the like capable of properly determining quality of water upon water intake from a river.SOLUTION: An information processing method comprises the steps of: acquiring images captured at a hydraulic power station to include a vicinity of a water intake inlet that takes water from a river; inputting the acquired images captured to include the vicinity of the intake inlet into a learned model which has learned through a deep learning to output a determination result determining whether quality of intake water is good, when inputting the images including the vicinity of the water intake inlet; and performing a process acquiring the determination result from the learned model using a computer.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing method, a program, a water intake control system, and a learned model generation method.

At a hydroelectric power plant, etc., a method for judging the current quality level of water quality when taking water from a river has been proposed. For example, in Patent Document 1, a river image is photographed by an industrial color television camera (ITV), the image data and numerical data are obtained by an image processing device, and this data is stored in a data storage unit together with water quality data. The quality of water is determined by comparing the current data obtained by the image processing apparatus with the past data obtained from the data storage unit by the water quality determination unit. A water intake control system is disclosed in which the water intake determination unit determines the gate opening of the water intake gate and water intake stop/restart based on the determination result and the water level detected by the water level detector, and the gate control command unit controls the water intake gate. ..

JP-A-6-346426

However, the invention according to Patent Document 1 determines the quality level of water quality by comparing the current river image with the river image stored in the data storage unit, and the computer determines that the water quality is good. It was necessary to manually create a rule for making a judgment, that is, a judgment logic.

In one aspect, an object of the present invention is to provide an information processing method and the like capable of appropriately determining the quality of water when taking water from a river.

In one aspect, the information processing method acquires an image captured so as to include the vicinity of an intake for taking water from a river in a hydroelectric power plant, and determines whether the intake is good or bad when an image including the vicinity of the intake is input. Inputting an image including the acquired vicinity of the intake to a learned model that has been learned by deep learning so as to output the determined determination result, and causing a computer to execute a process of acquiring the determination result from the learned model. Is characterized by.

According to one aspect, the quality of water can be appropriately determined when water is taken from a river.

It is a schematic diagram which shows the structural example of the water intake control system which concerns on Embodiment 1. It is a block diagram showing an example of composition of an information processor (server). It is explanatory drawing regarding the production|generation process of the learned model. It is explanatory drawing regarding the sample of the teacher data of the learned model. It is explanatory drawing regarding the production|generation process of a 2nd learned model. It is a functional block diagram which illustrates the functional part contained in the control part of an information processor (server). It is a flow chart which shows an example of a processing procedure by a control part of an information processor (server). 9 is a functional block diagram illustrating a functional unit included in a control unit according to a second embodiment (imaging condition). FIG. It is a flow chart which shows an example of a processing procedure by a control part of an information processor (server). It is a functional block diagram which illustrates the functional part contained in the control part concerning Embodiment 3 (imaging time zone). It is a flow chart which shows an example of a processing procedure by a control part of an information processor (server).

Hereinafter, the present invention will be described in detail with reference to the drawings showing an embodiment thereof.
(Embodiment 1)
FIG. 1 is a schematic diagram showing a configuration example of a water intake control system S according to the first embodiment. When operating a water storage type hydroelectric power plant (not shown), the water of the river R taken from the intake 4 is used as the water for rotating the turbine of the generator provided in the hydroelectric power plant. As shown in FIG. 1, the river R is provided with equipment including a plurality of water intakes 4 along the flow of the river R. A dam 5 that dams the flow (water flow) of the river R is provided downstream of the water intake 4. The dam 5 includes a sand discharge gate 51 for discharging earth and sand, and an overflow portion 52 provided near the upstream base of the dam 5.

When hydroelectric power generation is not performed, the sand removal gate 51 of the dam 5 is open (open) and the intake 4 is closed (closed), and the water of the river R passes through the dam 5 and reaches the downstream Flow to. When performing hydroelectric power generation, the sand discharge gate 51 of the dam 5 is closed (closed) and the intake 4 is open (open), and the water in the river R near the intake 4 is in the intake 4 To the hydroelectric power station located downstream of the intake 4.

The water intake control system controls the opening and closing of the sand discharge gate 51 and the intake 4 based on the presence or absence of foreign matter in the sand discharge gate 51 of the dam 5, and the state of the water in the river R near the intake 4. The information processing apparatus 1, the imaging unit 2, and the water intake control panel 3 are included. The information processing device 1, the imaging unit 2, and the intake control panel 3 are communicably connected.

The image capturing unit 2 is configured by, for example, an ITV (industrial television) or a camera, captures a moving image, or periodically captures a still image, and captures the captured image (moving image or still image) as the information processing device 1 Output to. The image capturing unit 2 captures an image of the river R and the sand discharge gate 51 near the water intake 4 (so as to fit within the frame). That is, in the image captured by the image capturing unit 2, the river R and the sand discharging gate 51 near the water intake 4 are stored as subjects. Alternatively, the image pickup unit 2 is composed of a plurality of ITVs, and the image pickup unit 2 includes an ITV that captures an image so as to include the intake port 4 (to fit in the frame) and a sand removal gate 51 (the frame). It may also include the ITV to be imaged (to fit within). Each of these ITVs outputs each captured image to the information processing device 1.

The information processing apparatus 1 is an information processing apparatus 1 capable of various information processing and information transmission/reception, and is, for example, a server device, a personal computer, or the like. The server device includes not only a single server device but also a cloud server device or a virtual server device configured by a plurality of computers. In the present embodiment, the information processing device 1 is assumed to be a server device, and will be read as server 1 for simplicity in the following description. The server 1 acquires an image captured by the image capturing unit 2 and performs processing to determine from the image whether the water in the river R near the intake 4 is good or not, and the presence or absence of foreign matter in the sand discharge gate 51 of the dam 5. The server 1 uses a learned model 101 and a second learned model 102, which will be described later, in making these determinations. The server 1 generates a signal for starting water intake based on these determination results, and outputs the generated signal to the water inlet control panel 3.

The intake control panel 3 is communicably connected to a sand removal gate drive unit (not shown) for opening and closing the sand removal gate 51 and an intake drive unit (not shown) for opening and closing the intake port 4. By controlling the opening and closing of the sand discharge gate 51 and the intake port 4 by outputting a drive control signal to the sand discharge gate drive unit and the intake port drive unit. The intake control panel 3 outputs a drive control signal based on the signal output from the information processing device 1 when controlling the opening and closing of the sand discharge gate 51 and the intake 4.

FIG. 2 is a block diagram showing a configuration example of the information processing device (server 1). The server 1 includes a control unit 10, a storage unit 11, and a communication unit 12.

The control unit 10 has an arithmetic processing unit such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), and GPUs (Graphics Processing Units), and reads the program P stored in the storage unit 11. By executing the above, various information processing, control processing, and the like related to the server 1 are performed. Alternatively, the control unit 10 may be composed of a quantum computer chip, and the server 1 may be a quantum computer.

The storage unit 11 includes a volatile storage area such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), and a flash memory, and a non-volatile storage area such as an EEPROM or a hard disk. The storage unit 11 stores a program P and data to be referred to during processing in advance. The program P stored in the storage unit 11 may be a program that stores the control program P read from a recording medium (not shown) readable by the server 1. Alternatively, the program P may be downloaded from an external computer (not shown) connected to a communication network (not shown) and stored in the storage unit 11. The storage unit 11 stores entity files that form the learned model 101 and the second learned model 102. These substance files may be configured as one part of the program P.

The communication unit 12 is a communication module or a communication interface for communicating with the imaging unit 2 and the intake control panel 3 by wire or wirelessly. The control unit 10 acquires (receives) the image output from the imaging unit 2 via the communication unit 12 and outputs (transmits) a signal to the water intake control panel 3.

FIG. 3 is an explanatory diagram regarding the generation processing of the learned model. The server 1 performs deep learning (deep learning) for learning the image feature amount regarding the turbidity, color tone, shading, etc. of the water of the river R in the image of the river R near the intake 4, thereby capturing the captured image. Is input, and a neural network that outputs information indicating whether the water of the river R is suitable for water intake is constructed (generated). Alternatively, a neural network that inputs the captured image and outputs numerical information regarding the turbidity of the water of the river R may be constructed (generated). For example, the neural network is a CNN (Convolution Neural Network), and an input layer that receives an input of a captured image, an output layer that outputs information (identification result) indicating the quality of water in the river R, and an image feature amount of the captured image are displayed. And an intermediate layer to be extracted.

The input layer has a plurality of neurons that receive the input of the pixel value of each pixel included in the captured image, and transfers the input pixel value to the intermediate layer. The intermediate layer has a plurality of neurons that extract the image feature amount of the captured image, and passes the extracted image feature amount to the output layer. For example, when the trained model 101 is CNN, the intermediate layer alternates between a convolution layer that convolves the pixel values of each pixel input from the input layer and a pooling layer that maps the pixel values convolved by the convolution layer. The feature amount of the image is finally extracted while compressing the pixel information of the captured image. The output layer has one or more neurons that output a determination result for determining the quality of the water in the river R near the water intake 4, and the quality of the water is determined based on the image feature amount output from the intermediate layer. Etc. That is, it is determined whether or not the water in the river R near the intake 4 is suitable for intake. That is, the learned model 101 is for determining whether or not the water in the river R near the intake 4 is suitable for intake, and is used for controlling the opening of the intake 4.

In the present embodiment, the learned model 101 is described as a CNN, but the learned model 101 is not limited to the CNN, and a neural network other than CNN, SVM (Support Vector Machine), Bayesian network, regression tree The learned model 101 constructed by another learning algorithm such as

The server 1 associates a plurality of images of the river R near the intake 4 with the quality of the water of the river R in each image, the turbidity of the water, or information indicating the quality of the water and the turbidity of the water. Learning is performed using the attached teacher data. The quality of the water of the river R is information indicating whether or not the water of the river R is suitable for intake of water for hydroelectric power generation. The turbidity of the water in the river R is a value indicating the turbidity of the water determined based on, for example, JIS K0101 (industrial water test method).

In hydroelectric power generation, when the turbidity of the water that rotates the water turbine is high, there is a concern that the water turbine may deteriorate due to wear or the like due to friction with impurities such as sand contained in the water. Therefore, the determination as to whether or not the water is suitable for water intake (quality of water in the river R) is made based on the color tone, shade, or turbidity of the water. That is, water with low turbidity and high transparency is suitable for intake, and water with high turbidity and low transparency is not suitable for intake. Based on such a viewpoint, the teacher data labels (associate) the quality of water intake and/or the value of turbidity (answer data) with respect to the image (problem data) of the river R near the water intake 4. Configured as a customized data set.

FIG. 4 is an explanatory diagram related to a sample of teacher data of a learned model. An image (problem data) obtained by imaging the river R near the water intake 4 is generated based on an image obtained by imaging the water in the graduated cylinder by making water poured into a container such as a graduated cylinder as shown in FIG. It may be one.

For example, a graduated cylinder into which turbid water having a turbidity (a large turbidity value) is determined so that water can be determined to be rejected is prepared, and the water in the graduated cylinder (FIG. 4A) is imaged. Further, even if the turbidity is the same, the illuminance is changed by irradiating the light, and the water (FIG. 4B) in the graduated cylinder is imaged. In this way, the illuminance is made different, and the water with the turbidity that makes the judgment of water intake is imaged, and the result that is not suitable for water intake (water intake: no judgment) is labeled (associated Used as teacher's data. Similarly, a graduated cylinder into which water with a turbidity (small turbidity value) that allows water to be judged good is prepared, and the water (FIG. 4C) in the graduated cylinder is imaged. Even if the turbidity is the same, the illuminance is changed by irradiating a light, and the water (FIG. 4D) in the graduated cylinder is imaged. In this way, the illuminance is made different, and water with turbidity that makes a good water intake is imaged, and the captured image is labeled with the result of being suitable for water intake (water intake: good judgment). ) Is used as teacher data. The image of the water in the measuring cylinder is subdivided, and the subdivided image is combined with the river R near the water intake 4 to generate an image of the river R near the water intake 4 having the turbidity. It may be. The control unit 10 of the server 1 extracts an image of water in the graduated cylinder from the input image of the graduated cylinder and subdivides the extracted image to generate a plurality of image units having a predetermined size. The control unit 10 duplicates each of the generated image units, increases the number of image units, and synthesizes each of the increased image units into the river R near the water inlet 4 to obtain the image unit, that is, the female unit. An image of the river R near the intake 4 is generated using the image of water in the cylinder. When synthesizing the increased image units into the river R near the water inlet 4, the control unit 10 generates a plurality of patterns of the image of the river R near the water inlet 4 by changing the arrangement of the respective image units. It may be one that does. Although the generation of the teacher data is supposed to be performed by the server 1, it is not limited to this. Computers other than the server 1 may perform the teaching data including the above-described synthesis of the image of water in the graduated cylinder. The server 1 may perform learning by using teacher data generated by another computer.

By using the method described above, it is possible to increase the teacher data even if the teacher data that captures the river R near the intake 4 is small. For example, even when the turbidity of the water in the river R is low throughout the year, it is possible to efficiently generate teacher data that is water that is not suitable for water intake (no water).

The server 1 inputs the image of the river R near the intake 4 included in the teacher data to the input layer, and through the arithmetic processing in the intermediate layer, determines whether the intake layer is good or bad or the turbidity value from the output layer. Get the result (identification result). The identification result output from the output layer may be a value that discretely indicates the quality or turbidity value (for example, a value of “0” or “1”), and a continuous probability value (for example, “0”). Value in the range from "" to "1").

The server 1 compares the identification result output from the output layer with the information labeled for the image (question data) in the teacher data, that is, the correct answer value (answer data), and the output value from the output layer is the correct answer value. The parameters used for the arithmetic processing in the intermediate layer are optimized so as to approach The parameter is, for example, a weight between neurons (coupling coefficient), a coefficient of an activation function used in each neuron, or the like. The method of optimizing the parameters is not particularly limited, but the server 1 optimizes various parameters using the error back propagation method, for example. The server 1 performs the above processing on each image included in the teacher data to generate the learned model 101.

FIG. 5 is an explanatory diagram regarding the generation processing of the second learned model 102. The server 1 uses the captured image as an input by performing deep learning for learning the image feature amount regarding the foreign matter or the like in the image captured by the sand removal gate 51 (within the captured image), and the presence or absence of the foreign matter or the like in the sand removal gate 51. Constructs (generates) a neural network that outputs information indicating The foreign matter or the like in the sand removal gate 51 may include not only foreign matter located on the movement path of the sand removal gate 51 when closing the sand removal gate 51 but also a person in the overflow section 52 of the dam 5. Good.

The server 1 performs deep learning for learning the image feature amount of a person in the overflow portion 52 of the dam 5, such as earth and sand accumulated on the moving path of the sand removal gate 51, in the image obtained by capturing the sand removal gate 51. By the deep learning, it is possible to construct (generate) a neural network that outputs information indicating the presence or absence of foreign matter or accumulated sediment in the sand discharge gate 51 and the presence or absence of a person in the overflow section 52 of the dam 5. The foreign matter includes, for example, driftwood and waste. Similar to the learned model 101, the neural network is, for example, a CNN, an input layer that receives an input of an image of the sand discharging gate 51, and an output layer that outputs information (identification result) indicating the presence or absence of a foreign substance or the like. And an intermediate layer for extracting the image feature amount of the captured image. The output layer has one or more neurons that output a determination result that determines the presence or absence of foreign matter or accumulated sediment in the sand removal gate 51 and the presence or absence of a person in the overflow section 52 of the dam 5, and outputs from the intermediate layer. The presence/absence of foreign matter or accumulated sediment in the sand removal gate 51 and the presence/absence of a person in the overflow section 52 of the dam 5 are determined based on the image feature amount thus obtained. That is, the second learned model is for determining the presence or absence of a foreign substance or the like in the sand removal gate 51, and is used to perform the closing control of the sand removal gate 51.

The server 1 associates a plurality of images of the sand removal gate 51 with information indicating the presence/absence of foreign matter on the moving path of the sand removal gate 51 in each image and the presence/absence of a person in the overflow section 52 of the dam 5. Learning is performed using the attached teacher data.

When closing the sand discharging gate 51, if foreign matter such as earth and sand is accumulated on the moving route of the sand discharging gate 51, the sand discharging gate 51 cannot be closed completely. Further, in the overflow portion 52 of the dam 5, for example, when a fisherman or the like is present, closing the sand discharge gate 51 raises the water level on the upstream side of the dam 5, so that the sand discharge gate 51 cannot be closed. .. Based on such a viewpoint, the teacher data is configured as a data set in which the presence or absence of foreign matter and a person (answer data) is labeled (associated) with the image (question data) obtained by capturing the sand removal gate 51. To be done.

The server 1 inputs the image of the sand removal gate 51, which is included in the teacher data, into the input layer, performs the arithmetic processing in the intermediate layer, and obtains the identification result indicating the presence or absence of the foreign matter and the human from the output layer. As with the learned model 101, the server 1 optimizes various parameters using the error backpropagation method. The server 1 performs the above processing on each image included in the teacher data to generate the second learned model 102.

A plurality of intakes 4 and dams 5 are provided in the river R nationwide. The server 1 constructs a learned model 101 and a second learned model 102 separately for each of the intake 4 and the dam 5 provided in the river R nationwide. Alternatively, the server 1 may build a learned model 101 and a second learned model 102 that are common to the intakes 4 and the dams 5 provided in a plurality of rivers R nationwide.

FIG. 6 is a functional block diagram illustrating the functional units included in the control unit 10 of the information processing device (server 1). The control unit 10 functions as the acquisition unit 103 that acquires the image received from the imaging unit 2 via the communication unit 12 by executing the program P stored in the storage unit 11. The control unit 10 functions as the learned model 101 by executing the program P stored in the storage unit 11 or reading the substance file forming the learned model 101. The control unit 10 functions as the second learned model 102 by executing the program P stored in the storage unit 11 or reading the substance file forming the second learned model 102. The control unit 10 executes the program P stored in the storage unit 11 to output a signal to the intake control panel 3 based on the determination result output from the learned model 101 or the second learned model 102. Functions as the signal generation unit 104 that generates As described above, the control unit 10 functions as the acquisition unit 103, the learned model 101, the second learned model 102, and the signal generation unit 104 by executing the program P or the like, and in FIG. These parts are shown as functional parts.

By the acquisition unit 103 acquiring an image including the river R near the intake 4 and an image including the dam 5, these images are input to the control unit 10. The control unit 10 functions as the acquisition unit 103 by executing the program P using these images as arguments of the program P, for example.

The acquisition unit 103 is described as two in order to correspond to the image including the river R near the intake 4 and the image including the dam 5, but the present invention is not limited to this. With one acquisition unit 103, an image including both the river R and the dam 5 near the intake 4 is acquired, the acquired image is subjected to, for example, filter processing, and an image of a region including the river R near the intake 4 is obtained. , The image of the region including the dam 5, and the divided images may be input to the corresponding learned model 101 and second learned model 102, respectively. Alternatively, an image including both the river R near the intake 4 and the dam 5 may be input to the learned model 101 and the second learned model 102.

The image input to the acquisition unit 103 is not limited to the case where the image captured by the image capturing unit 2 is input as it is. The image input to the acquisition unit 103 may be an image obtained by cutting out the region of the river R or the region of the dam 5 near the intake 4 in the image captured by the image capturing unit 2 as preprocessing.

The image including the river R near the intake 4 acquired by the acquisition unit 103 is input to the learned model 101. The learned model 101 receives information regarding the quality of the water in the river R, the turbidity value of the water, or the quality and the turbidity value according to the input image including the river R near the intake 4 (determination). The result) is output to the signal generation unit 104.

The image including the dam 5 acquired by the acquisition unit 103 is input to the second learned model 102. The second learned model 102, in accordance with the input image including the dam 5, information about the presence or absence of foreign matter or accumulated sediment in the sand discharge gate 51, and the presence or absence of a person in the overflow section 52 of the dam 5 (determination result). ) Is output to the signal generation unit 104.

The signal generation unit 104 generates a signal to be output (transmitted) to the intake control panel 3 based on the information (determination result) output from the learned model 101 and the second learned model 102. The signal generated by the signal generation unit 104 includes a signal for closing the sand discharge gate 51 and a signal for opening the water intake 4.

The signal generation unit 104 closes the sand removal gate 51 when the information (determination result) output from the second learned model 102 is that there is no foreign matter or the like in the sand removal gate 51 and there is no person in the overflow section 52. A signal is generated and the signal is output to the water inlet control panel 3. When the information (determination result) output from the second learned model 102 is free of foreign matter or the like in the sand removal gate 51 and there is no person in the overflow section 52, the signal generation unit 104 outputs the sand removal gate 51 multiple times. May be generated and the signal may be output to the water inlet control panel 3. The acquisition unit periodically acquires the image input from the imaging unit. Therefore, all of the information (determination result) output from the second learned model 102 by the images acquired a predetermined number of times in succession is free of foreign matter or the like in the sand discharge gate 51 and there is no human in the overflow section 52. In this case, the signal generator 104 may generate a signal for closing the sand discharge gate 51 and output the signal to the water inlet control panel 3. Alternatively, if 80% or more of the information output from the second learned model 102 is obtained from the images continuously acquired a predetermined number of times, if there is no foreign matter or the like in the sand removal gate 51 and there is no human in the overflow section 52, The signal generation unit 104 may generate a signal for closing the sand discharge gate 51 and output the signal to the water inlet control panel 3.

Further, the signal generation unit 104 generates a signal for notifying the manager of the intake control panel 3 or the person in the overflow section 52 based on the information (determination result) output from the second learned model 102. However, the signal may be output to a notification device (not shown). The notification device includes, for example, a display device such as a display for notifying a manager of the water intake control panel 3 and a sound generating device such as a speaker for notifying a person in the overflow section 52. When the image captured so as to include the dam 5 (sand removal gate 51) is input to the second learned model 102, the presence/absence of foreign matter in the sand removal gate 51 and the presence/absence of a person in the overflow 52 are determined. The result is output to the signal generator 104. The signal generation unit 104 outputs a signal to a display device for notifying the administrator of the water intake control panel 3 when the determination result indicating that there is a foreign substance or the like in the sand discharge gate 51 is output. When the determination result indicating that there is a person in the overflow unit 52 is output, the signal generation unit 104 outputs a signal to the sound generation device for notifying the person in the overflow unit 52. By generating and outputting a different signal according to the content of the determination result output from the second learned model 102, it is possible to take an appropriate measure when closing the sand removal gate 51.

The signal generation unit 104 outputs the information (determination result) output from the learned model 101 when the sand removal gate 51 is closed or the water in the river R is good, or is output from the learned model 101. When the value of the turbidity of the water is smaller than a predetermined value stored in the storage unit 11 in advance, a signal for opening the water intake 4 is generated and the signal is output to the water intake control panel 3. When the information (determination result) output from the learned model 101 is good for the water of the river R or the value of the turbidity of the water output from the learned model 101 is output to the signal generation unit 104 a plurality of times. May be smaller than a predetermined value stored in the storage unit 11 in advance, a signal for opening the water intake 4 may be generated and the signal may be output to the water intake control panel 3. In other words, the signal generation unit 104, for example, of the information (determination result) output from the learned model 101 by the images continuously acquired a predetermined number of times, as in the case of generating the signal for closing the sand removal gate 51. If all the water in the river R is good, or if 80% or more of the output information (judgment results) is good in the water in the river R, a signal to open the intake 4 is generated. However, the signal may be output to the water inlet control panel 3.

FIG. 7 is a flowchart showing an example of a processing procedure by the control unit 10 of the information processing device (server 1). The control unit 10 acquires the image of the sand discharging gate 51 captured by the image capturing unit 2 (S10). The image is an image of the sand removal gate 51 taken by the imaging unit 2 or the vicinity of the intake 4 including the sand removal gate 51. Therefore, the sand removal gate 51 is included in the image as a subject.

The control unit 10 inputs the acquired image into the second learned model 102, and acquires the determination result of the presence or absence of foreign matter or the like in the sand removal gate 51 (S11).

The control unit 10 performs a determination process (branching process based on the determination result) whether there is a foreign substance or the like in the sand discharge gate 51 based on the determination result acquired from the second learned model 102 (S12).

If there is a foreign matter or the like (S12: NO), that is, if the determination result output from the second learned model 102 is a foreign matter or the like in the sand discharge gate 51 or a person is in the overflow section 52, the control unit 10 A loop process is performed to execute the process of S10 again.

When there is no foreign matter or the like (S12: YES), that is, when the determination result output from the second learned model 102 is that there is no foreign matter or the like in the sand removal gate 51 and there is no person in the overflow section 52, the control unit 10 A signal for closing the sand discharging gate 51 is output (S13). Based on the signal output from the control unit 10 of the server 1 for closing the sand discharging gate 51, the water intake control panel 3 that receives the signal closes the sand discharging gate 51. When the sand removal gate 51 is closed (closed), the water level of the river R near the sand removal gate 51 and the intake 4 rises.

The control unit 10 acquires an image near the water intake 4 captured by the image capturing unit 2 (S14). The image is an image obtained by imaging the vicinity of the intake 4 by the imaging unit 2. Therefore, the river R near the water intake 4 is included in the image as a subject. As described above, the imaging unit 2 may be configured by one camera, and the camera may image the river R near the sand discharge gate 51 and the intake port 4. In this case, the river R near the sand discharge gate 51 and the intake 4 is imaged (stored in the frame) with a single image. Alternatively, the imaging unit 2 may be configured by a plurality of cameras, and the control unit 10 may acquire an image from a camera that captures the image including the water intake port 4 (to fit within the frame).

The control unit 10 inputs the acquired image into the learned model 101 and acquires the determination result of the quality of the water of the river R (S15).

The control unit 10 performs a determination process (branching process based on the determination result) whether the water in the river R is good based on the determination result acquired from the learned model 101 (S16).

The water of the river R is not good (S16: NO), that is, when the determination result from the learned model 101 is determined to be unsuitable for water intake (no determination), or the turbidity of the water output from the learned model 101. Is larger than the predetermined value stored in the storage unit 11 in advance, the control unit 10 performs the loop process again to execute the process of S14.

The water of the river R is good (S16: YES), that is, when the determination result from the learned model 101 is determined to be suitable for water intake (good determination), or the water output from the learned model 101 is turbid. When the degree is less than or equal to the predetermined value stored in the storage unit 11 in advance, the control unit 10 outputs a signal for opening the water intake 4 (S17).

Based on the signal output from the server 1 for opening the water intake 4, the water intake control panel 3 receiving the signal opens the water intake 4. When the water intake 4 is opened (opened), the water wheel of the generator is rotated by the water of the river R flowing from the water intake 4, and hydroelectric power generation is started.

In the present embodiment, the control unit 10 generates a signal using the learned model 101 and the second learned model 102 based on the image acquired from the imaging unit 2, and outputs the signal to the intake control panel 3. However, it is not limited to this. The control unit 10 may further create teacher data by using the image acquired from the imaging unit 2 and re-learn using the teacher data. By performing the re-learning, the accuracy of determination of the learned model 101 and the second learned model 102 can be further improved.

According to the present embodiment, when the image including the vicinity of the intake 4 is input, the learned model 101 that has been learned by the deep learning is used so as to output the determination result of the quality of the intake. When taking water, it is possible to properly judge the quality of water. Further, by applying the information processing method including the determination of the quality of water intake by the learned model 101 to the water storage type hydroelectric power plant, the turbidity degree of the stored water can be appropriately determined.

According to the present embodiment, when an image including the sand removal gate 51 is input, the second learning that has been learned by the deep learning so as to output the determination result of determining the presence or absence of the foreign matter or the accumulated sediment in the sand removal gate 51. The completed model 102 is used. By using the second learned model 102, in the sand removal gate 51 that requires a closing operation as a preparatory operation for starting the intake of water from the river R, a foreign matter or sediment that obstructs the closing operation of the sand removal gate 51 Presence/absence can be appropriately determined. Further, when the image including the dam 5 is input, the second learned model 102 that has been learned by deep learning is further output so as to output the determination result of determining the presence/absence of a person in the overflow portion 52 provided in the dam 5. By using it, it is possible to appropriately determine the presence or absence of a person in the overflow section 52.

According to the present embodiment, by the water intake control system including the learned model 101 and the second learned model 102, it is possible to appropriately determine the presence/absence of foreign matter or sediment in the sand discharge gate 51, and the quality of water intake. A water intake control system that efficiently controls water intake can be provided.

(Embodiment 2)
FIG. 8 is a functional block diagram illustrating the functional units included in the control unit 10 according to the second embodiment (imaging conditions). In the present embodiment, the server 1 includes an image capturing condition determining unit 106 and a switching unit 105 as functional units in the control unit 10, and the learned model 101 and the second learned model 102 are respectively based on the image capturing time as the image capturing condition, for example. Is different from that of the first embodiment. In addition, about the content which overlaps with Embodiment 1, the same code|symbol is attached|subjected and description is abbreviate|omitted.

The control unit 10 has a clock function and a calendar function, and functions as the imaging condition determination unit 106 by executing the program P stored in the storage unit 11. The control unit 10 functions as the learned model 101 as in the first embodiment by executing the program P stored in the storage unit 11 or reading the substance file forming the learned model 101.

The learned model 101 includes a plurality of learned models 101 learned by different images according to the imaging timing. The plurality of learned models 101 are, for example, the learned model 101 (March to August) learned based on the image captured from March to August, and the image captured from September to February. It is the learned model 101 (September to February) learned based on the above.

The control unit 10 functions as the second learned model 102 by executing the program P stored in the storage unit 11 or reading the substance file forming the second learned model 102.

The second learned model 102 includes a plurality of second learned models 102 learned by different images according to the imaging timing. The plurality of second learned models 102 are, for example, the second learned model 102 (March to August) learned based on the image captured from March to August, and the image captured from September to February. It is the second learned model 102 (September to February) learned based on the acquired image.

The control unit 10 executes the program P stored in the storage unit 11 to input the image acquired by the acquisition unit 103 based on the image capturing time (image capturing condition) determined by the image capturing condition determining unit 106. It functions as a switching unit 105 that switches the already-used model 101 and the second learned model 102.

The imaging condition determination unit 106 determines the imaging timing, which is an imaging condition, based on the current date. As described above, since the control unit 10 has the calendar function, it is possible to obtain the current date and month. The image capturing condition determining unit 106 inputs information regarding the image capturing time into each of the switching units 105 corresponding to each of the learned model 101 and the second learned model 102.

The switching unit 105 switches to the learned model 101 and the second learned model 102 corresponding to the input imaging time (imaging condition) based on the input imaging time. As a result, the image acquired by the acquisition unit 103 is input to the switched learned model 101 and second learned model 102. For example, when the current month is May, the switching unit 105 causes the learned model 101 (March to August) and the second learned model learned from the images (spring and summer images) from March to August. Switch from 102 (March to August). For example, when the current month is December, the switching unit 105 causes the learned model 101 (September to February) and the second learned model 102 learned from the images (fall/winter images) from September to February. (September to February) Switch.

The learned model 101 and the second learned model 102 learned by the images different according to the image pickup time are not limited to those in the semi-annual unit, and correspond to each month learned by the image of each month. The learned model 101 and the second learned model 102 may be used.

According to the present embodiment, the image is input to different learned models 101 according to the image capturing timing (image capturing condition) and the determination result is acquired, so that an appropriate learned model corresponding to the image capturing condition is acquired. Using 101, it is possible to accurately determine the quality of water.

FIG. 9 is a flowchart showing an example of a processing procedure by the control unit 10 of the information processing device (server 1). The control unit 10 determines the imaging condition and switches the learned model 101 and the second learned model 102 (S20). The control unit 10 acquires the present month (imaging time) month or month/day as an imaging condition by exerting a calendar function. Depending on the current month, it is determined which of the learned model 101 and the second learned model 102 to be used, and the imaging unit 2 sets the learned model 101 and the second learned model 102 corresponding to the month. Switch so that the captured image is input. The switching process for the learned model 101 and the second learned model 102 is executed by the internal process of the program P that performs conditional branching using, for example, a case statement or a switch statement.

The control unit 10 performs the processing from S21 to S28, similarly to the processing S10 to S17 of the first embodiment. It goes without saying that the second learned model 102 in S22 and the learned model 101 in S26 are the learned model 101 and the second learned model 102 switched in the process of S20.

By switching to the learned model 101 and the second learned model 102 according to the imaging timing and using the switched learned model 101 and the second learned model 102, the quality of the water in the river R and the sand removal can be accurately performed. It is possible to acquire the determination result of the presence/absence of foreign matter or the like in the gate 51 (including the presence/absence of a person in the overflow section 52). In the present embodiment, the learned model 101 and the second learned model 102 are switched according to the imaging timing, but the present invention is not limited to this. The control unit 10 may switch only the learned model 101 according to the imaging timing.

(Embodiment 3)
FIG. 10 is a functional block diagram illustrating the functional units included in the control unit 10 according to the third embodiment (imaging time period). In the present embodiment, the server 1 includes the imaging condition determination unit 106 and the switching unit 105 as the functional units in the control unit 10, and the learned model 101, the second learned model 102, and the second learned model 102 based on the imaging time, for example, as the imaging condition. The difference from the first embodiment is that the images input to the learned model 101 or the second learned model 102 are switched. In addition, about the content which overlaps with Embodiment 1, the same code|symbol is attached|subjected and description is abbreviate|omitted.

The control unit 10 of the third embodiment has a clock function and a calendar function as in the second embodiment, and functions as the imaging condition determination unit 106 and the switching unit 105 by executing the program P stored in the storage unit 11. To do.

The learned model 101 includes a learned model 101 (visible light) learned by a visible light image and a learned model 101 (infrared) learned by an infrared image.

The second learned model 102 includes a second learned model 102 (visible light) learned by a visible light image and a second learned model 102 (infrared) learned by an infrared image.

The imaging unit 2 includes a camera (visible light camera) that captures a visible light image (visible light image) and a camera (infrared camera) that captures an infrared image (infrared image). Alternatively, the image capturing unit 2 may be a compound camera (hybrid camera) capable of capturing both a visible light image and an infrared image together. Therefore, the visible light image and the infrared image of the river R and the dam 5 near the intake 4 are input to the acquisition unit 103.

The imaging condition determination unit 106 determines the imaging time, which is the imaging condition, based on the current time. As described above, since the control unit 10 has the clock function, the current time can be acquired. The image condition determining unit inputs information regarding the imaging time to each of the switching units 105 corresponding to each of the learned model 101 and the second learned model 102.

The switching unit 105, based on the input imaging time (imaging condition), any image (visible light image or infrared image) corresponding to the time, and the learned model 101 and the second learned model corresponding to the image. Switch to 102. For example, when the imaging time is in the daytime, the switching unit 105 switches to the visible light image and the learned model 101 and the second learned model 102 learned by the visible light image. When the imaging time is in the evening time zone, the switching unit 105 switches to the infrared image and the learned model 101 and the second learned model 102 learned by the infrared image.

The imaging condition determination unit 106 determines the imaging time, and the switching unit 105 switches to the image, the learned model 101, and the second learned model 102 based on the input imaging time, but the invention is not limited to this. The imaging condition determination unit 106 determines, for example, the current day and night based on an output value (illuminance) from an illuminance meter communicatively connected to the server 1, or stores the illuminance and the storage unit 11 in advance. The image pickup condition may be determined by comparing with a predetermined value. When the current time is daytime or when the illuminance is larger than the predetermined value, the switching unit 105 switches to the visible light image and the learned model 101 and the second learned model 102 learned by the visible light image. When the current time is at night or the illuminance is equal to or less than a predetermined value, the switching unit 105 may switch to the infrared image and the learned model 101 and the second learned model 102 learned by the infrared image.

FIG. 11 is a flowchart illustrating an example of a processing procedure by the control unit 10 of the information processing device (server 1). The control unit 10 determines the imaging condition and switches the input image, the learned model 101, and the second learned model 102 (S30). The control unit 10 acquires, for example, a current time (imaging time) as an imaging condition by exercising a clock function. Alternatively, the control unit 10 acquires an output value (illuminance) from an illuminance meter connected to be communicable. The control unit 10 determines, for example, day and night at the time of image capturing based on the acquired time (image capturing time) or illuminance, and determines the image capturing condition. When the image capturing time is in the daytime, the control unit 10 switches the input image to the visible light image, and the visible light image is the learned model 101 and the second learned already learned by the visible light image. The input is switched to the model 102. When the image capturing time is in the evening time zone, the control unit 10 switches the input image to the infrared image, and the infrared image becomes the learned model 101 and the second learned model 102 learned by the infrared image. Switch to input. Similar to the second embodiment, such switching is executed by the internal processing of the program P that performs conditional branching using, for example, a case statement or a switch statement.

The control unit 10 performs the processing from S31 to S38, similarly to the processing S10 to S17 of the first embodiment. The image in S32, the second learned model 102, and the image in S36 and the learned model 101 are the images switched in the process in S30, the learned model 101, and the second learned model 102. Needless to say.

The imaging condition is different depending on, for example, a time zone when the image is captured, and the control unit 10 acquires the visible light image or the infrared image according to the time zone when the image including the vicinity of the intake 4 is captured. Therefore, as the time zone, for example, by acquiring an infrared image at night, it is possible to acquire an image for the learned model 101 to appropriately determine even when the visible light image is underexposed. Since the learned model 101 is switched and input according to the visible light image or the infrared image, the visible light image or the infrared image appropriately corresponding to each learned model 101 can be input, and the quality of water is appropriately determined. Can be determined.

The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above meaning but by the scope of the claims, and is intended to include meanings equivalent to the scope of the claims and all modifications within the scope.

S Water intake control system 1 Server (information processing device)
10 Control Unit 101 Trained Model 102 Second Trained Model
103 acquisition unit 104 signal generation unit 105 switching unit 106 imaging condition determination unit 11 storage unit 12 communication unit P program 2 imaging unit 3 intake port control panel 4 intake port 5 dam 51 drainage gate 52 overflow unit
R river

Claims (10)

At the hydroelectric power station, we took an image of the area around the intake that takes water from the river,
When the image including the vicinity of the intake is input, the learned model that has been learned by deep learning so as to output the determination result that determines the quality of the intake is input, and the image including the acquired vicinity of the intake is input.
An information processing method comprising causing a computer to execute a process of acquiring the determination result from the learned model. The hydroelectric power plant is a water storage type hydroelectric power plant,
The information processing method according to claim 1, wherein the determination result of determining the degree of turbidity of the stored water is acquired from the learned model. Acquiring an image captured so as to include a dam that stores water by blocking the water flow of the river,
When an image including the dam is input, the second learned model that has been learned by deep learning is output so as to output a determination result that determines the presence or absence of foreign matter or sediment in the sand removal gate provided in the dam. Enter the image containing the sand removal gate,
The information processing method according to claim 1 or 2, wherein a determination result regarding presence/absence of foreign matter or sediment in the sand removal gate is acquired from the second learned model. The second learned model is already learned by deep learning so as to output a determination result of determining the presence/absence of a person in the overflow portion provided in the dam when the image including the dam is input,
The information processing method according to claim 3, wherein a determination result of determining the presence or absence of a person in an overflow portion provided in the dam is acquired from the second learned model. The determination result is acquired by inputting an image including the vicinity of the intake into the different learned models according to an image capturing condition of the image including the vicinity of the intake. The information processing method described in one. Acquire a visible light image or infrared image according to the time zone in which the image including the vicinity of the water intake was captured,
The information processing method according to claim 5, wherein the learned model is switched and input according to whether the visible light image or the infrared image is acquired. At the hydroelectric power station, we took an image of the area around the intake that takes water from the river,
When the image including the vicinity of the intake is input, the learned model that has been learned by deep learning so as to output the determination result of the quality of the intake is input, and the image including the acquired vicinity of the intake is input.
A program for causing a computer to execute a process of acquiring the determination result from the learned model. An intake control system for a water storage type hydroelectric power plant that takes in river water from an intake,
An imaging unit that captures an image of the vicinity of the intake and the dam to store water by blocking the water flow of the river,
An intake control panel that controls opening and closing of the intake port and opening and closing of a sand removal gate of the dam;
An information processing device that generates a signal based on an image captured by the image capturing unit and outputs the signal to the water intake control panel,
A learned model that has been learned by deep learning so as to output a determination result of whether the intake is good or bad when an image including the vicinity of the intake is input,
A second learned model that has been learned by deep learning so as to output a determination result of determining the presence or absence of foreign matter or sediment in the sand removal gate when an image including the dam is input,
The information processing device,
An image including the water intake and an image including the dam are captured by the imaging unit,
Input the acquired image including the intake into the learned model,
The acquired image including the dam is input to the second trained model,
Acquiring the determination result output by each of the learned model and the second learned model,
When the determination result acquired from the second learned model is that the foreign matter or sediment is not present, a signal for closing the sand removal gate is output to the water intake control panel,
A water intake control system which outputs a signal for opening the water intake port to the water intake control panel when the result of determination obtained from the learned model is that the water intake is good. Acquiring teacher data in which information indicating the quality of water intake is associated with an image of the vicinity of the water intake at which water is taken from a river at a hydroelectric power plant,
Deep learning based on the teacher data is performed, and when the image is input, a learned model that outputs a determination result that determines whether the water intake is good or not is generated. Generation of a learned model characterized by causing a computer to execute processing. Method. 10. The method for generating a learned model according to claim 9, wherein the teacher data uses, as problem data, an image obtained by imaging water poured into a container so as to have a predetermined turbidity.