JP2019218771A - Monitoring system - Google Patents

Abstract

To provide a monitoring system that can detect accurately floating matter that flows in a channel.SOLUTION: A monitoring system 1 includes: a monitoring camera 300 that images a channel for supplying water to facilities; a binarization processing unit 121 that binarizes each of continuous frame images which are obtained by imaging; a centroid position calculation unit 122 that calculates a centroid position C of one or more white sections J for each of binarized images obtained by binarization; a determination unit 123 that determines whether or not the one or more white sections J include floating matter based on a temporal change mode of the centroid position C; and notification means that performs predetermined notifications based on determination that at least one of the white sections J includes floating matter.SELECTED DRAWING: Figure 11

Description

  The present disclosure relates to a monitoring system, a monitoring method, and a program.

  2. Description of the Related Art Conventionally, snow jams may occur in a waterway that supplies water for power generation from a river to a power plant in winter or the like. In addition, jellyfish may flow into a waterway of a power plant that uses seawater as cooling water.

  Patent Document 1 discloses a snow jam detection device. Non-Patent Document 1 discloses a system that detects an inflow of jellyfish using an image of a surveillance camera.

JP 2001-64952 A

Hiroaki Kawasumi Two other "Jellyfish inflow detection system using surveillance camera images", [online], 2001, T.IEE Japan, Vol.121-C No.5, [Search May 28, 2018 ], Internet <URL: https://www.jstage.jst.go.jp/article/ieejeiss1987/121/5/121_5_854/_pdf>

  As described in Non-Patent Document 1, it is difficult to detect a floating object (for example, a snow jam or a jellyfish swarm) from an image of a water surface having a wave wind with the conventional technology.

  The present disclosure has been made in view of the above problems, and an object of the present disclosure is to provide a monitoring system, a monitoring method, and a program capable of accurately detecting a floating substance flowing in a water channel.

  According to an aspect of the present disclosure, the monitoring system includes: an imaging unit that captures an image of a water channel that supplies water to the facility; and a binarization unit that binarizes each of a plurality of continuous frame images obtained by imaging. In each of the binarized images obtained by binarization, based on the calculation means for calculating the position of the center of gravity of the white portion and the temporal change mode of the position of the center of gravity, whether the white portion contains a floating substance A determination unit configured to determine whether the white portion includes a floating substance; and a notification unit configured to perform a predetermined notification based on the determination that a floating substance is included in the white portion.

  According to the present disclosure, it is possible to accurately detect a floating substance flowing through a water channel.

It is a figure for explaining a snow jam. It is the figure showing the image obtained by imaging the waterway which supplies water to a power generation facility (power station). It is a figure for explaining a temporal change of a center of gravity position when a snow jam occurs. It is a figure for explaining a temporal change of a barycenter position when a snow jam does not occur. It is a figure for explaining the system configuration of a monitoring system. FIG. 4 is a diagram for explaining a coordinate system of a barycentric position of a white portion in a binarized image. It is a figure showing movement of the position of the center of gravity in the positive direction of the X-axis, and movement of the position of the center of gravity in the negative direction of the X-axis. It is a figure showing the example of a shift of the position of the center of gravity in a predetermined period. It is a figure showing movement of the center of gravity position in the positive direction of the Y-axis direction, and movement of the center of gravity position in the negative direction of the Y-axis direction. It is a figure showing the example of a shift of the position of the center of gravity in a predetermined period. FIG. 3 is a diagram illustrating a functional configuration of a server device. It is a flowchart showing the first half part of the determination processing. It is the flowchart which showed the latter half part of the determination process. FIG. 3 is a diagram illustrating a typical example of a hardware configuration of a server device. FIG. 8 is a diagram in which a method using a difference between frame images (comparative example) and feature points of processing with the present example are arranged.

  Hereinafter, a system according to each embodiment of the present invention will be described with reference to the drawings. In the following description, the same members are denoted by the same reference numerals. Their names and functions are the same. Therefore, detailed description thereof will not be repeated.

  Hereinafter, a snow jam will be described as an example of a floating substance flowing in a waterway (an object that blocks the waterway). However, the suspended matter is not limited to snow jam. For example, in the case of a power plant using seawater as cooling means, the suspended matter may be a group of jellyfish.

<A. Snow Jam>
FIG. 1 is a diagram for explaining a snow jam.

  Referring to FIG. 1, as shown in a state (i), snow accumulated on a mountain falls. Next, as shown in a state (ii), the fallen snow flows into the waterway. As a result, a snow jam occurs. Furthermore, as shown in state (iii), the upper water tank provided upstream of the power plant is clogged by snow jam. Thereby, as shown in the state (iv), the water overflows from the water channel. As a result, water (cooling water) in the water channel is not supplied to the power plant, and power generation stops, as shown in state (v). Therefore, if the occurrence of a snow jam can be detected early, it is possible to prevent the power generation from stopping.

  In addition, a place where a snow jam occurs is usually a place where workers of the power plant are not resident. Therefore, in this example, the occurrence of a snow jam is detected using an image of a waterway captured by a camera.

  By the way, from the early stage to the middle stage of the occurrence of snow jam, the water channel is not clogged with snow jam and there is a water flow. Therefore, snow jam flows through the waterways. On the other hand, in the late stage of the occurrence of snow jam, the waterway is blocked, that is, the snow starts to stay in one place. In this example, in consideration of such a feature, the detection of the snow jam from the early stage to the middle stage of the occurrence of the snow jam is performed.

  FIG. 2 is a diagram illustrating an image obtained by imaging a water channel that supplies water to a power generation facility (power station). Specifically, the image G1 is a diagram showing an image (still image, one frame image) at a certain point in time of a video (moving image). More specifically, the image G1 is an image of a place where the water intake is provided in the waterway in winter.

  Referring to FIG. 2, a plurality of snow jams (objects displayed in white) are floating on the water surface in the waterway. In addition, snow is piled on the upper part of the water intake (the upper left part of the image G1).

  In such a waterway, the state of the water surface changes due to the formation of waves due to the flow of water itself and the wind. In addition, the water surface is illuminated by nighttime illumination, and the nighttime illumination also occurs on the water floor. Daytime snowfall and nighttime snowfall also occur. As described above, in a waterway, an image to be captured is not uniform due to a difference in weather conditions including a difference in sunshine conditions including day and night.

  In this example, a system that can accurately detect the occurrence of snow jam even under such conditions will be described. As will be described in detail later, in this example, it is determined whether or not a snow jam is included in the white portion based on the temporal change of the center of gravity of the white portion in the binarized image. In other words, it is determined whether a snow jam is included in a region having a certain brightness or higher based on a temporal change mode of a center of gravity of a region having a certain brightness or higher in a frame image extracted (identified) by binarization. .

  The reason why the frame image is binarized is that when an image taken while a snow jam is occurring is checked, the snow jam appears white both day and night.

  FIG. 3 is a diagram for explaining a temporal change in the position of the center of gravity when a snow jam occurs. FIG. 3A is a diagram illustrating a frame image G21 of a video at a certain point in time and a binarized image B21 obtained by performing a binarization process on the frame image G21. FIG. 3B is a diagram illustrating a frame image G22 at a time point several tens of frames after the frame image G21 and a binarized image B22 obtained by performing a binarization process on the frame image G22. It is.

  With reference to FIGS. 3A and 3B, it can be seen that the center of gravity position C of the white portion has largely moved over time. In addition, when checking a plurality of other frame images, it was found that the center of gravity position C tends to move in one direction.

  FIG. 4 is a diagram for explaining a temporal change in the position of the center of gravity when a snow jam does not occur. FIG. 4A is a diagram illustrating a frame image G31 of a video at a certain point in time and a binarized image B31 obtained by performing a binarization process on the frame image G31. FIG. 4B is a diagram illustrating a frame image G32 at a time point several tens of frames after the frame image G31 and a binarized image B32 obtained by performing a binarization process on the frame image G32. It is.

  Referring to FIGS. 4A and 4B, the position of the center of gravity C of the white portion does not change significantly with the passage of time. In addition, when a plurality of other frame images were checked, it was found that the center of gravity position C was vibratingly moved in one place regardless of weather conditions.

  In this example, the occurrence of a snow jam is detected based on the difference in the change in the center of gravity position C between when the snow jam occurs and when it does not occur.

<B. System Configuration>
FIG. 5 is a diagram for explaining the system configuration of the monitoring system.

  Referring to FIG. 5, monitoring system 1 includes server device 100, terminal device 200, and monitoring camera 300. The monitoring camera 300 is communicably connected to the server device 100. The terminal device 200 is communicably connected to the server device 100 via a network.

  Surveillance camera 300 is installed beside waterway 500 in order to image waterway 500. In this example, the monitoring camera 300 is installed so as to capture an image of a place where the water intake port 700 is provided in order to monitor the state of the water intake port 700 in the water channel 500. The water intake 700 is provided with a fence so that large foreign matters (such as driftwood) having a predetermined shape do not flow into the power generation equipment.

  The video (moving image) captured by the monitoring camera 300 is sent to the server device 100. The video can be viewed on a device such as the terminal device 200, for example. In other words, a power station worker or the like can monitor the state of the water intake port 700 with the video.

  Further, the monitoring system 1 detects the occurrence of a snow jam using the video captured by the monitoring camera 300. Specifically, the server device 100 determines whether or not a snow jam has occurred based on a temporal change of the center of gravity position C during a predetermined period Ta (for example, several seconds). That is, whether or not a snow jam has occurred is determined for each predetermined period Ta (predetermined period Ta).

<C. Detection process>
FIG. 6 is a diagram for explaining the coordinate system of the center of gravity position C of the white portion J in the binary image.

  Referring to FIG. 6, an axis parallel to one side of the binarized image is referred to as “X axis”, and an axis perpendicular to the X axis is referred to as “Y axis”. Since the binarized image is obtained by binarizing the frame image, the X axis is an axis parallel to one side of the frame image, and the Y axis is an axis perpendicular to the one side.

  The position of the center of gravity C is defined in such an orthogonal coordinate system including the X axis and the Y axis. In the following, “positive direction” in the X-axis direction means a direction in which the X coordinate value increases (rightward in FIG. 6), and “negative direction” in the X-axis direction means that the X coordinate value is It means the direction in which it becomes smaller (leftward in FIG. 6). Further, “positive direction” in the Y-axis direction means a direction in which the Y coordinate value increases (upward in FIG. 6), and “negative direction” in the Y-axis direction indicates a direction in which the Y coordinate value decreases. (Downward in FIG. 6).

(C1. Movement in the X-axis direction)
The server device 100 determines the direction of movement of the center of gravity C in the X-axis direction between two consecutive binary images (between binary images obtained based on two consecutive frame images). Specifically, the server apparatus 100 determines whether the center of gravity position C has moved in the positive direction in the X-axis direction based on the center of gravity position C in the immediately preceding binarized image and the center of gravity position C in the latest frame image. , Or not in the X-axis direction or not in the X-axis direction.

  FIG. 7 is a diagram illustrating the movement of the center of gravity C in the positive direction in the X-axis direction and the movement of the center of gravity C in the negative direction in the X-axis direction.

  Referring to FIG. 7, as shown in state (A), in the (k + 1) th frame image next to the kth (k is a natural number) frame image, the center of gravity position C has moved in the positive direction in the X-axis direction. In this case, the server device 100 increments the value of the variable Xp stored in the memory of the server device 100 (increases it by one). That is, the server device 100 uses the variable Xp to count the number of movements in the positive X-axis direction during the predetermined period Ta.

  As shown in state (B), in the (k + 1) th frame image next to the k-th (k is a natural number) frame image, when the center of gravity position C moves in the negative direction along the X-axis, the server apparatus 100 Increments the value of the variable Xn stored in the memory of the server device 100 (increases it by one). That is, the server device 100 uses the variable Xn to count the number of times of movement in the negative direction in the X-axis direction during the predetermined period Ta.

  As described above, the value of the variable Xp indicates the number of movements in the positive direction in the X-axis direction during the predetermined period Ta, and the value of the variable Xn indicates the movement in the negative direction in the X-axis direction during the predetermined period Ta. Indicates the number of times.

  The server device 100 determines whether or not the white portion is a snow jam based on the number of movements of the center of gravity C in the positive direction and the number of movements in the negative direction. Typically, the absolute value Dx (Dx = | Xp−Xn |) of the difference between the number of movements in the positive direction and the number of movements in the negative direction is calculated, and the absolute value Dx of the difference is set to a predetermined threshold. It is determined that the white portion is a snow jam on condition that it is equal to or greater than Th.

  Note that, instead of using two variables such as the variables Xp and Xn, the number of movements in the positive direction in the X-axis direction and the number of movements in the negative direction The difference from the number of times of movement may be calculated. For example, when the position of the center of gravity C moves in the positive direction in the X-axis direction, the server device 100 increments the value of the variable X (increases by 1), and moves the position of the center of gravity C in the negative direction in the X-axis direction. In this case, the value of the variable X may be decremented (decreased by 1).

  In addition, without using a variable that is updated each time the above-described movement is performed, the frame number (identification number of the frame image) or the number of the binarized image (identification number) is shifted in a positive direction. In the case of movement, a numerical value “+1” may be associated, and in the case of movement in a negative direction, a numerical value “−1” may be associated. In this case, after the predetermined period Ta has elapsed, the above difference is obtained by calculating the sum (Σ) of the numerical values associated with each frame.

  In any case, the server device 100 determines whether or not the white portion is a snow jam based on the number of movements of the center of gravity position C in the positive direction and the number of movements in the negative direction.

  FIG. 8 is a diagram illustrating an example of movement of the center of gravity position C during the predetermined period Ta. Specifically, FIG. 8 is a diagram illustrating a configuration for calculating the sum (和) of numerical values associated with each frame after a predetermined period Ta has elapsed, as described above.

  Referring to FIG. 8, when the position of the center of gravity C moves in the positive direction in the X-axis direction, a numerical value “+1” is associated with a frame number (the identification number of the current frame image). When the center of gravity position C moves in the negative direction in the X-axis direction, a numerical value “−1” is associated with the frame number. If the center of gravity position C does not move in the X-axis direction, 0 is associated with the frame number.

  As described above, the server device 100 calculates the sum (Σ) of the numerical values associated with each frame after the elapse of the predetermined period Ta. If the sum is equal to or larger than the threshold Th, the server device 100 determines that a snow jam has occurred. If the sum is less than the threshold Th, the server device 100 determines that a snow jam has not occurred.

(C2. Movement in the Y-axis direction)
The server device 100 determines the direction of movement of the center of gravity C in the Y-axis direction between two consecutive binary images. Specifically, the server apparatus 100 determines whether the center of gravity position C has moved in the positive direction in the Y-axis direction based on the center of gravity position C in the immediately preceding frame image and the center of gravity position C in the latest binary image. , Or not in the Y-axis direction or not in the Y-axis direction.

  FIG. 9 is a diagram illustrating the movement of the center of gravity C in the positive direction in the Y-axis direction and the movement of the center of gravity C in the negative direction in the Y-axis direction.

  Referring to FIG. 9, as shown in state (A), in the (k + 1) th frame image next to the kth (k is a natural number) frame image, the center of gravity position C has moved in the positive direction in the Y-axis direction. In this case, the server device 100 increments the value of the variable Yp stored in the memory of the server device 100 (increases it by one). That is, the server device 100 uses the variable Yp to count the number of movements in the positive direction in the Y-axis direction during the predetermined period Ta.

  As shown in the state (B), when the center of gravity position C moves in the negative direction in the Y-axis direction in the (k + 1) th frame image next to the k-th (k is a natural number) frame image, the server apparatus 100 Increments the value of the variable Yn stored in the memory of the server device 100 (increases it by one). That is, the server device 100 uses the variable Yn to count the number of movements in the negative direction in the Y-axis direction during the predetermined period Ta.

  As described above, the value of the variable Yp indicates the number of movements in the positive direction in the Y-axis direction during the predetermined period Ta, and the value of the variable Yn indicates the movement in the negative direction in the Y-axis direction during the predetermined period Ta. Indicates the number of times.

  The server device 100 determines whether or not the white portion is a snow jam based on the number of movements of the center of gravity C in the positive direction and the number of movements in the negative direction. Typically, the absolute value Dy (Dy = | Yp−Yn |) of the difference between the number of movements in the positive direction and the number of movements in the negative direction is calculated, and the absolute value Dy of the difference is set to a predetermined threshold. It is determined that the white portion is a snow jam on condition that it is equal to or greater than Th.

  It should be noted that the number of movements in the positive direction in the Y-axis direction and the number of movements in the negative direction in the Y-axis direction during a predetermined period Ta using one variable Y without using two variables such as the variables Yp and Yn. The difference from the number of times of movement may be calculated. For example, when the position of the center of gravity C moves in the positive direction in the Y-axis direction, the server apparatus 100 increments the value of the variable Y (increases by 1) and moves the position of the center of gravity C in the negative direction in the Y-axis direction. In this case, the value of the variable Y may be decremented (decreased by 1).

  In addition, without using a variable that is updated each time the above-described movement is performed, the frame number (identification number of the frame image) or the number of the binarized image (identification number) is shifted in a positive direction. In the case of movement, a numerical value “+1” may be associated, and in the case of movement in a negative direction, a numerical value “−1” may be associated. In this case, after the predetermined period Ta has elapsed, the above difference is obtained by calculating the sum (Σ) of the numerical values associated with each frame.

  In any case, the server device 100 determines whether or not the white portion is a snow jam based on the number of movements of the center of gravity position C in the positive direction and the number of movements in the negative direction.

  FIG. 10 is a diagram illustrating an example of movement of the center of gravity position C during the predetermined period Ta. More specifically, FIG. 10 is a diagram illustrating a configuration for calculating the sum (Σ) of numerical values associated with each frame after the elapse of the predetermined period Ta, as described above.

  Referring to FIG. 10, when the position of center of gravity C moves in the positive direction in the Y-axis direction, a numerical value “+1” is associated with a frame number (identification number of the current frame image). When the center of gravity position C moves in the negative direction in the Y-axis direction, a numerical value “−1” is associated with the frame number. If the center of gravity position C does not move in the Y-axis direction, 0 is associated with the frame number.

  As described above, the server device 100 calculates the sum (Σ) of the numerical values associated with each frame after the elapse of the predetermined period Ta. If the sum is equal to or larger than the threshold Th, the server device 100 determines that a snow jam has occurred. If the sum is less than the threshold Th, the server device 100 determines that a snow jam has not occurred.

  As described above, the server device 100 detects the movement in the Y-axis direction separately from the movement in the X-axis direction. Thereby, it is possible to detect the occurrence of a snow jam regardless of the posture of the monitoring camera 300 and / or the moving direction of the snow jam.

<D. Functional configuration>
FIG. 11 is a diagram for explaining the functional configuration of the server device 100. Note that FIG. 11 illustrates an example in which the number of times of movement of the center of gravity position C is counted in each of the positive direction and the negative direction. That is, the case where the variables Xp and Xn and the variables Yp and Yn are used will be described as an example.

  Referring to FIG. 11, server device 100 includes communication IF unit 101 and control unit 102. The communication IF unit 101 includes a receiving unit 111 and a transmitting unit 112. The control unit 102 includes a binarization processing unit 121, a center-of-gravity position calculation unit 122, and a determination unit 123. The determination unit 123 includes movement number counting units 131 to 134, difference calculation units 135 and 136, and comparison units 137 and 138.

(Pre-processing of judgment)
The communication unit IF 101 is an interface for communicating with an external device. The receiving unit 111 receives an image of the place where the water intake port 700 of the waterway 500 is provided from the monitoring camera 300 in real time. That is, the receiving unit 111 sequentially receives continuous frame images. Each of the frame images is sent to the binarization processing unit 121.

  The binarization processing section 121 performs a binarization process for each frame image. Specifically, the binarization processing unit 121 converts the image into two gradations of white and black. In the binarization process, white is converted when the pixel value is larger than a predetermined threshold, and black is converted when the pixel value is smaller. The binarization processing unit 121 sends the binarized image obtained by binarizing the frame image to the centroid position calculation unit 122.

  The center-of-gravity position calculator 122 calculates a center-of-gravity position C for each binarized image. The center-of-gravity position calculation unit 122 sends the center-of-gravity position C (specifically, the coordinate values in the XY coordinate system) to the determination unit 123.

  The determination unit 123 determines the direction of movement of the center of gravity position C between two consecutive binary images. Specifically, the determination unit 123 determines whether the direction of movement of the center of gravity position C is positive or negative with respect to the X-axis direction. Further, the determination unit 123 determines whether the direction of movement of the center of gravity position C is positive or negative with respect to the Y-axis direction.

(Judgment based on movement in the X-axis direction)
The number-of-movements counting unit 131 counts the number of movements of the center of gravity position C in the positive X-axis direction between two successive binary images. The number-of-movements counting section 132 counts the number of movements of the center of gravity position C in the negative X-axis direction between two consecutive binary images.

  The difference calculation unit 135 uses the information on the number of movements counted by the number-of-movements counting units 131 and 132 to determine the number of movements of the center of gravity position C in the positive X-axis direction and the number of movements of the center of gravity C during a predetermined period Ta. The absolute value Dx of the difference from the number of movements in the negative direction in the X-axis direction is calculated.

  The comparing unit 137 compares the absolute value Dx calculated by the difference calculating unit 135 with a predetermined threshold Th.

  When the absolute value Dx is equal to or larger than the threshold Th as a result of the comparison by the comparing unit 137, the determining unit 123 determines that a snow jam has occurred. In this case, determination section 123 outputs a determination result to transmitting section 112. When the absolute value Dx is less than the threshold Th, the determination unit 123 determines that a snow jam has not occurred.

(Determination based on movement in the Y-axis direction)
The movement number counting unit 133 counts the number of movements of the center of gravity position C in the positive Y-axis direction between two consecutive binary images. The number-of-movements counting unit 134 counts the number of movements of the center of gravity position C in the negative Y-axis direction between two consecutive binary images.

  The difference calculation unit 136 uses the information on the number of movements counted by the number-of-movements counting units 133 and 134 to determine the number of movements of the center of gravity position C in the positive Y-axis direction and the number of movements of the center of gravity C during a predetermined period Ta. The absolute value Dy of the difference from the number of movements in the negative direction in the Y-axis direction is calculated.

  The comparing unit 138 compares the absolute value Dy calculated by the difference calculating unit 136 with a predetermined threshold Th.

  When the absolute value Dy is equal to or greater than the threshold Th as a result of the comparison by the comparing unit 138, the determining unit 123 determines that a snow jam has occurred. In this case, determination section 123 outputs a determination result to transmitting section 112. When the absolute value Dy is less than the threshold Th, the determination unit 123 determines that no snow jam has occurred.

(Notification process)
When the determining unit 123 determines that a snow jam has occurred, the transmitting unit 112 notifies the terminal device 200 that a snow jam has occurred.

  When the terminal device 200 receives the notification from the server device 100, the terminal device 200 performs a predetermined notification on the display by displaying a predetermined message on a display or outputting a predetermined sound from a speaker.

  In addition, the server device 100 may notify the terminal device 200 and notify that a snow jam has occurred on a display or the like of the server device 100.

  As described above, the subject of the notification may be the terminal device 200 (specifically, the display and speaker of the terminal device 200) or the server device 100 itself (specifically, the display and speaker of the server device 100). Is also good. In addition to such notification to the user, the above notification from the server device 100 to the terminal device 200 may be regarded as notification (that is, notification to the device).

  Note that, even when the determination unit 123 determines that a snow jam has not occurred, the determination unit 123 may output the determination result to the transmission unit 112. In this case, the transmitting unit 112 may notify the terminal device 200 that no snow jam has occurred. When receiving the notification, the terminal device 200 may display information indicating that no snow jam has occurred. Alternatively, the terminal device 200 does not have to display information indicating that no snow jam has occurred, for example, only to return the reception confirmation of the notification to the server device 100.

(Brief Summary)
The monitoring system 1 includes a monitoring camera 300 that captures an image of a water channel 500 that supplies water to the equipment, a binarization processing unit 121 that binarizes each of a plurality of continuous frame images obtained by the imaging, and a binarization process. In each of the binarized images obtained by the above, whether the white portion is a snow jam or not is determined based on a temporal change mode of the center of gravity position C and a centroid position calculating unit 122 that calculates the center of gravity C of the white portion. And a notification unit for performing a predetermined notification based on the determination that the white portion includes snow jam.

  According to such a configuration, it is possible to accurately detect snow jam regardless of weather conditions, day and night, and lighting conditions.

<E. Control structure>
Next, the flow of processing when variables Xp and Xn and variables Yp and Yn are used will be described.

  FIG. 12 is a flowchart illustrating the first half of the determination process. FIG. 13 is a flowchart showing the latter half of the determination process. More specifically, FIGS. 12 and 13 are diagrams illustrating a flow of a process executed in each cycle (each predetermined time period Ta) in a series of processes executed in the cycle of the predetermined time period Ta. That is, the processing in FIGS. 12 and 13 is executed in each cycle (every predetermined period Ta).

  Referring to FIG. 12, in step S1, server device 100 (specifically, the processor of server device 100) resets the values of variables Xp, Xn, Yp, and Yn to zero.

  In step S2, the server device 100 executes a binarization process on one frame image to obtain a binarized image. In step S3, the center of gravity C of the white portion in the binarized image is calculated.

  In step S4, the server device 100 determines the direction of movement of the center of gravity position C in the X-axis direction on the condition that the previous center of gravity position C exists in this cycle. When it is determined that the moving direction is the positive direction (positive in step S5), in step S6, the value of the variable Xp is incremented. When it is determined that the moving direction is the negative direction (negative in step S5), in step S14, the value of the variable Xn is incremented.

  In step S7, the server device 100 determines the direction of movement of the center of gravity position C in the Y-axis direction on the condition that the previous center of gravity position C exists in this cycle. When it is determined that the moving direction is the positive direction (positive in step S8), in step S9, the value of the variable Yp is incremented. When it is determined that the moving direction is the negative direction (negative in step S8), the value of the variable Yn is incremented in step S15.

  In step S10, the server device 100 determines whether a predetermined period Ta has elapsed. In other words, since the frame rate of the frame images is constant, it is determined whether or not the number of frame images corresponding to the predetermined period Ta has been binarized.

  If it is determined that the predetermined period Ta has not elapsed (NO in step S10), server device 100 returns the process to step S2 and binarizes the next frame image. If it is determined that predetermined period Ta has elapsed (YES in step S10), and in step S11, server device 100 determines in step S11 that absolute value Dx of the difference between Xp and Xn is equal to or greater than threshold Th. It is determined whether or not.

  When it is determined that the absolute value Dx of the difference between Xp and Xn is equal to or greater than threshold Th (YES in step S11), server device 100 determines that a snow jam has occurred (step S12). In this case, in step S13, the server device 100 sends a predetermined notification to the terminal device 200. Note that, upon receiving the notification, the terminal device 200 performs a predetermined notification to the worker.

  If it is determined that the absolute value Dx of the difference between Xp and Xn is less than the threshold Th (NO in step S11), the server device 100 determines in step S16 that the absolute value of the difference between Yp and Yn It is determined whether the value Dy is equal to or greater than the threshold Th.

  When it is determined that the absolute value Dy of the difference between Yp and Yn is greater than or equal to threshold Th (YES in step S16), server device 100 determines that a snow jam has occurred (step S12). When it is determined that the absolute value Dy of the difference between Yp and Yn is less than threshold value Th (NO in step S16), server device 100 determines that a snow jam has not occurred (step S17).

  The series of determination processing shown in FIGS. 12 and 13 is an example, and the present invention is not limited to this. In order to continuously and repeatedly perform a series of determination processes for determining whether or not a snow jam has occurred (that is, to periodically perform the determination process), the server device 100 transmits one frame image a plurality of times as described below. May be used for the determination processing of.

  In the case where the server device 100 determines whether or not a snow jam has occurred using the continuous m frame images, the server device 100 performs the m-th frame from the j-th to the (j + m-1) -th frame in a certain cycle. When the presence or absence of the occurrence of snow jam is determined using an image (specifically, a binarized image), in the next cycle, the presence or absence of occurrence of the snow jam is determined using m frame images from the (j + 1) th to the (j + m) th frames. to decide. Note that m and j are natural numbers.

  That is, in the next cycle, the server device 100 deletes the oldest frame image among the plurality of frame images used in the cycle immediately before the next cycle, and replaces the new frame image (specifically, By adding a frame image next to the newest frame image included in the previous cycle, it is determined whether or not a snow jam has occurred. As described above, the server device 100 determines whether or not a snow jam has occurred each time one frame image is obtained. According to such a configuration, highly accurate detection is possible.

<F. Hardware Configuration>
FIG. 14 is a diagram illustrating a typical example of the hardware configuration of the server device 100.

  Referring to FIG. 14, server device 100 includes, as main components, processor 151 for executing a program, ROM 152 for storing data in a nonvolatile manner, data generated by execution of the program by processor 151, or an input device. RAM 153 for volatilely storing data input through the HDD, an HDD 154 for nonvolatilely storing data, a communication IF (Interface) 155, operation keys 156, a power supply circuit 157, and a display 158. Each component is mutually connected by a data bus. The communication IF 155 is an interface for performing communication with another device.

  The processing in the server device 100 is realized by each hardware and software executed by the processor 151. Such software may be stored in the HDD 154 in advance. The software may be stored in another storage medium and distributed as a program product. Alternatively, the software may be provided as a downloadable program product by an information provider connected to the so-called Internet. Such software is temporarily stored in the HDD 154 after being read from the storage medium by the reading device or downloaded via the communication IF 155 or the like. The software is read from the HDD 154 by the processor 151 and stored in the RAM 153 in the form of an executable program. Processor 151 executes the program.

  Each component constituting the server device 100 shown in FIG. 1 is a general one. Therefore, it can be said that an essential part of the present invention is the software stored in the RAM 153, the HDD 154, the storage medium, or the software downloadable via the network. The operation of each piece of hardware of server device 100 is well-known, and thus detailed description will not be repeated.

  Since the terminal device 200 has a configuration similar to that of the server device 100, the hardware configuration of the terminal device 200 will not be described repeatedly.

<G. Supplement>
As described above, in the present embodiment, the frame image is binarized, the barycenter position of the white portion is obtained, and the presence or absence of a snow jam is determined by focusing on the shift of the barycenter position. In other words, whether or not a snow jam has occurred is determined based on the movement mode of the center of gravity C of a region having a certain luminance or higher in the frame image extracted (specified) by binarization.

  For the movement of the position of the center of gravity of the moving object, it is conceivable to use the difference between the frame images, but this method cannot detect the occurrence of snow jam, as described later.

  FIG. 15 is a diagram in which the method using the difference between the frame images (comparative example) and the feature points of the processing with this example are arranged.

  Referring to FIG. 15, in the comparative example, in a quiet indoor environment (an environment in which a moving object occurs spontaneously), determination of the presence or absence of the moving object, identification of the moving object, determination of the moving object movement tendency, and determination of the moving object moving direction are performed. Judgment is possible. However, in this comparative example, in an outdoor natural environment (environment where a moving object continuously occurs), all of the determination of the presence or absence of the moving object, the identification of the moving object, the determination of the moving object moving tendency, and the determination of the moving object moving direction are all performed. Can not. This is because (i) the moving object itself becomes the background when the difference between the frame images is calculated, and the region of the moving object cannot be extracted. (Ii) External light, reflection of lighting, waves, ripples, and rain This is because noise caused by nature such as snow and the like becomes an obstacle when extracting a moving body region.

  On the other hand, in this example, in a quiet indoor environment and a natural outdoor environment, it is not possible to specify a moving object and determine the moving direction of the moving object. However, in a calm indoor environment and a natural outdoor environment, the movement of the center of gravity C of the white portion of the binarized image (in other words, the position of the center of gravity of a region having a certain brightness or higher in the frame image) is observed as described above. By doing so, it is possible to determine the presence or absence of a moving object and the tendency of moving object movement.

  As described above, the binarization process is performed, and it is possible to determine whether or not a snow jam has occurred in the waterway 500 based on the temporal change of the position of the center of gravity of the white portion.

<H. Modification>
(1) In the above description, a configuration in which it is determined whether or not a snow jam has occurred every predetermined period Ta has been described as an example, but the present invention is not limited to this. For example, the presence or absence of occurrence of a snow jam may be determined by the following method.

  The server device 100 detects occurrence of a snow jam based on the frame image during the predetermined period Tb. Here, Tb is N times (N is an integer of 2 or more) times Ta. That is, Tb = N × Ta.

  In each predetermined period Ta, the server device 100 determines whether the absolute value Dx of the difference between Xp and Xn is greater than the threshold Th (YES in step S11 of FIG. 13), or the absolute value Dy of the difference between Yp and Yn. Is larger than the threshold Th (YES in step S16 in FIG. 13), the value of the separately provided variable Tg is incremented. On the other hand, when both of the absolute values Dx and Dy are less than the threshold Th, the value of the variable Tg is maintained.

  The server device 100 determines that a snow jam has occurred when the value of the variable Tg is equal to or larger than a predetermined threshold value smaller than N in the predetermined period Tb. For example, when N is 10, the threshold is set to 7.

  That is, the server device 100 determines the occurrence of a snow jam based on the determination that the snow jam has occurred at a predetermined ratio or more in a plurality of consecutive predetermined periods Ta. According to such processing, occurrence of a snow jam can be detected with high accuracy.

  (2) In the above description, the location of the water intake is imaged by the monitoring camera 300, but the invention is not limited to this. However, a monitoring camera is often provided at the intake port for the purpose of checking the vicinity of the intake port with an image, and the cost can be reduced by using the monitoring camera. Further, at the water intake port 700, since suspended matter is easily retained by the fence, it is more suitable for detecting the presence or absence of suspended matter than at other places.

  (3) In the above description, one side of the binarized image is set as the X axis, and the other side is set as the Y axis. However, the present invention is not limited to this. An XY coordinate system may be set so that the X axis and the Y axis are not parallel to the sides of the binarized image.

  (4) The facility that receives the supply of water from the water channel 500 is not limited to a power generation facility (power station).

<I. Appendix>
[Configuration 1] A monitoring system 1 includes an imaging unit (surveillance camera 300) that captures an image of a water channel 500 that supplies water to equipment, and a binary that binarizes each of a plurality of continuous frame images obtained by the imaging. Converting means (binary processing section 121), and calculating means (centroid position calculating section 122) for calculating the center of gravity (C) of the white portion in each of the binarized images obtained by the binarization. Determining means (determining unit 123) for determining whether or not a floating substance is contained in the white portion based on a temporal change mode of the center of gravity position (C); Is included in the notification device (display and / or speaker of server device 100 or terminal device 200 (in detail, display of terminal device 200) Preliminary / or speaker)) and a.

[Configuration 2]
The determining unit (determining unit 123) determines a direction of movement of the position of the center of gravity between two consecutive binary images in an axial direction of a first axis parallel to one side of the binary image. Then, based on the number of times of movement in the positive direction and the number of times of movement in the negative direction among the directions of movement, it is determined whether or not the white portion includes the floating substance.

[Configuration 3]
The determination means (determination unit 123) calculates a difference between the number of movements in the positive direction (the value of the variable Xp) and the number of movements in the negative direction (the value of the variable Xn), and calculates the absolute value of the difference (the value of the variable Xn). On the condition that Dx) is equal to or greater than a predetermined threshold (Th), it is determined that the white portion contains the floating substance.

[Configuration 4]
The determination unit (determination unit 123) further determines the direction of movement of the position of the center of gravity between two consecutive binary images in an axial direction of a second axis orthogonal to the first axis. Among the directions of movement in the axial direction of the second axis, based on the number of movements in the positive direction and the number of movements in the negative direction, determine whether or not the floating material is included in the white portion. Judge further.

[Configuration 5]
The determination means (determination unit 123) calculates a difference between the number of movements in the positive direction (the value of the variable Yp) and the number of movements in the negative direction (the value of the variable Yn) in the axial direction of the second axis. It is determined that the floating portion is included in the white portion on the condition that the absolute value Dy of the difference is equal to or larger than a predetermined threshold (Th).

[Configuration 6]
The imaging means (surveillance camera 300) captures an image of the water channel at a location where an intake port is provided.

[Configuration 7]
The facility is a power generation facility, and the suspended matter is a snow jam, a jellyfish swarm, or a driftwood.

  The embodiment disclosed this time is an exemplification, and is not limited to the above contents. The scope of the present invention is defined by the terms of the claims, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  Reference Signs List 1 monitoring system, 100 server device, 101 communication IF unit, 102 control unit, 111 reception unit, 112 transmission unit, 121 binarization processing unit, 122 center of gravity position calculation unit, 123 determination unit, 131, 132, 133, 134 Move Number counting section, 135, 136 Difference calculation section, 137, 138 Comparison section, 151 processor, 152 ROM, 153 RAM, 156 operation keys, 157 power supply circuit, 158 display, 200 terminal device, 300 monitoring camera, 500 water channel, 700 water intake Mouth, B21, B22, B31, B32 binarized image, C center of gravity position, G1, G21, G22, G31, G32 frame image, J white part.

Claims (9)

Imaging means for imaging a water channel for supplying water to the equipment,
Binarization means for binarizing each of a plurality of continuous frame images obtained by the imaging,
In each of the binarized images obtained by the binarization, a calculating unit that calculates a center of gravity of a white portion,
Based on the temporal change mode of the position of the center of gravity, determining means for determining whether or not a floating substance is included in the white portion,
A monitoring unit that performs a predetermined notification based on a determination that the floating portion is included in the white portion. The determining means includes:
In the axial direction of a first axis parallel to one side of the binarized image, determine the direction of movement of the center of gravity position between two consecutive binarized images,
The monitoring according to claim 1, wherein, based on the number of times of movement in the positive direction and the number of times of movement in the negative direction, the white portion includes the floating matter. system. The determining means includes:
Calculate the difference between the number of movements in the positive direction and the number of movements in the negative direction,
The monitoring system according to claim 2, wherein it is determined that the floating portion is included in the white portion on a condition that an absolute value of the difference is equal to or larger than a predetermined threshold. The determining means includes:
Further determining the direction of movement of the position of the center of gravity between two consecutive binary images in an axial direction of a second axis orthogonal to the first axis;
Based on the number of movements in the positive direction and the number of movements in the negative direction, among the movement directions in the axial direction of the second axis, it is further determined whether or not the floating material is included in the white portion. The monitoring system according to claim 2, wherein the determination is performed. The determining means includes:
Calculating a difference between the number of movements in the positive direction and the number of movements in the negative direction in the axial direction of the second axis;
The monitoring system according to claim 4, wherein it is determined that the floating portion is included in the white portion on a condition that an absolute value of the difference is equal to or greater than a predetermined threshold.   The monitoring system according to claim 1, wherein the imaging unit captures an image of a location of the water channel where an intake port is provided. The equipment is a power generation equipment,
The monitoring system according to any one of claims 1 to 6, wherein the suspended matter is a snow jam, a jellyfish swarm, or a driftwood. Imaging a channel for supplying water to the facility;
Binarizing each of a plurality of continuous frame images obtained by the imaging,
In each of the binarized images obtained by the binarization, a step of calculating a center of gravity of a white portion;
Based on the temporal change mode of the center of gravity position, a step of determining whether or not a floating substance is included in the white portion,
Performing a predetermined notification based on the determination that the floating substance is contained in the white portion. Binarizing each of a plurality of continuous frame images obtained by imaging a water channel that supplies water to the facility,
In each of the binarized images obtained by the binarization, a step of calculating a center of gravity of a white portion;
Based on the temporal change mode of the center of gravity position, a step of determining whether or not a floating substance is included in the white portion,
Causing the processor of the information processing device to execute the step of outputting the determination result.