JP7306204B2 - Channel monitoring device and channel monitoring method - Google Patents

Description

The present invention relates to a waterway monitor and the like.

An example of a system for monitoring the state of water in waterways such as rivers is disclosed in Patent Documents 1 and 2.

In Patent Document 1, a plurality of time-series background images are created from a plurality of time-series images captured by a monocular camera, and based on the pixel change regions extracted from the plurality of background images, the flow rate of a waterway is calculated. A water flow monitoring system is disclosed for determining whether a is in a normal state. The changed pixel region is extracted by background difference processing for each pixel between the latest background update image and the previous background update image.

Patent Literature 2 discloses a water level measuring device that extracts a waterfront line in a captured image. The water level measuring device selects a pixel of interest from a designated area designated from an image captured by a surveillance camera, and then extracts a plurality of identification images in contact with the pixel of interest. The water level measuring device calculates an identification strength indicating the degree to which the area corresponding to each of the plurality of identification images is a water area based on the results of machine learning related to the identification of the water area and the non-water area. A waterfront line in the captured image is extracted based on the identification strength.

Japanese Patent Application Laid-Open No. 2012-60268 JP 2019-45191 A

The technique disclosed in Patent Document 1 uses a single type of image processing (image analysis algorithm) called background subtraction processing on a captured image. Therefore, with the technique of Patent Literature 1, depending on the imaging environment (eg, the intensity of sunlight or the state of its change), it may not be possible to extract a changed pixel region corresponding to the actual change in the amount of flowing water. Even with the technique of Patent Document 2, depending on the imaging environment, there is a possibility that the water region and the non-water region cannot be identified in the captured image so as to correspond to the actual river and land.

Therefore, in the techniques of Patent Documents 1 and 2, there is a possibility that the abnormality may not be detected or may be erroneously detected in determining whether or not an abnormality has occurred in the water state based on the measurement result of the flow rate or water level in the captured image. have a nature.

An object of one aspect of the present invention is to realize a waterway monitoring device or the like that can accurately determine whether or not an abnormality has occurred in the state of water.

In order to solve the above problems, a channel monitoring device according to one aspect of the present invention is a channel monitoring device for monitoring the state of water in a channel, wherein the channel or water overflowing from the channel is stored. an image processing unit that performs a plurality of types of image processing on images captured by a surveillance camera that captures an image of a water storage facility, and a result of each of the plurality of types of image processing performed by the image processing unit. and a determination unit that determines whether or not an abnormality has occurred in the state of the water based on the above.

In order to solve the above problems, a waterway monitoring method according to one aspect of the present invention is a waterway monitoring method for monitoring the state of water in a waterway, wherein the waterway or water overflowing from the waterway is stored. an image processing step of performing a plurality of types of image processing on images captured by a surveillance camera that captures an image of a water storage facility; and results of each of the plurality of types of image processing performed by the image processing step. and a determination step of determining whether or not an abnormality has occurred in the state of the water based on the above.

According to one aspect of the present invention, it is possible to accurately determine whether or not an abnormality has occurred in the state of water.

It is a figure which shows the structural example of a channel monitoring system. FIG. 5 is a diagram showing an example of a result of feature point extraction processing by an image processing unit and an example of a result of feature point matching processing by an image processing unit; It is a figure which shows an example of the image which concerns on a background subtraction process. It is a figure which shows an example of the image which concerns on edge detection processing. FIG. 4 is a diagram showing an example of a luminance histogram in a captured image near an inlet; It is a figure which shows an example of the captured image of a diversion channel. FIG. 4 is a diagram showing an example of captured images at each installation location of a monitoring camera; FIG. 5 is a diagram showing an example of a combination of AND setting image processing and OR setting image processing set in association with each installation location of a monitoring camera; It is a graph for demonstrating the process concerning the 1st determination and 2nd determination by a determination part. It is a figure which shows an example of the abnormality notification process in a control part. It is a figure which shows an example of the notification process of abnormality occurrence cancellation|release in a control part.

An embodiment of the present invention will be described in detail below.

<Waterway monitoring system>
FIG. 1 is a diagram showing a configuration example of a waterway monitoring system 1. As shown in FIG. The waterway monitoring system 1 of this embodiment is a system that monitors the state of water in a waterway 110 (eg, water flow or water level). The waterway monitoring system 1 includes a monitoring camera 10 , a waterway monitoring device 20 , a central monitoring system 30 and a mobile terminal 40 .

The monitoring camera 10 is an imaging device that captures images of the waterway 110 or the water storage facility 120 in order to monitor the state of water in the waterway 110 or the water storage facility 120 . Image data representing a captured image captured by the monitoring camera 10 is transmitted to the waterway monitoring device 20 . When the captured image is a moving image, the monitoring camera 10 extracts a still image every predetermined time (eg, 10 seconds) and transmits the still image as image data. The monitoring camera 10 may capture still images at predetermined time intervals.

A watercourse 110 is an area of water on land where flooding can occur due to increased water, eg, a river, an artificially constructed waterway (eg, an agricultural or industrial drainage channel). In this embodiment, the waterway 110 may be a concept that includes a dam, a lake, a pond, or the like as the area.

The water storage facility 120 (water tank) is a facility for storing water overflowing from the waterway 110 over the side wall of the waterway 110 . The water storage facility 120 communicates with the waterway 110 through a drain pipe 130 (tunnel). One end of the drain pipe 130 is provided on the side wall of the water channel 110 , and guides water overflowing from the water channel 110 over the side wall of the water channel 110 due to heavy rain or the like to the water storage facility 120 .

The monitoring cameras 10 are installed at one or more points on the waterway 110 and at one or more points on the water storage facility 120 . In the water storage facility 120, for example, an inflow port through which water flows from the water channel 110 to the water storage facility 120, which is communicated with the drain pipe 130, is installed so as to be able to be imaged. When a plurality of water storage facilities 120 are provided for one waterway 110 , at least one or more monitoring cameras 10 are installed in each water storage facility 120 . The monitoring camera 10 may be provided in either one of the waterway 110 and the water storage facility 120 . In other words, the monitoring cameras 10 may be installed at a plurality of locations in at least one of the waterway 110 and the water storage facility 120 .

The waterway monitoring device 20 is a device that monitors the state of water in the waterway 110 . Specifically, the waterway monitoring device 20 determines whether or not there is an abnormality in the state of the water in the waterway 110 based on the captured image. Details of the channel monitoring device 20 will be described later.

Abnormalities in the state of water in the waterway 110 refer to, for example, a state in which the water in the waterway 110 increases and causes an inflow or an overflow. In other words, the state of water in which an abnormality has occurred includes a plurality of types of states such as inflow and overflow. Inflow refers to a phenomenon in which water overflowing from the channel 110 over the sidewall of the channel 110 flows into the water storage facility 120 through the drain pipe 130 . Overflow refers to a phenomenon in which water in channel 110 overflows the channel 110 over the sidewalls of channel 110 .

In the present embodiment, when an inflow or an overflow occurs, it is assumed that an abnormality has occurred in the state of water. However, the abnormality in the state of the water to be determined by the waterway monitoring device 20 is not limited to the phenomenon accompanying an increase in water, and is the case where the water level of the waterway 110 falls below a predetermined height (a region in which a predetermined amount of water should exist). It may be a phenomenon associated with a decrease in water, such as when the amount of water is less than a predetermined amount at the time.

The central monitoring system 30 is a superordinate system of the monitoring camera 10 and the waterway monitoring device 20 that enables the administrator of the waterway 110 to monitor the entire waterway 110 . The central monitoring system 30 is communicably connected to the waterway monitoring device 20 . The central monitoring system 30 includes a display device, and for example, by receiving data indicating a captured image or a log related to imaging, the acquired captured image or log is displayed on the display device. This allows the administrator to grasp the current or past situation of the waterway 110 . The central monitoring system 30 may be communicably connected to the monitoring cameras 10 and directly acquire the above data from the monitoring cameras 10 .

The central monitoring system 30 is equipped with a warning light, and controls lighting (blinking) or extinguishing of the warning light based on the result of determination by the channel monitoring device 20 as to whether or not inflow or overflow has occurred. When an inflow or overflow occurs, the central monitoring system 30 receives an abnormality occurrence notification from the waterway monitoring device 20 and turns on a warning light. The warning light is extinguished in response to the notification of cancellation of the occurrence of abnormality from the monitoring device 20. - 特許庁As a result, the manager can grasp the state of the water in the waterway 110 without constantly monitoring it. The notification of the occurrence of an abnormality or the notification of cancellation of the occurrence of an abnormality may be given by a display device, or by sound output from a speaker.

When the central monitoring system 30 has an operation reception unit, the user can remotely operate the processing and the like by the waterway monitoring device 20 . For example, the central monitoring system 30 can change various parameters (eg, setting parameters) for image processing by the waterway monitoring device 20 based on an input operation from the administrator via the operation reception unit. The central monitoring system 30 can also change the combination of multiple types of image processing executed by the waterway monitoring device 20 based on the administrator's input operation. The channel monitoring device 20 may be provided with an operation reception unit, and the channel monitoring device 20 may directly receive the input operation.

The mobile terminal 40 is a terminal carried by an administrator, such as a smart phone or a tablet. The mobile terminal 40 is communicably connected to the central monitoring system 30 . The mobile terminal 40 receives an abnormality occurrence notification or an abnormality occurrence release notification from the waterway monitoring device 20 in the form of e-mail, for example. The mobile terminal 40 may receive the latest image data stored in the waterway monitoring device 20 together with the abnormality occurrence notification or the abnormality occurrence cancellation notification. The administrator can check the state of the water in the channel 110 by visually checking the "inflow (or overflow) occurrence" or "inflow (or overflow) stop" displayed on the mobile terminal 40 and the latest captured image. The status can be grasped without constant monitoring.

<Waterway monitoring device>
The waterway monitoring device 20 includes a storage section 21 , a control section 22 and a communication section 23 . The storage unit 21 stores or stores various data used by the control unit 22 . The control unit 22 controls the waterway monitoring device 20 in an integrated manner. The communication unit 23 communicates with the central monitoring system 30 and the mobile terminal 40 .

The storage unit 21 stores, for example, programs executed by the control unit 22 (eg, programs (algorithms) related to multiple types of image processing), setting parameters, and image data acquired from the monitoring camera 10 (extracted or captured at predetermined time intervals). captured image (still image)).

Setting parameters include, for example, a set area within a captured image to be subjected to image processing, a first threshold value for determining that an abnormality has occurred in the state of water, and a determination that the state of water has returned to a normal state. A second threshold for The setting area, first threshold, and second threshold are set for each of a plurality of types of image processing. Through experiments or the like, a region where it is easy to identify an abnormality in the state of water is set as the setting region, a value that enables identification of an abnormality in the water state is set as the first threshold value, and a value that allows identification of the abnormality in the water state is set as the second threshold value. A value that can identify that the state of has returned to a normal state is set. In this embodiment, different values are set for the first threshold and the second threshold for each of the plurality of types of image processing.

The control unit 22 mainly includes an image processing unit 221 and a determination unit 222 . The image processing unit 221 performs a plurality of types of image processing on the captured image. The determination unit 222 determines whether or not an abnormality has occurred in the state of water based on the results of each of the plurality of types of image processing executed by the image processing unit 221 . Details of the image processing unit 221 and the determination unit 222 will be described later.

Under the control of the control unit 22, the communication unit 23 transmits, for example, the central monitoring system 30 and the mobile terminal 40 with an abnormality occurrence notification or an abnormality occurrence cancellation notification and image data, or sends a log to the central monitoring system 30. or send the data shown. As described above, the communication unit 23 may transmit an abnormality occurrence notification or the like to the mobile terminal 40 in the form of e-mail. When the communication unit 23 receives an instruction to change the setting parameters and the like from the central monitoring system 30, the control unit 22 changes the setting parameters and the like stored in the storage unit 21 according to the change instruction.

<Example of multiple types of image processing and determination processing>
The image processing unit 221 performs image processing on the captured image every predetermined time (for example, every 10 seconds). The multiple types of image processing executed by the image processing unit 221 include, for example, (1) feature point extraction processing, (2) background subtraction processing, (3) edge detection processing, (4) histogram creation processing, and (5) ) luminance specific processing.

(1) Feature Point Extraction Processing (1-1) Feature Point Number Comparison Processing The image processing unit 221 extracts pixels having a feature amount equal to or larger than a predetermined amount as feature points in the captured image. The feature amount is a value based on the luminance difference between the pixel of interest and the adjacent pixels, the direction of luminance change (gradient direction) between the pixel of interest and the adjacent pixels, and the like. The predetermined value may be set by experiments or the like to such an extent that it is possible to extract splashes, waves, or the like that occur on the surface of water as feature points.

As the processing for extracting feature points, a known technique can be used, for example, the Accelerated-KAZE (A-KAZE) method). By using the A-KAZE method, it is possible to reduce the possibility of erroneous determination due to external factors such as shaking of the monitoring camera 10 due to an earthquake or the like. From this point of view, the processing for extracting feature points is not limited to the A-KAZE method, and a method less susceptible to external factors should be used.

Reference numeral 2001 in FIG. 2 denotes an example of a result of feature point extraction processing by the image processing unit 221 . A captured image 2001 is an image of the water storage facility 120 near the inlet communicating with the drain pipe 130 . A shaded portion in the captured image 2001 indicates water flowing into the water storage facility 120 from the drainage pipe 130 . Circles shown in the captured image 2001 are locations where feature points are extracted. Reference numerals 2002, 3001, 3002, 5001, and 7001 are also captured images near the inlet.

Based on the number of feature points in the captured image, the determination unit 222 determines that an abnormal change (eg, abnormal splashes, waves) has occurred on the surface of the water in the waterway 110 (including the drainpipe 130). For example, by specifying in advance the number of feature points when splashes or waves that can be identified as abnormal fluctuations occur through experiments, etc., a threshold value (first threshold value) for determining that abnormal fluctuations have occurred is set. Keep The determining unit 222 determines that an abnormal change has occurred when determining that the number of feature points (actually measured value) extracted from the captured image by the image processing unit 221 is equal to or greater than the threshold.

The region in which the determination unit 222 counts the number of feature points may be within the set region set as one of the set parameters. In this process, a portion of the water flow that causes abnormal fluctuations is specified as the setting area.

According to this process, the determination unit 222 can determine whether or not an abnormality has occurred in the state of water (whether or not an abnormal change has occurred) based on the extracted feature points. Further, the determination unit 222 can determine whether or not an abnormal change has occurred using one captured image. The brightness value of the captured image is affected by environmental changes (disturbance ) and is subject to change. In particular, when image processing is performed using a plurality of (eg, two) captured images, the result is likely to be affected by disturbance. Since the image processing unit 221 performs image processing using one captured image, it is less likely to be affected by disturbance compared to the case where a plurality of captured images are used. In other words, the determination unit 222 can determine whether the state of water is abnormal while the influence of disturbance is reduced.

(1-2) Processing Based on Feature Point Matching Feature points are extracted from the captured image by the above-described method (eg, A-KAZE method). The first captured image is, for example, the latest captured image stored in the storage unit 21, and the second captured image is the first captured image (eg, the latest captured image) stored in the storage unit 21, for example. It is an image captured an hour ago (eg, 5 minutes ago).

The image processing unit 221 searches for corresponding feature points between the first captured image and the second captured image (feature point matching processing). A known technique can be used for this feature point matching process. For example, the image processing unit 221 calculates a hamming distance between an arbitrary feature point of the first captured image and each of all feature points of the second captured image, and the feature points having the smallest hamming distance correspond to each other. identified as feature points. The image processing unit 221 performs this process on all feature points extracted from the first captured image.

Reference numeral 2002 in FIG. 2 is a diagram showing an example of the result of feature point matching processing by the image processing unit 221 . In reference numeral 2002, feature points that are connected by straight lines between the first captured image IM1 and the second captured image IM2 are corresponding feature points in the first captured image IM1 and the second captured image IM2.

If the first captured image includes a predetermined number (first threshold value) or more of feature points (actual measurement values) that do not correspond to the feature points of the second captured image, the determination unit 222 determines that the waterway 110 (drain pipe 130) is determined to have undergone an abnormal change on the surface of the water. The predetermined number is the number of pairs of feature points when no water splashes or waves that can be identified as abnormal fluctuations are generated by experiments, etc., and the number of pairs of feature points when the water splashes or the like are generated. By specifying the difference in advance, it is specified.

The area in which the determination unit 222 determines whether or not the number of feature points is equal to or greater than a predetermined number is set as a setting area, which is one of setting parameters. In this process, a portion of the water flow that causes abnormal fluctuations is specified as the setting area.

In this process as well, the determination unit 222 can determine whether or not an abnormality has occurred in the state of water based on the extracted feature points.

(2) Background Difference Processing The image processing unit 221 generates a difference image between the first captured image and the second captured image. A known technique can be used as a method for generating the difference image. The image processing unit 221, for example, calculates the difference in luminance value for all corresponding pixels in the first captured image (input image) and the second captured image (background image), and uses the difference as the luminance value. A difference image composed of pixels is generated. The image processing unit 221 may perform noise removal processing (eg, morphological transformation) on the difference image.

By the process of generating the difference image (background subtraction process), if there is a pixel whose luminance value is equal to or greater than a predetermined value in the difference image, the pixel can be extracted. As the predetermined value, a brightness value is specified in advance by experiments or the like when light reflection, water spray, or waves that can be specified as an abnormal change occurs. As a result, the image processing unit 221 can generate a difference image in which the luminance value of the pixel corresponding to the location where the abnormal variation occurs in the captured image is relatively higher than the luminance value of the other pixels.

In FIG. 3, reference numeral 3001 indicates an example of the first captured image IM1, and reference numeral 3002 indicates an example of the second captured image IM2. The first captured image IM1 shows an example of a state in which water is flowing from the drainage pipe 130 into the water storage facility 120, and the second captured image IM2 shows an example of a state in which water is not flowing. Reference numeral 3003 denotes an example of a difference image IMd between the first captured image IM1 indicated by reference numeral 3001 and the second captured image IM2 indicated by reference numeral 3002. FIG. In the difference image IMd indicated by reference numeral 3003, white areas are detected in which the brightness values differ by a predetermined value or more between the first captured image IM1 and the second captured image IM2.

If the difference image includes a pixel having a luminance value (actually measured value) equal to or greater than a predetermined value (first threshold), the determination unit 222 determines that the water surface in the water channel 110 (including the drain pipe 130) is abnormal. It is determined that a significant change (e.g. light reflection, water splash, wave) has occurred. The region in which the determination unit 222 determines whether or not the pixel is included is set as a setting region, which is one of the setting parameters. In this process, a part of the surface of the water in which abnormal fluctuations occur is specified as the setting area. The determination unit 222 may determine that an abnormal change has occurred when a predetermined number of pixels are included in the set region. The predetermined number is set to such an extent that it can be specified by experiments or the like that an abnormal variation has occurred.

In the differential image IMd indicated by reference numeral 3003 in FIG. 3, the set region SR is specified, and the set region SR includes a predetermined number of pixels having luminance values equal to or greater than a predetermined value. Therefore, the determination unit 222 determines that an abnormal change has occurred.

According to this process, it is possible to determine whether or not an abnormality (abnormal light reflection or the like) has occurred in the state of the water based on the difference image.

(3) Edge Detection Processing The image processing unit 221 detects the boundary line between the water area and the non-water area in the captured image. The water region is the region where the image of water in the channel 110 exists in the captured image (symbol Rw in FIG. 4), and the non-water region is the region where the image of water in the channel 110 does not exist (symbol Rnw in FIG. 4). ). In this embodiment, even in the drainage pipe 130, the area where the image of water exists is defined as the water area Rw, and the area where the image of water does not exist is defined as the non-water area Rnw. A known technique (eg, Canny method, Hough transform method) can be used as edge detection processing for detecting a boundary line.

The position of the image of the water channel 110 can be specified in advance in the captured image. Therefore, the image processing unit 221 can identify the water area and the non-water area in the captured image, and can identify the edge corresponding to the boundary line from the edges detected by the image processing unit 221 . Reference numerals 4001 and 4002 in FIG. 4 are examples of images showing detection results of the boundary line Br.

The determination unit 222 determines that the water level is abnormal when the boundary line (actual measurement value) is included in the predetermined area (first threshold value) in the captured image. The predetermined area is set as a setting area, which is one of setting parameters, and is an area in which it can be determined that an overflow has occurred. In the captured image 4001, the state in which the water runs up from the side wall 110w of the channel 110 to the upper wall 110wt of the side wall 110w (the state in which the water exceeds the side wall 110w) is the state in which the overflow occurs. Therefore, as shown in the captured image 4001, a set region SR as the predetermined region PR is provided near the upper wall 110wt.

According to this processing, it is possible to determine that an abnormality has occurred in the water state (for example, the water level has reached a level at which flooding should be alarmed) based on the detection of the boundary line. Further, the determination unit 222 can determine whether or not an abnormality has occurred in the state of water using one captured image. Therefore, the judging section 222 can judge whether the state of the water is abnormal while the influence of the disturbance is reduced.

(4) Histogram Creation Processing The image processing unit 221 identifies a first area included in the water area and a second area that can be either a water area or a non-water area. The image processing unit 221 calculates (generates) the luminance distribution of the specified first area and the luminance distribution of the second area. The first area and the second area are set as setting areas, which are one of the setting parameters. The first area is an area included in the water area regardless of whether or not a water abnormality occurs. The second area is an area that is included in the non-water area when no water anomaly has occurred, and is included in the water area when a water anomaly has occurred. The first area and the second area are specified in advance by, for example, experiments.

The luminance distribution may be based on pixels whose gradation values are expressed in RGB, or may be based on pixels whose gradation values are expressed in grayscale. The same applies to luminance specifying processing, which will be described later. A known technique can be used for the method of creating the luminance distribution.

FIG. 5 is a diagram showing an example of a luminance histogram in a captured image near the inlet. As shown in FIG. 5, the vertical axis of the histograms GR1 and GR2 indicates the number of pixels, and the horizontal axis indicates the luminance value. The luminance values of the pixels forming the image of the water region Rw (including the water region inside the drain pipe 130) are relatively low. Therefore, the image processing unit 221 generates a histogram Gr1 (luminance distribution) in which there are many pixels with relatively low luminance values in the partial region Rw1 of the water region Rw in FIG. The luminance values of the pixels forming the image of the non-water area Rnw (including the non-water area inside the drain pipe 130) are relatively high. Therefore, the image processing unit 221 generates a histogram Gr2 (luminance distribution) in which there are many pixels with relatively high luminance values in the partial region Rnw1 of the non-water region Rnw in FIG.

FIG. 6 is a diagram showing an example of a captured image of the water channel 110 (diversion channel) provided with the drain pipe 130. As shown in FIG. A captured image 6001 shows the channel 110 when the water condition is normal, and a captured image 6002 shows a state in which the water in the channel 110 is increased and inflow is occurring. Reference numeral 131 denotes a lattice window provided at one end of the drain pipe 130 communicating with the water channel 110 .

As shown in FIG. 6, a first region R1 and a second region R2 are set as the set region SR. That is, the first region R1 is within the water region Rw in the captured images 6001 and 6002. FIG. The second region R2 is in the non-water region Rnw in the captured image 6001, but is in the water region Rw in the captured image 6002. In this embodiment, the second region R2 is set to a part of the side wall 110w. Therefore, in the first region R1, a histogram like the histogram Gr1 in FIG. 5 is generated regardless of whether the water has increased. In the second region R2, when the water condition is normal, a histogram like the histogram Gr2 in FIG. 5 is generated. 5 histogram Gr1 is generated.

Therefore, when an abnormality occurs in the state of water (abnormal water level), the degree of similarity between the histogram of the first region R1 and the histogram of the second region R2 increases. Using this similarity change, the determination unit 222 determines whether or not there is an abnormality in the state of water. That is, when determining that the degree of similarity (measured value) is equal to or greater than a predetermined value (first threshold value), the determination unit 222 determines that an abnormality has occurred in the water level.

A known technique can be used as a method of comparing histograms. For example, the determination unit 222 determines a first predetermined range (range of luminance (horizontal axis)) including the peak in the histogram of the first region, a second predetermined range corresponding to the first predetermined range in the histogram of the second region, identify. The determination unit 222 calculates the degree of similarity between the first predetermined range and the second predetermined range. The determination unit 222 calculates, for example, the ratio of the second area included in the first area to the first area as the degree of similarity. The first area is the area within the range between the histogram within the first predetermined range and the horizontal axis, and the second area is the area within the range between the histogram within the second predetermined range and the horizontal axis. be. The determination unit 222 determines that the water level is abnormal when the ratio is equal to or greater than a predetermined value. As the predetermined value, the above ratio when an abnormality in the water level occurs is set by experiment or the like.

The determination unit 222 may specify, for example, the range including the peak in the histogram of the second region as the second predetermined range. In this case, the determination unit 222 calculates the area where the first predetermined range and the second predetermined range overlap as the degree of similarity. The determination unit 222 determines that the water level is abnormal when the area is equal to or greater than a predetermined value. As the predetermined value, an area when an abnormality in the water level occurs is set by experiment or the like.

According to this processing, it is possible to capture color changes in the captured image based on the histogram. Therefore, it can be determined that an abnormality has occurred in the water condition (for example, the water level has reached a level at which flooding should be alarmed). Further, the determination unit 222 can determine whether or not an abnormality has occurred in the state of water using one captured image. Therefore, the judging section 222 can judge whether the state of the water is abnormal while the influence of the disturbance is reduced.

(5) Luminance Identification Processing Instead of the histogram, it may be determined whether or not an abnormality has occurred in the state of water by comparing the luminance values of the first region and the luminance values of the second region.

In this case, the determination unit 222 determines that the water level is abnormal when the difference (measured value) between the luminance value of the first area and the luminance value of the second area is within a predetermined range (first threshold value). For example, the determination unit 222 calculates a first average value of luminance values of the plurality of pixels forming the first region and a second average value of the luminance values of the plurality of pixels forming the second region. The determination unit 222 determines whether an abnormality has occurred in the water level by determining whether the absolute value of the difference between the first average value and the second average value is within a predetermined range. The predetermined range is set by experiment or the like as the range of the above absolute values when an abnormality in the water level occurs.

(Modification of Histogram Creation Processing and Luminance Identification Processing)
In the captured image, an area having a luminance value corresponding to the water area (eg, a drain) may be set as the first area. Further, the determination unit 222 may compare the second area and the third area, which is an area having a relatively high luminance value, instead of comparing the first area and the second area. Examples of the third area include a non-water area in fine weather (eg, the dry side wall 110w) and a signboard with a white background (also referred to as a white signboard). Also in this case, the image processing unit 221 calculates the luminance value or luminance distribution of the second area and the luminance value or luminance distribution of the third area, similarly to the case of comparing the first area and the second area. The determination unit 222 determines whether (1) the difference between the luminance value of the second region and the luminance value of the third region is within a predetermined range, or (2) the difference between the luminance distribution of the second region and the luminance distribution of the third region. If the degree of similarity is equal to or greater than a predetermined value, it is determined that there is an abnormality in the water level.

<Image processing target>
As setting parameters, multiple types of image processing may be set in association with each of multiple types of abnormal water states, such as inflow and overflow. In this case, the image processing unit 221 executes each of the plurality of types of image processing set in association with at least one of the plurality of types of states on the captured image. The determination unit 222 determines whether or not an abnormality has occurred in the water state by determining whether or not the water state is at least one of a plurality of types of water states.

In this way, by executing multiple types of image processing that are set for each state of water in which an abnormality has occurred (e.g., overflow or inflow), multiple types of states of water in which an abnormality has occurred can be obtained. , can be determined with high accuracy.

For example, determination of inflow is associated with multiple types of image processing suitable for inflow. For example, determination of inflow is associated with feature point extraction processing for determination of abnormal water spray and background subtraction processing for determination of abnormal light reflection. In this case, the image processing unit 221 performs these two image processes, and the determination unit 222 performs abnormality determination based on the results of these two image processes.

For example, determination of overflow is associated with a plurality of types of image processing suitable for overflow. For example, determination of overflow is associated with feature point extraction processing, background subtraction processing, edge detection processing capable of detecting the water level, histogram creation processing, and brightness specification processing. In this case, the image processing unit 221 performs these five image processes, and the determination unit 222 performs abnormality determination based on the results of each of these five image processes.

The image processing unit 221 and the determination unit 222 may perform processing for determining either inflow or overflow, or may perform processing for determining both inflow and overflow. .

Further, as setting parameters, multiple types of image processing may be set in association with each of multiple locations where the monitoring cameras 10 are installed. In this case, the image processing unit 221 performs a plurality of types of image processing that are set in association with the locations where the captured images are captured, for each of the captured images captured by the monitoring cameras 10 installed at each of the plurality of locations. each of the The determination unit 222 determines whether or not an abnormality has occurred at the location where the captured image was captured.

In this way, by executing a plurality of types of image processing set for each installation location (monitoring location) of the monitoring camera 10, it is possible to accurately determine abnormalities that may occur at each monitoring location.

For example, the captured image near the inlet as shown in FIG. 2 or the like is associated with the same image processing as the inflow determination. For example, the captured image of the diversion channel as shown in FIG. 6 is associated with the same image processing as the determination of the overflow. This is because determination of inflow is performed in the vicinity of the inlet, and determination of overflow is performed in the diversion channel.

The combination of multiple types of image processing is set for each water state or for each installation location of the monitoring camera 10 so that the water state (determination target) in which an abnormality has occurred can be determined with high accuracy through experiments or the like. All you have to do is In other words, as long as the abnormality of the water state can be determined with high accuracy, the plurality of types of image processing may be combined in any way, and the number thereof is not limited.

<AND setting and OR setting>
If the determination unit 222 determines that an abnormality has occurred in the state of water based on the results of at least two types of image processing among the plurality of types of image processing, the abnormality has not occurred based on the results of other image processing. Even if it is determined that the abnormality has occurred, it is determined that the abnormality has occurred. The determination unit 222 can determine whether the state of water is abnormal according to the priority set for multiple types of image processing.

Specifically, in the setting parameters, the plural types of image processing are set to either the first image processing or the second image processing described below.
- First image processing: at least one or more types of image processing essential for determining whether or not an abnormality has occurred in the state of water. Image processing with relatively high priority. Henceforth, it is also called AND setting image processing.
• Second image processing: At least one type of image processing that is not essential for the above determination. Image processing with relatively low priority. Henceforth, it is also called OR setting image processing.

The determination unit 222 determines that an abnormality has occurred in the state of water based on all the results of the AND setting image processing executed by the image processing unit 221 (determination process 1). The determination unit 222 determines that the actual measurement value calculated as the result of each AND-setting image processing is equal to or greater than or within the range of the first threshold set corresponding to each AND-setting image processing. In this case, it is determined that an abnormality has occurred in the water condition. The determination unit 222 determines that an abnormality has occurred in the state of water based on the result of at least one type of OR setting image processing among the OR setting image processing executed by the image processing unit 221 (determination processing 2). The determination unit 222 determines that at least one of the measured values calculated as the results of each of the OR-setting image processing is equal to or greater than or within the range of the first threshold set corresponding to each of the OR-setting image processing. , it is determined that an abnormality has occurred in the state of the water. If it is determined in both determination processing 1 and determination processing 2 that the water condition is abnormal, the determination unit 222 outputs a final determination result indicating that the water condition is abnormal.

In this case, the determination unit 222 may perform the abnormality determination based on a plurality of types of image processing necessary for determining abnormality of the water state, and it is necessary to reflect all of the plurality of types of image processing in the final determination result. There is no Therefore, the processing efficiency of the determination unit 222 can be improved.

<Specific example>
In the present embodiment, the abnormal state of water to be determined (eg, inflow, overflow), the installation location of the monitoring camera 10 (eg, the inlet, the diversion channel), AND setting image processing and OR setting image processing is set as a setting parameter. FIG. 7 is a diagram showing an example of a captured image at each installation location, and FIG. 8 is a diagram showing an example of a combination of AND setting image processing and OR setting image processing set in association with each installation location. . "AND" in FIG. 8 indicates AND setting image processing, and "OR" indicates OR setting image processing. Also, the numbers inside the circles indicate the processing order in which the image processing is performed. When there are a plurality of AND setting image processes, the order of the processes does not matter, and they may be processed in parallel. The same applies to OR setting image processing.

Reference numeral 7001 in FIG. 7 is an example of a captured image of the inlet. The captured image 7001 includes a setting area SR11 for determining a change in color inside the drain pipe 130, a setting area SR12 for determining whether abnormal water splashes occur, and light reflection occurring on the surface of the water. A setting area SR13 is set for determining the presence/absence. As the setting region SR11, a first region R1 (setting region SR11a) included in the water region Rw and a second region R2 (setting region SR11b) that can be either the water region Rw or the non-water region Rnw are set. ing.

Reference numeral 7002 in FIG. 7 is an example of a captured image of a diversion channel. In the captured image 7002, a setting area SR21 for judging the water level and a setting area SR22 for judging whether or not an abnormal water level occurs based on the color change of the side wall 110w are set. Also, in the captured image 7002, a setting region SR23 is set for determining whether or not abnormal water splashes occur near the side wall 110w. The set region SR23 is set near the corners of the two side walls 110w in the captured image 7002. As shown in FIG. In the set region SR22, the lower region of the side wall 110w included in the water region Rw is the first region R1 (set region SR22a), and the upper region of the side wall 110w that can be both the water region Rw and the non-water region Rnw is the second region. Each is set as an area R2 (setting area SR22b).

Reference numeral 7003 in FIG. 7 is an example of a captured image showing a partial area (riverside) of the waterway 110 . In the captured image 7003, a setting area SR31 for judging the water level and a setting area SR32 for judging whether or not an abnormal water level occurs based on the color change of the side wall 110w are set. In the set region SR32, the region above the side wall 110w is set as a second region R2 (set region SR32b), and the region including the white signboard image with a relatively high luminance value is set as a third region R3 (set region SR32c). ing. In addition, the captured image 7003 includes a setting area SR33 for determining whether an abnormal water level has occurred based on the color change of the upper wall 110wt of the side wall 110w, and a setting area SR33 for determining whether an abnormal water splash has occurred. SR34 is set. In the setting region SR33, the region including the image of the drainage ditch having the luminance value corresponding to the water region Rw is defined as the first region R1 (setting region SR33a), and a portion of the upper wall 110wt is defined as the second region R2 (setting region SR33b). are set, respectively.

In the present embodiment, it is determined whether or not inflow occurs at the inflow port and the diversion channel. At the riverside, the presence or absence of overflow is targeted for determination.

In the example of determining the presence or absence of inflow at the inlet in FIG. 8, the image processing unit 221 performs histogram creation processing in the setting region SR11, feature point extraction processing in the setting region SR12, and background subtraction processing in the setting region SR13. to run. The determination unit 222 outputs determination results for each of the setting regions SR11 to SR13. In the process of determining whether or not an inflow occurs at the inlet, the determination process of determining whether or not abnormal water splashing occurs is an essential determination process. Therefore, the feature point extraction process is set to the AND setting image process, and the histogram creation process and the background subtraction process are set to the OR setting image process.

Therefore, in this determination processing, first, the image processing unit 221 executes the feature point extraction processing, and only when the determination unit 222 determines that an abnormal water splash is occurring, the image processing unit 221 and the determination unit 222 executes processing related to histogram generation processing and background subtraction processing. If the determining unit 222 determines that a color change has occurred in the drain pipe 130 based on the histogram creation process, or if it determines that abnormal light reflection has occurred in the background subtraction process, the water state Output the final determination result that an abnormality has occurred.

When the determination unit 222 determines that abnormal water splashes have not occurred, the image processing unit 221 and the determination unit 222 do not execute the processing related to the histogram creation processing and the background subtraction processing, and the state of the water is determined to be abnormal. output the final judgment result that no Even if the determination unit 222 determines that an abnormal splash of water has occurred, if it determines that neither the change in color nor the abnormal light reflection has occurred, it determines that there is no abnormality in the state of the water. output the final judgment result of When the determination unit 222 outputs the final determination result that there is no abnormality in the state of water, the control unit 22 stores the next image data after a predetermined time (for example, 10 seconds) has passed. 21, and using the captured image related to the image data, abnormality determination of the state of water is performed. The captured image includes the latest captured image at the time of acquisition and, if necessary, a captured image a predetermined time ago (eg, 5 minutes ago).

In the example of determining whether or not an inflow has occurred in the diversion channel of FIG. Perform diff processing. The determination unit 222 outputs determination results for each of the setting regions SR21 to SR23. In the process of determining whether an inflow has occurred in the diversion channel, the process of determining whether an abnormal splash has occurred and the process of determining whether there is an abnormality in the water level are essential determination processes. Therefore, edge detection processing and background subtraction processing are set to AND setting image processing, and histogram creation processing and brightness specifying processing are set to OR setting image processing.

Therefore, in this determination process, first, the image processing unit 221 executes edge detection processing and background subtraction processing. Only when the determination unit 222 determines that an abnormal water splash has occurred and an abnormality has occurred in the water level based on the results of these image processing, the image processing unit 221 and the determination unit 222 perform histogram creation processing and Execute processing related to luminance specifying processing. If the determination unit 222 determines that the side wall 110w has undergone a color change based on the histogram creation process or the luminance identification process, it outputs a final determination result that an abnormality has occurred in the state of water.

When the determination unit 222 determines that an abnormal water splash has not occurred or that an abnormality has not occurred in the water level, the image processing unit 221 and the determination unit 222 perform processing related to histogram generation processing and luminance identification processing. don't run In this case, the determination unit 222 outputs a final determination result that there is no abnormality in the state of water. Even if the determination unit 222 determines that an abnormal water splash has occurred and that the water level is abnormal, it has determined that the color change has not occurred in both the histogram creation process and the brightness identification process. In this case, the final determination result that there is no abnormality in the state of water is output. When the determination unit 222 outputs the final determination result that there is no abnormality in the state of water, the control unit 22 uses the next captured image to determine the abnormality in the water state in the same manner as described above. conduct.

In the example of determining whether or not an overflow has occurred at the waterfront in FIG. 8, the same processing as the example of determining whether or not an inflow has occurred in the diversion channel is performed. Therefore, explanation of specific processing is omitted.

<Summary>
In this way, with respect to the image captured by each monitoring camera 10, the area where the water condition is likely to be abnormal is specified as a set area for each abnormal water condition (eg, inflow, overflow) to be determined. . In addition, a plurality of types of image processing suitable for determining whether or not an abnormality has occurred is associated with each abnormal state of water. Furthermore, at least one type of image processing essential for determining whether or not an abnormality has occurred is set as AND setting image processing for each abnormal state of the water, and at least one type that is not essential for the determination is set as OR setting image processing.

The image processing unit 221 performs a plurality of types of image processing set as AND setting image processing and/or OR setting image processing for each determination target in the specified setting region for the captured image. The determination unit 222 outputs a final determination result that an abnormality has occurred in the state of water in the following cases.
・It is determined that an abnormality has occurred in the state of water based on the results of all AND setting image processing, and the state of water is determined to be abnormal in at least one determination processing based on the results of a plurality of types of image processing. If it is determined that it has occurred.

As a result, the determination unit 222 can accurately determine whether or not an abnormality has occurred for each determination target, thereby reducing the possibility of erroneous determination (non-detection or erroneous detection) of the abnormality. Further, the determination unit 222 can accurately determine various abnormal states of water.

Moreover, each of the multiple types of image processing is not suitable for identifying all changes in various indices such as splashes, waves, water levels, and colors. For each index, there is image processing suitable for identifying changes in the index (eg, feature extraction processing in the case of water splashes). Since a plurality of types of image processing are specified for each determination target in consideration of the above indices, the determination unit 222 can accurately determine the abnormal state of water for each determination target.

In addition, image processing (eg, histogram creation processing) that can be performed on one captured image is resistant to changes in light (eg, sunlight) or changes in time, as described above, and is calculated by the image processing. The actual measured value is less susceptible to disturbances. In this embodiment, a combination of a plurality of types of image processing for each determination target is specified to include image processing that can be executed on one captured image while considering the above indices. Therefore, even in a monitoring location (eg, outdoors) that is susceptible to disturbance, the determination unit 222 can accurately determine whether the water state is abnormal.

Furthermore, since each of the plurality of types of image processing is associated with the AND setting image processing or the OR setting image processing, the determination unit 222 can efficiently determine whether the state of water is abnormal.

<Determination of no abnormality>
The determination unit 222 can determine that there is no abnormality in the state of water by the same method as described above. When it is determined that there is no abnormality in the state of water, the control unit 22 controls the communication unit 23 to issue an abnormality cancellation notification.

The determination unit 222 determines that there is no abnormality in the state of water based on the results of at least two types of image processing, and determines that there is an abnormality in the state of water based on the results of other image processing. Even if there is, it is determined that there is no abnormality in the state of the water.

Specifically, based on all the results of the AND setting image processing executed by the image processing unit 221, the determination unit 222 determines that there is no abnormality in the state of water (determination process 3). The determination unit 222 determines that the actual measurement value calculated as the result of each AND setting image processing is equal to or less than or within the range of the second threshold set corresponding to each AND setting image processing. In this case, it is judged that there is no abnormality in the state of the water. The determination unit 222 determines that there is no abnormality in the state of water based on the result of at least one type of OR setting image processing among the OR setting image processing executed by the image processing unit 221 (determination processing 4). . The determination unit 222 determines that at least one of the actual measurement values calculated as the results of each of the OR-setting image processing is equal to or less than or within the range of the second threshold set corresponding to each of the OR-setting image processing. , it is determined that there is no abnormality in the state of the water. When determining that there is no abnormality in the water state in both determination processing 3 and determination processing 4, the determination unit 222 outputs a result indicating that there is no abnormality in the water state as the final determination result. do.

When judging that there is no abnormality in the state of water, the judging section 222 does not judge whether or not there is an abnormality in the state of water. That is, a first determination as to whether or not an abnormality has occurred in the state of the water on the premise of notifying the occurrence of an abnormality, and whether or not an abnormality has occurred in the state of the water on the premise of notifying the cancellation of the abnormality occurrence. The second determination is not performed at the same time. For example, when the determination unit 222 determines that an abnormality has occurred in the state of the water as a result of performing the processing related to the first determination, the control unit 22 causes the communication unit 23 to notify the occurrence of abnormality, and then performs the second determination. Perform processing related to When the determination unit 222 determines that there is no abnormality in the state of the water as a result of performing the processing related to the second determination, the control unit 22 causes the communication unit 23 to perform an abnormality occurrence cancellation notification, and then performs the second determination. 1 Perform processing related to determination. When the waterway monitoring device 20 is powered on, the control unit 22 first performs the processing related to the second determination, determines that there is no abnormality in the water condition, and then performs the processing related to the first determination.

9A and 9B are graphs for explaining the processing related to the first determination and the second determination by the determination unit 222. FIG. The vertical axis in FIG. 9 indicates actual measurement values (values are arbitrary) in arbitrary image processing, and the horizontal axis indicates elapsed time from the start of monitoring. In this example, the first threshold Th11 is set to 100, and the second threshold Th12 is set to 40.

The control unit 22 first performs processing related to the second determination. As shown in FIG. 9, for example, at time T1, the actual measurement value is less than the second threshold Th12, so the determination unit 222 determines that there is no abnormality in the state of water. Therefore, after that, the control unit 22 performs the process related to the first determination.

At time T2, the measured value reaches the first threshold Th11. Therefore, the determination unit 222 determines that an abnormality has occurred in the state of water, and the communication unit 23 transmits an abnormality occurrence notification to the central monitoring system 30 and the mobile terminal 40 . At time T3, the actual measurement value becomes less than the first threshold value, so the control unit 22 then performs processing related to the second determination. After switching from the process related to the first determination to the process related to the second determination, the control unit 22 performs the first No processing related to judgment is performed. That is, at time T4, the actual measurement value reaches the first threshold value Th11 again, but the process related to the first determination is not performed.

At time T5, the measured value becomes less than the first threshold Th11 again, and at time T6, the measured value decreases to the second threshold Th12. Therefore, the determination unit 222 determines that there is no abnormality in the state of water, and the communication unit 23 transmits an abnormality occurrence cancellation notification to the central monitoring system 30 and the mobile terminal 40 .

After that, for example, when it is determined that the second threshold Th12 is exceeded, the control unit 22 switches the processing related to the first determination. In addition, when the determination unit 222 continuously determines that the actual measurement value is equal to or greater than the first threshold Th11 or equal to or less than the second threshold Th12 for a predetermined number of times (a predetermined time), the determination result related to arbitrary image processing is as follows. It may be determined that an abnormality has occurred or has not occurred in the state of water. In this case, it is possible to reduce the possibility of erroneous determination based on abrupt changes in measured values.

<Number of judgments>
The determination unit 222 may determine whether or not there is an abnormality in the state of water a plurality of times. In this case, when the determination unit 222 outputs the same determination result continuously for a predetermined number of times, the determination result is determined as the final determination result. As a result, it is possible to further improve the accuracy of water condition abnormality determination. The predetermined number of times may be set by experiments or the like to such an extent that the accuracy required by the user can be secured.

The determination unit 222 determines that an inflow or an overflow has occurred based on the results of a plurality of types of image processing set for determining whether an inflow or an overflow has occurred. In this case, the image processing unit 221 and the determination unit 222 re-execute the processing related to the plurality of types of image processing (perform the second processing). The image processing unit 221 and the determination unit 222 execute the processing related to the plurality of types of image processing until it reaches a predetermined number of times as long as it is determined that the inflow or overflow has occurred. If it is determined that an inflow or an overflow has occurred in the result of the process executed last, the determination unit 222 finally determines that an inflow or an overflow has occurred.

<Process flow: Abnormal occurrence notification process>
FIG. 10 is a diagram showing an example of an abnormality occurrence notification process (waterway monitoring method) in the control unit 22. As shown in FIG. The image processing unit 221 acquires a captured image from the storage unit 21 (S1). The image processing unit 221 identifies the location where the acquired captured image was captured (installation location of the surveillance camera 10) and multiple types of image processing associated with the type of water state in which an abnormality has occurred ( S2). The image processing unit 221 also reads out setting parameters associated with multiple types of image processing.

The image processing unit 221 executes the specified plural types of image processing on the captured image (S3: image processing step). The determination unit 222 determines whether or not an abnormality has occurred in the state of water based on the results of multiple types of image processing (determination step).

Specifically, the determination unit 222 determines whether or not an abnormality has occurred in the state of water based on all of the results of at least one type of AND setting image processing among the specified plurality of types of image processing ( S4). Based on all the results of the AND setting image processing, the determination unit 222 performs the following processing when it determines that an abnormality has occurred in the state of water (YES in S4). The determination unit 222 determines whether or not an abnormality has occurred in the state of water based on the result of at least one type of OR setting image processing among the specified plurality of types of image processing (S5). If it is determined that an abnormality has occurred in the state of water for at least one type of OR setting image processing (YES in S5), the determination unit 222 determines that an abnormality has occurred in the state of water.

In the case of YES in S5, the determination unit 222 repeats the processes of S4 and S5 a predetermined number of times, and determines whether or not it has been determined that an abnormality has occurred in the state of water for a predetermined number of times in succession (S6). When the determining unit 222 outputs the determination result that the water condition is abnormal for a predetermined number of times in succession (YES in S6), the determination unit 222 outputs the determination result that the water condition is abnormal as the final determination result. Output. In this case, the control unit 22 outputs an abnormality occurrence notification to the central monitoring system 30 and the mobile terminal 40 by controlling the communication unit 23 (S7). This allows the user to recognize that an abnormality has occurred in the state of water.

If NO in the processes of S4 to S6, the process returns to S1, and the image processing unit 221 acquires another captured image (eg, the latest captured image at the time of acquisition).

<Process flow: Abnormal occurrence notification process>
FIG. 11 is a diagram showing an example of an abnormality cancellation notification process in the control unit 22. As shown in FIG. S11 to S13 are the same as the processing of S1 to S3 in FIG. However, as for the threshold, the second threshold is acquired. The determination unit 222 uses the second threshold value to determine whether or not the abnormal state of water has converged. Specifically, after S<b>14 , the determination unit 222 determines whether or not there is an abnormality in the state of water based on the results of the multiple types of image processing specified by the image processing unit 221 .

The determination unit 222 determines whether or not there is an abnormality in the state of water based on all the results of at least one type of AND setting image processing among the specified plural types of image processing (S14). Based on all the results of the AND setting image processing, the determination unit 222 performs the following processing when it determines that there is no abnormality in the state of water (YES in S14). The determination unit 222 determines whether or not there is an abnormality in the state of water based on the result of at least one type of OR setting image processing among the specified plurality of types of image processing (S15). If the determining unit 222 determines that there is no abnormality in the state of water based on the results of at least one type of OR setting image processing (YES in S15), it determines that there is no abnormality in the state of water.

In the case of YES in S15, the determination unit 222 repeats the processes of S14 and S15 a predetermined number of times, and determines whether or not it has been determined that there is no abnormality in the state of water for a predetermined number of times (S16). If the judgment unit 222 outputs the judgment result that the water condition is not abnormal for a predetermined number of times (YES in S16), the final judgment result is that the water condition is not abnormal. output the judgment result. In this case, the control unit 22 outputs the notification of cancellation of the occurrence of abnormality to the central monitoring system 30 and the mobile terminal 40 by controlling the communication unit 23 (S17). This allows the user to recognize that the abnormal state of water has been resolved.

If NO in the processes of S14 to S16, the process returns to S11, and the image processing unit 221 acquires another captured image (eg, the latest captured image at the time of acquisition).

[Example of realization by software]
The control block of the waterway monitoring device 20 (especially the image processing unit 221 and the determination unit 222 of the control unit 22) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software. may be realized by

In the latter case, the waterway monitoring device 20 comprises a computer that executes program instructions, which are software that implements each function. This computer includes, for example, one or more processors, and a computer-readable recording medium storing the program. In the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. In addition, a RAM (Random Access Memory) for developing the above program may be further provided. Also, the program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be implemented in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

[Additional notes]
The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention.

10 surveillance camera 20 waterway monitoring device 110 waterway 120 water storage facility 221 image processing unit 222 determination unit 2001, IM1, IM2, 3001, 3002, 4001, 4002, 5001, 6001, 6002, 7001 to 7003 Captured image IM1 First captured image IM2 Second captured image IMd Difference image Rw Water region Rnw Non-water region Br Boundary line SR Setting region (predetermined region)
GR1, GR2 Histogram (luminance distribution)

Claims (10)

A channel monitoring device for monitoring the state of water in a channel,
an image processing unit that performs a plurality of types of image processing on an image captured by a surveillance camera that captures an image of the waterway or a water storage facility that stores water overflowing from the waterway;
a determination unit that determines whether an abnormality has occurred in the state of the water based on the results of each of the plurality of types of image processing executed by the image processing unit ;
The state of the water in which the abnormality occurs includes a plurality of types of states,
The plurality of types of image processing are set in association with each of the plurality of types of states,
The image processing unit executes each of the plurality of types of image processing set in association with at least one of the plurality of types of states on the captured image,
The determination unit determines whether the abnormality has occurred by determining whether the state of the water is at least one of the plurality of types of states,
Each of the plurality of types of image processing includes (1) feature point extraction processing, (2) background subtraction processing, (3) edge detection processing, (4) histogram creation processing, and (5) luminance identification processing. surveillance equipment. For the multiple types of image processing,
At least one type of first image processing essential for determining whether or not the abnormality has occurred;
At least one type of second image processing that is not essential for the determination, and
The determination unit is
(1) determining that the abnormality has occurred based on all the results of the first image processing executed by the image processing unit; and (2) performing the second image processing executed by the image processing unit. 2. The waterway monitoring apparatus according to claim 1, wherein it is determined that said abnormality has occurred when it is determined that said abnormality has occurred based on the result of said second image processing of at least one of them. The surveillance cameras are installed at a plurality of locations in at least one of the waterway and the water storage facility,
The plurality of types of image processing are set in association with each of the plurality of locations,
The image processing unit performs the plurality of types of image processing set for each of the captured images captured by the surveillance cameras installed at each of the plurality of locations in association with the location where the captured image was captured. and run each of the
The waterway monitoring device according to claim 1 or 2 , wherein the determination unit determines whether or not the abnormality has occurred at a location where the captured image was captured. A channel monitoring device for monitoring the state of water in a channel,
an image processing unit that performs a plurality of types of image processing on an image captured by a surveillance camera that captures an image of the waterway or a water storage facility that stores water overflowing from the waterway;
a determination unit that determines whether an abnormality has occurred in the state of the water based on the results of each of the plurality of types of image processing executed by the image processing unit;
The image processing unit, as one of the plurality of types of image processing, extracts, as a feature point, a pixel having a feature amount equal to or greater than a predetermined amount in the captured image captured by the surveillance camera,
The waterway monitoring device, wherein the determination unit determines that an abnormal change has occurred on the surface of the water based on the number of the feature points in the captured image. A channel monitoring device for monitoring the state of water in a channel,
an image processing unit that performs a plurality of types of image processing on an image captured by a surveillance camera that captures an image of the waterway or a water storage facility that stores water overflowing from the waterway;
a determination unit that determines whether an abnormality has occurred in the state of the water based on the results of each of the plurality of types of image processing executed by the image processing unit;
The image processing unit performs, as one of the plurality of types of image processing, a first captured image captured by the monitoring camera at an arbitrary time and a first captured image captured by the monitoring camera before capturing the first captured image. 2 extracting a pixel having a feature amount equal to or larger than a predetermined amount as a feature point in the captured image,
The determination unit determines that an abnormal change has occurred on the surface of the water when the first captured image includes a predetermined number or more of feature points that do not correspond to the feature points of the second captured image . Road monitoring device. The image processing unit performs, as one of the plurality of types of image processing, a first captured image captured by the monitoring camera at an arbitrary time and a first captured image captured by the monitoring camera before capturing the first captured image. 2 Generate a difference image from the captured image,
6. The method according to any one of claims 1 to 5 , wherein when the difference image includes a pixel having a luminance value equal to or greater than a predetermined value, the determination unit determines that an abnormal change has occurred on the surface of the water. A waterway monitor as described. In the captured image captured by the surveillance camera, a region in which the image of water exists is defined as a water region, and a region in which the image of water does not exist is defined as a non-water region,
The image processing unit detects a boundary line between the water region and the non-water region as one of the plurality of types of image processing,
The waterway monitoring apparatus according to any one of claims 1 to 6 , wherein the determination unit determines that an abnormality has occurred in the water level when the boundary line is included in a predetermined area in the captured image. In the captured image captured by the surveillance camera, a region in which the image of water exists is defined as a water region, and a region in which the image of water does not exist is defined as a non-water region,
The image processing unit, as one of the plurality of types of image processing,
identifying a first area included in the water area and a second area that can be either the water area or the non-water area;
calculating the luminance value or luminance distribution of the first region and the luminance value or luminance distribution of the second region;
(1) when the difference between the luminance value of the first region and the luminance value of the second region is within a predetermined range; or (2) the luminance distribution of the first region and the second region. 8. The waterway monitoring apparatus according to any one of claims 1 to 7 , wherein it is determined that an abnormality has occurred in the water level when the similarity with the luminance distribution of is equal to or greater than a predetermined value. The determination unit is
Determining whether or not the abnormality has occurred is performed multiple times,
The waterway monitoring apparatus according to any one of claims 1 to 8 , wherein when the same determination result is output continuously for a predetermined number of times, the determination result is determined as the final determination result. A channel monitoring method for monitoring the state of water in a channel, comprising:
an image processing step of performing a plurality of types of image processing on an image captured by a surveillance camera that captures images of the waterway or a water storage facility that stores water overflowing from the waterway;
a determination step of determining whether or not an abnormality has occurred in the state of the water based on the results of each of the plurality of types of image processing executed in the image processing step ;
The state of the water in which the abnormality occurs includes a plurality of types of states,
The plurality of types of image processing are set in association with each of the plurality of types of states,
In the image processing step, each of the plurality of types of image processing set in association with at least one of the plurality of types of states is performed on the captured image;
In the determination step, it is determined whether or not the abnormality has occurred by determining whether or not the state of the water is at least one of the plurality of types of states,
Each of the plurality of types of image processing includes (1) feature point extraction processing, (2) background subtraction processing, (3) edge detection processing, (4) histogram creation processing, and (5) luminance identification processing. Monitoring method.

Images