JP2022102609A - River monitoring device and river monitoring system - Google Patents

Abstract

To promote appropriate handling of a river image with an abnormality.SOLUTION: A river monitoring device comprises: a river monitoring camera that monitors a river; a region detection unit that detects, in a river image taken by the river monitoring camera, a first region where an imaging state changes in accordance with change in the water surface of the river and a second region which is different from the first region and where the imaging state does not change even if the water surface of the river changes; a subject detection unit that detects subjects rendered in the first region and the second region; an abnormality determination unit that determines the presence/absence of an abnormality in the river image on the basis of the state of a subject rendered in at least one of the first region and the second region; and an abnormality notification unit that gives notification of an abnormality when the abnormality determination unit determines that there is an abnormality in the river image.SELECTED DRAWING: Figure 1

Description

The present invention relates to a river monitoring device and a river monitoring system that monitor a river by taking an image with a camera.

The river monitoring device disclosed in Patent Document 1 is known. In this river monitoring device, the image processing unit cuts out the area to be image processed excluding the areas other than the river from the video signal captured by the image pickup unit (camera) that monitors the river, and the feature amount for the brightness value of the area. To extract. Then, when the data processing unit detects a floating substance in the river based on the feature amount, an alarm is issued by an alarm device.

Japanese Unexamined Patent Publication No. 7-85385

Since it is difficult for a person to constantly visually check a river image captured by a camera, the river image is temporarily stored (stored) in a storage unit. Then, it is common for the manager to periodically read the river image from the storage unit and visually display the river image displayed on the display unit.
For example, if foreign matter such as dirt or dust adheres to the lens of the camera (imaging unit), or if the image quality deteriorates due to a malfunction of the camera, an abnormality that gives a sense of discomfort to the person who sees the river image captured by the camera. Occurs in the river image. In this case, in the technique of Patent Document 1, the river image with the abnormality is continuously captured by the camera unless the administrator visually recognizes the river image with the abnormality and takes appropriate measures. A river image with such anomalies may not function as a river monitoring image, and it is difficult for the administrator to grasp the situation of the river or the surroundings of the river even if the administrator visually observes the river image with the anomaly. It becomes difficult to grasp.

Therefore, in view of the above problems, it is an object of the present invention to promote appropriate measures for an abnormal river image.

The technical means of the present invention for solving the above technical problems is characterized by the following points.
The river monitoring device according to one aspect of the present invention includes a river monitoring camera that monitors a river, a first region in which the imaging state of a river image captured by the river monitoring camera changes according to a change in the water surface of the river, and the said. A region detection unit that detects a second region that is different from the first region and whose imaging state does not change even if the water surface of the river changes, and a subject that detects subjects depicted in the first region and the second region. The detection unit, the abnormality determination unit that determines the presence or absence of an abnormality in the river image based on the state of the subject depicted in at least one of the first region and the second region, and the abnormality determination unit have an abnormality in the river image. It is provided with an abnormality notification unit for notifying the abnormality when it is determined.

Further, in one aspect of the present invention, the abnormality determination unit determines whether or not there is an abnormality in the river image based on the size or change of the first subject other than the water surface of the river detected in the first region by the subject detection unit. do.
Further, in one aspect of the present invention, the abnormality determination unit determines the presence or absence of an abnormality in the river image based on the size or change of the second subject other than the structure detected in the second region by the subject detection unit. ..

Further, in one aspect of the present invention, the abnormality determination unit is a subject that is not detected by the subject detection unit at the initial stage among the subjects that are continuously detected by the subject detection unit in the first region or the second region for a predetermined time or longer. When the size of the subject is greater than or equal to a predetermined value, or when the subject changes so as to become larger, it is determined that there is an abnormality in the river imaging.
Further, in one aspect of the present invention, the abnormality determination unit performs river imaging when the subject is in a predetermined position which interferes with the monitoring of the river, or when the subject makes a change not related to the change in the water surface of the river. Judge that there is an abnormality.

Further, in one aspect of the present invention, the abnormality determination unit determines the presence or absence of an abnormality in the river image based on the image information including at least one of the brightness, hue, saturation, and brightness of the river image. ..
Further, in one aspect of the present invention, the detection of the first region and the second region by the region detection unit, the detection of the subject by the subject detection unit, and the determination of the abnormality of the river image by the abnormality determination unit are executed at a predetermined cycle. To.

Further, in one aspect of the present invention, the abnormality notification unit transmits the abnormality information indicating that the river image has an abnormality and the data of the river image having the abnormality to the terminal device owned by the administrator.
Further, in one aspect of the present invention, the river monitoring device includes an image storage unit that stores data of a river image captured by a river monitoring camera, and an image display unit that displays a river image based on the data.

A river monitoring system according to an aspect of the present invention includes a river monitoring device and a terminal device owned by an administrator and capable of communicating with the river monitoring device.
Further, in one aspect of the present invention, the river monitoring device includes a river monitoring camera that monitors the river, a river image management device having an image storage unit that stores data of the river image captured by the river monitoring camera, and a river image. The river image management device receives the river image data from the river monitoring camera and stores the data in the image storage unit, including a display device having an image display unit that displays the river image based on the data of the above. And the data is transmitted to the display device.

Further, in one aspect of the present invention, the river image management device is a first region in which the imaging state changes according to a change in the water surface of the river in the river image, and a region different from the first region of the river. A region detection unit that detects a second region in which the imaging state does not change even if the water surface changes, a subject detection unit that detects subjects depicted in the first region and the second region, and the first region and the second region. An abnormality determination unit that determines whether or not there is an abnormality in the river image based on the condition of the subject depicted on at least one of them, and an abnormality that notifies the abnormality when the abnormality determination unit determines that there is an abnormality in the river image. It has at least one of a notification unit.

According to the present invention, it is possible to promote appropriate measures for an abnormal river image.

It is a block diagram of a river monitoring system. It is a flowchart which shows the operation of a river image management apparatus. It is a flowchart which shows the operation of a river image management apparatus. It is a figure which shows an example of the early normal river image. It is a figure which shows an example of a normal river image at the time of operation. It is a figure which shows an example of an abnormal river image at the time of operation. It is a figure which shows another example of an abnormal river image at the time of operation. It is a figure which shows another example of an abnormal river image at the time of operation.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, the configuration of the river monitoring system 1 of the present embodiment will be described.
FIG. 1 is a configuration diagram of a river monitoring system 1.
The river monitoring system 1 includes a river monitoring device 2 and a mobile terminal 6. The river monitoring device 2 includes a river monitoring camera 3, a river image management device 4, and a display device 5.

The river monitoring camera 3 is installed near the monitored part of the river and monitors the monitored part of the river by taking an image. The river monitoring camera 3 includes a control unit 3a, a storage unit 3b, an image pickup unit 3c, an image processing unit 3d, and a communication unit 3e. The control unit 3a is composed of a CPU, a memory, and the like. The control unit 3a controls the operation of each unit of the river monitoring camera 3. The storage unit 3b is composed of a volatile or non-volatile memory or the like.

The imaging unit 3c images the monitored portion of the river and its surroundings. The image processing unit 3d processes the analog signal of the river image output from the image pickup unit 3c to generate digital data.
The communication unit 3e is composed of a communication module for communicating with the river image management device 4. The control unit 3a transmits the data of the river image imaged by the image pickup unit 3c and processed by the image processing unit 3d to the river image management device 4 by the communication unit 3e.

The river image management device 4 is a server or the like installed in a management center (not shown). The river image management device 4 includes a control unit 4a, a storage unit 4b, an image processing unit 4d, and a communication unit 4e. The control unit 4a is composed of a CPU, a memory, and the like. The control unit 4a controls the operation of each unit of the river image management device 4. The control unit 4a is provided with an area detection unit 4f, a subject detection unit 4g, an abnormality determination unit 4h, and an abnormality notification unit 4i. The area detection unit 4f, the subject detection unit 4g, the abnormality determination unit 4h, and the abnormality notification unit 4i are configured by a software program. As another example, each of these parts 4f, 4g, 4h, and 4i may be configured by hardware. The storage unit 4b is composed of a memory, a hard disk, and the like.

The communication unit 4e is composed of a river monitoring camera 3, a display device 5, and a communication module for communicating with the mobile terminal 6. The control unit 4a receives the river image data transmitted from the river monitoring camera 3 by the communication unit 4e, and stores (saves) the data in the storage unit 4b. The storage unit 4b is an image storage unit that stores a river image captured by the river monitoring camera 3. The image processing unit 4d reproduces the river image data read from the storage unit 4b by the control unit 4a. The control unit 4a reads out the river image data stored in the storage unit 4b and transmits the data to the display device 5 or the mobile terminal 6.

The area detection unit 4f is a first region in which the imaging state changes according to a change in the water surface of the river in a river image based on the data image-processed by the image processing unit 4d, and a region different from the first region. Therefore, the second region where the imaging state does not change even if the water surface of the river changes is detected. The subject detection unit 4g detects the subject depicted in the first region and the second region. The subjects depicted in the first region include rivers (monitoring points), water surfaces of rivers, structures around rivers, and other foreign substances. The subject depicted in the second region includes structures around the river and other foreign substances.

The structure is a structure that is composed of a plurality of materials, members, etc. and whose weight is supported by a foundation, etc., and is, for example, a concrete structure made of concrete or a main member. Includes steel structures made of steel, and soil structures such as embankments, embankments, and banks that use soil.
The abnormality determination unit 4h determines whether or not there is an abnormality in the river image based on the state of the subject depicted in at least one of the first region and the second region of the river image. The abnormality notification unit 4i notifies the abnormality when the abnormality determination unit 4h determines that the river image has an abnormality. Specifically, the abnormality notification unit 4i transmits the abnormality information indicating that the river image has an abnormality to the mobile terminal 6 by the communication unit 4e. As another example, the river image management device 4 notifies the managers in the vicinity of the abnormality in the river image audibly, visually, or tactilely by sound, light, vibration, or the like. A notification unit may be configured.

The display device 5 is a personal computer or the like installed at an arbitrary place. The display device 5 includes a control unit 5a, a storage unit 5b, a display unit 5c, an operation unit 5d, and a communication unit 5e. The control unit 5a is composed of a CPU, a memory, and the like. The control unit 5a controls the operation of each unit of the display device 5. The storage unit 5b is composed of a memory or the like. The display unit 5c is composed of a display such as a liquid crystal display. The operation unit 5d is composed of operation keys, a mouse, and the like.

The communication unit 5e is composed of a communication module for communicating with the river image management device 4. Based on the instruction input by the operation unit 5d, the control unit 5a transmits a signal indicating the content of the instruction to the river image management device 4 by the communication unit 5e. The signal transmitted from the terminal device 5 to the river image management device 4 includes, for example, a request signal instructing the transmission of the river image. When the communication unit 5e receives the river image data transmitted from the river image management device 4 in response to this request signal, the control unit 5a causes the display unit 5c to display the river image based on the data. The display unit 5c is an image display unit that displays a river image captured by the river monitoring camera 3.

The mobile terminal 6 is a terminal device such as a portable smartphone or tablet machine owned by the administrator. The mobile terminal 6 includes a control unit 6a, a storage unit 6b, a display unit operation unit 6c, and a communication unit 6e. The control unit 6a is composed of a CPU, a memory, and the like. The control unit 6a controls the operation of each unit of the mobile terminal 6. The storage unit 6b is composed of a memory or the like. The display operation unit 6c is composed of a touch pad or the like.

The communication unit 6e is composed of a communication module for communicating with the river image management device 4. Based on the instruction input by the display operation unit 6c, the control unit 6a transmits a signal indicating the content of the instruction to the river image management device 4 by the communication unit 6e. Further, the control unit 6a receives the river image data transmitted from the river image management device 4 by the communication unit 6e, and causes the display operation unit 6c to display the river image based on the data.

The communication units 3e, 4e, 5e, and 6e of each device 3 to 6 are, for example, Wi-Fi (Wireless Fidelity, registered trademark), BLE (Bluetooth (registered trademark) Low Energy) of the IEEE802.11 series, which are communication standards. Wireless communication can be performed by LPWA (Low Power, Wide Area), LPWAN (Low-Power Wide-Area Network), or the like.
As another example, the communication units 3e, 4e, 5e, and 6e may be configured so as to be able to communicate wirelessly or by wire via a public communication network such as the Internet, a mobile phone communication network, or another data communication network. Further, although FIG. 1 shows each device 3 to 6 one by one, the present invention is not limited to this, and each device 3 to 6 may be incorporated into several river monitoring systems 1 as appropriate.

Next, the operation of the river monitoring system 1 will be described.
2A and 2B are flowcharts showing the operation of the river image management device 4. 3 to 7 are diagrams showing an example of a river image captured by the river monitoring camera 3. Specifically, FIG. 3 shows an example of a normal river image at the initial stage (before operation) of the river monitoring system 1, and FIG. 4 shows an example of a normal river image during operation of the river monitoring system 1. FIG. -FIG. 7 shows an example and another example of an abnormal river image during operation.

During operation of the river monitoring system 1, in the river monitoring camera 3, the control unit 3a constantly images the river by the image pickup unit 3c, and the data of the river image input from the image pickup unit 3c via the image processing unit 3d is input. It is transmitted to the river image management device 4 by the communication unit 3e at a predetermined cycle. Then, in the river image management device 4, the control unit 4a receives the river image data transmitted from the river monitoring camera 3 by the communication unit 4e, and stores (saves) the data in the storage unit 4b (FIG. 2A). S1). At this time, the control unit 4a associates the river image data with the date and time when the river image was captured and stores it in the storage unit 4b.

Even in the initial stage (before operation) of the river monitoring system 1, the river image captured by the river monitoring camera 3 is stored (stored) in the storage unit 4b of the river image management device 4 as described above. This initial river image is stored in a storage area of the storage unit 4b, which is different from the river image at the time of operation, as an example of a normal river image.
Next, the control unit 4a of the river image management device 4 confirms whether or not it is a predetermined abnormality determination timing by referring to the time counting time of the built-in timer or the like. While the abnormality determination timing is not reached (S2: NO), the control unit 4a repeatedly receives and stores river image data from the river monitoring camera 3 (S1). Then, when the abnormality determination timing is reached (S2: YES), the control unit 4a reads out the data of a plurality of river images captured at a predetermined time from the storage unit 4b (S3).

By waiting for the abnormality determination timing in the process S2, the subsequent processes are executed in a predetermined cycle. In the process S3, the control unit 4a reads out a plurality of river image data, for example, from the latest river image data stored in the storage unit 4b to the river image data stored N times (N ≧ 2) ago. .. However, it is not necessary to read out all the river image data stored in the storage unit 4b.

Next, the control unit 4a processes the data of the plurality of river images read from the storage unit 4b by the image processing unit 4d, and reproduces the river image based on the data. Then, the abnormality determination unit 4h detects image information including at least one of the lightness / darkness, hue, saturation, and lightness of each river image (S4).
Next, the abnormality determination unit 4h confirms whether or not the image value (brightness, hue, saturation, lightness, etc.) indicated by the image information is within the corresponding predetermined normal range stored in advance. The normal range of the image value is set, for example, based on the image value detected from the river image captured once or more by the river monitoring camera 3 before the operation of the river monitoring system 1, and is stored in the storage unit 4b. There is.

When each regenerated river image is, for example, a normal river image G1 as shown in FIG. 4, the image value of each river image (if a plurality of image values are detected, all the image values) is It falls within the normal range (S5: YES). In this case, the region detection unit 4f detects the first region and the second region in each river image based on the data of the plurality of river images (S7).
On the other hand, the image values such as brightness, hue, saturation, or brightness of any of the river images have deteriorated, and the river image has a sense of discomfort (unclearness, etc.) to the viewer, as shown in FIG. 5, for example. In the case of an abnormal river image G2 that gives the above, the image value of the river image (if a plurality of image values are detected, one of the image values) is out of the normal range (S5: NO). In this case, the abnormality determination unit 4h determines that the river image in which the image value is detected has an abnormality, and records the abnormality information indicating the content of the abnormality in the storage unit 4b (S6). Then, the region detection unit 4f detects the first region and the second region in each river image (S7).

In the process S7, the region detection unit 4f detects a region in the river image in which the imaging state changes according to the change in the water surface of the river as the first region, and the region in which the imaging state does not change even if the water surface of the river changes. Is detected as the second region. Specifically, for example, the region detection unit 4f detects the initial first region and the second region in the river image in advance based on the data of a plurality of initial normal river images stored in the storage unit 4b. It is stored in the storage unit 4b. Then, with reference to the initial first region and the second region, the first region and the second region in the river image read by the process S3 are detected.

Specifically, for example, in the river image G1 as shown in FIG. 4, the region where the height of the water surface W of the river can change (the region below the top of the water level indicator 13) is the first region R1 (dotted line). It is detected by the area detection unit 4f as (within the enclosed range), and the other parts are detected by the area detection unit 4f as the second area R2 (the same applies to FIGS. 3, 6, and 7). The positions of the first region and the second region may be constant in all river images, or the positions may change depending on the river image.

Next, the subject detection unit 4g detects the water surface of the river and the first subject other than the water surface of the river from the first region of each river image (S8). At this time, the subject detection unit 4g was not detected in the first region of the initial river image stored in the storage unit 4b among the subjects other than the water surface of the river that have been continuously depicted in the first region for a predetermined time or longer. The subject is detected as the first subject.
Further, the subject detection unit 4g detects a structure and a second subject other than the structure from the second region of each river image (S9). At this time, the subject detection unit 4g is a subject other than the structure that has been continuously depicted in the second region for a predetermined time or longer and is not detected in the second region in the initial river image stored in the storage unit 4b. Is detected as the second subject.

For example, in the case of the river image G1 as shown in FIG. 4, first, the subject detection unit 4g has the water surface W of the river, the wire mesh 11, the concrete wall 12, the water level indicator 13, the steel material 14, as the subject from the first region R1. And foreign matter 15 such as fallen leaves and dust is detected. Further, the subject detection unit 4g detects the wire mesh 11, the concrete walls 12, 16 and the foreign matter 17 as the subject from the second region R2. Next, the subject detection unit 4g excludes subjects 11 to 17 that are not depicted in other river images. By this exclusion process, the subject temporarily depicted in the river image, that is, the subject disappearing with the passage of time, does not become a cause of the abnormality in the river image.

Then, among the remaining subjects 11 to 15 in the first region R1, the subject (foreign matter) 15 that was not detected in the first region R1 of the initial river image G0 as shown in FIG. 3 is detected by the subject detection unit 4g. Detect as one subject. Further, among the remaining subjects 11, 12, 16 and 17 in the second region R2, the subjects 11, 12, 16 detected in the second region R2 of the initial river image G0 are used as a structure by the subject detection unit 4g. To detect. Further, among the remaining subjects 11, 12, 16 and 17 in the second region R2, the subject (foreign matter) 17 that was not detected in the second region R2 of the initial river image G0 is detected by the subject detection unit 4g as the second subject. Detect as.

Further, for example, when foreign matter 18 such as dirt or dust adheres to the surface of the lens of the image pickup unit 3c of the river monitoring camera 3, an abnormal river image G3 in which the foreign matter 18 is depicted as shown in FIG. 6 is used for river monitoring. It is imaged by the camera 3 and stored in the storage unit 4b. In this case, the subject detection unit 4g detects the water surface W of the river from the first region R1 and also detects the foreign matter 18 as the first subject. Further, the subject detection unit 4g detects only the structures 11, 12, and 16 from the second region R2.

Further, for example, when a spider web 19 is stretched on the surface of the lens of the imaging unit 3c or immediately in front of the surface, as shown in FIG. 7, an abnormal river image G4 in which the spider web 19 is depicted is used for river monitoring. It is imaged by the camera 3 and stored in the storage unit 4b. In this case, the subject detection unit 4g detects the water surface W of the river from the first region R1 and detects the spider web (foreign matter) 19 as the first subject. Further, the subject detection unit 4g detects the structures 11, 12, and 16 from the second region R2, but does not detect the second subject because the spider web 19 does not extend to the second region R2 so much. If the spider web is large and covers most of the second region R2, the spider web may be detected as the second subject.

Next, the abnormality determination unit 4h compares the sizes of the first subject and the second subject detected by the subject detection unit 4g with the corresponding predetermined threshold values stored in advance. The sizes of the first subject and the second subject are, for example, the ratio of the depiction area (number of depiction pixels) of the first subject to the depiction area (number of depiction pixels) of the first region of the river image, and the depiction area of the second region. It is the ratio of the depiction area of the second subject to. The threshold value to be compared with the sizes of the first subject and the second subject is set to, for example, a depiction area that interferes with the monitoring (imaging) of the river and the vicinity of the river by the river monitoring camera 3.

When the size of at least one of the first subject and the second subject is equal to or larger than the threshold value (S10: YES in FIG. 2B), the abnormality determination unit 4h has an abnormality in the river image in which the subject is depicted. (S11), the abnormality information indicating the content of the abnormality is recorded in the storage unit 4b. For example, as in the river image G3 shown in FIG. 6, when the first subject (foreign matter) 18 is depicted at a large ratio (for example, 25% or more) with respect to the first region R1, the size of the first subject 18 is large. Is equal to or greater than the threshold value, and the abnormality determination unit 4h determines that the river image G3 has an abnormality. Then, the abnormality determination unit 4h confirms the change in size between the first subject and the second subject detected by the subject detection unit 4g.

Further, when the sizes of both the first subject and the second subject are less than the threshold value (S10: NO), the abnormality determination unit 4h immediately confirms the change in the sizes of the first subject and the second subject. For example, as in the river image G1 shown in FIG. 4, the first subject (foreign matter) 15 and the second subject (foreign matter) 17 are in a small ratio (for example, less than 10%) with respect to the first region R1 and the second region R2. When depicted, the sizes of the first subject 15 and the second subject 17 are less than the threshold value.

Next, the abnormality determination unit 4h confirms that at least one of the first subject and the second subject is changed so as to be larger (the depiction area tends to be enlarged) (S12: YES). .. In this case, the abnormality determination unit 4h determines that the river image in which the subject is depicted has an abnormality, and records the abnormality information indicating the content of the abnormality in the storage unit 4b (S13). For example, in the river image G4 shown in FIG. 7, when the spider web 19 which is the first subject spreads in the first region R1, the abnormality determination unit 4h changes so that the subject 19 becomes larger. Is confirmed, and it is determined that there is an abnormality in the river image G3. Then, the abnormality determination unit 4h detects the depiction positions of the first subject and the second subject.

Further, the abnormality determination unit 4h confirms that both the first subject and the second subject have not changed so as to be large (S10: NO). In this case, the abnormality determination unit 4h immediately detects the depiction positions of the first subject and the second subject.
Next, the abnormality determination unit 4h confirms that at least one of the first subject and the second subject is in the monitoring obstruction position (S14: YES). In this case, the abnormality determination unit 4h determines that the river image in which the subject is depicted has an abnormality, and records the abnormality information indicating the content of the abnormality in the storage unit 4b (S15). The monitoring obstruction position is, for example, a depiction position that hides the water level indicator 13 (FIGS. 3 to 7) indicating the height of the water surface of the river, the structures around the water surface, and the water surface of the river in the river image. That is.

For example, as in the river images G3 and G4 shown in FIGS. 6 and 7, when the foreign matter 18 or 19 as the first subject is in a depiction position that hides most of the water surface W of the river or the water level indicator 13, an abnormality is determined. Unit 4h determines that the river images G3 and G4 have an abnormality because the subjects 18 and 19 are in the monitoring obstruction position (S15). Then, the abnormality determination unit 4h detects the change state between the first subject and the second subject.

Further, the abnormality determination unit 4h confirms that both the first subject and the second subject are not in the monitoring obstruction position (S14: NO). In this case, the abnormality determination unit 4h immediately detects the change state between the first subject and the second subject. For example, as shown in the river image G1 shown in FIG. 4, the first subject (foreign matter) 15 and the second subject (foreign matter) 17 almost hide the water surface W of the river, the surrounding structures 11 to 14, and the water level indicator 13. If not, since the subjects 15 and 17 are not in the monitoring obstruction position, the abnormality determination unit 4h does not determine that the river image G3 has an abnormality in the process S15 of FIG. 2B.

Next, the abnormality determination unit 4h confirms that at least one of the first subject and the second subject has a change different from the water surface of the river (S16: YES). In this case, the abnormality determination unit 4h determines that the river image in which the subject is depicted has an abnormality, and records the abnormality information indicating the content of the abnormality in the storage unit 4b (S17). The change state of the water surface of the river is detected by the control unit 4a and stored in the storage unit 4b, for example, based on the data of a plurality of river images acquired at the initial stage.

For example, the subjects 18 and 19 shown in FIGS. 6 and 7 change differently from the water surface W of the river (enlargement or movement larger than the water surface W of the river). It is judged that there is. Then, the abnormality notification unit 4i confirms whether or not the abnormality information indicating the abnormality of the river image is recorded in the storage unit 4b.
Further, the abnormality determination unit 4h confirms that both the first subject and the second subject do not change differently from the water surface of the river (S16: NO). In this case, the abnormality notification unit 4i immediately confirms whether or not the abnormality information is recorded.

When the abnormality notification unit 4i confirms that the abnormality information is recorded in the storage unit 4b (S18: YES), the abnormality notification unit 4i communicates the abnormality information with the river image data indicating that the abnormality information has an abnormality. The unit 4e notifies (transmits) to the mobile terminal 6 (S19). After that, the processes from the process S1 in FIG. 2A to the subsequent processes are repeated. Further, if the abnormality information is not stored in the storage unit 4b (S18: NO), the abnormal information and the like are not notified, and the processes from the process S1 in FIG. 2A to the subsequent processes are repeated.

In the mobile terminal 6, when the communication unit 6e (FIG. 1) receives the abnormality information notified by the abnormality notification unit 4i and the data of the river image having the abnormality as described above, the control unit 6a is based on the data. The display operation unit 6c displays the river image with the abnormality and the abnormality information.
The river manager usually displays the river image captured by the river monitoring camera 3 on the display device 5 periodically, and monitors (visually) the river image. However, when an abnormality occurs in the river image, the mobile terminal 6 owned by the administrator is notified of the abnormality information indicating the content of the abnormality and the river image having the abnormality as described above. Therefore, the administrator can immediately confirm the river image in which the abnormality has occurred by the mobile terminal 6 and the content of the abnormality, and recognize the seriousness of the abnormality. Then, for example, the river monitoring camera 3 is used to take appropriate measures such as going to the site where the river monitoring camera 3 is installed, removing foreign substances 17 to 19, and performing maintenance on the river monitoring camera 3. The captured river image can be restored to the normal state.

In the above-described embodiment, as shown in FIG. 1, the river image management device 4 is provided with an area detection unit 4f, a subject detection unit 4g, an abnormality determination unit 4h, and an abnormality notification unit 4i. The river monitoring camera 3 or the display device 5 may be provided with an area detection unit, a subject detection unit, an abnormality determination unit, or an abnormality notification unit. Further, the image display unit for displaying the river image may be provided in the river image management device 4 instead of the display unit 5c of the display device 5.

Further, the river image handled by the river monitoring camera 3, the river image management device 4, the display device 5, or the mobile terminal 6 may be a still image or a moving image.
Further, for example, data of a plurality of normal river images captured by the river monitoring camera 3 at the initial stage or data of a plurality of abnormal river images captured by the river monitoring camera 3 are stored in advance as a river image filter and operated. Occasionally, it may be determined whether or not there is an abnormality in the river image based on the result of comparing the data of the river image captured by the river monitoring camera 3 with the river image filter.

Further, the presence or absence of an abnormality in the river image may be determined based on the focus information indicating the focal position of the image pickup unit 3c when the river image is captured by the image pickup unit 3c of the river monitoring camera 3. The image pickup unit 3c normally adjusts the focal position so that the focus is on the water surface of the river, and captures the image of the river. For example, there is a subject in front of the water surface of the river or a subject larger than the water surface of the river. In such a case, the focus position may be adjusted so that the subject is in focus, and a river image may be taken. In this case, in the captured river image, the water surface of the river is captured in a so-called out-of-focus state. This is also an abnormal river image.

In order to determine the abnormality of the river image as described above, for example, in the river monitoring camera 3, the control unit 3a communicates the focus information indicating the focal position of the image pickup unit 3c when the river image is imaged together with the river image data. It is transmitted to the river image management device 4 by the unit 3e. Then, in the river image management device 4, when the abnormality determination unit 4h focuses on the focal position indicated by the focal information of the river image and a predetermined focal position stored in advance (for example, when the water surface of the river is focused at the time of initial imaging). The difference from the focal position) is detected, and when the difference is equal to or greater than a predetermined threshold value, it is determined that there is an abnormality in the river image.

Further, instead of the above focal position, it may be determined whether or not there is an abnormality in the river image based on the time required for the imaging unit 3c to adjust the focal position (focus adjustment time) and a predetermined threshold value. .. Further, the number of river images whose focal position or focus adjustment time is equal to or greater than the threshold value is counted, and when the number of the river images is equal to or greater than a predetermined number, it is determined that the river image has an abnormality. You may notify the abnormality.

In addition, artificial intelligence (AI) may be used to determine the presence or absence of abnormalities in river images. Specifically, for example, in the river image management device 4, there is an abnormality that reads out a plurality of river images stored in the storage unit 4b, performs deep learning by artificial intelligence, and gives a sense of discomfort to the viewer. Build river image trends as a trained model. After that, the control unit 4a determines whether or not there is an abnormality in the river image based on the trained model and the river image captured by the river monitoring camera 3.

The river monitoring device 2 and the river monitoring system 1 of the present embodiment have the following effects.
The river monitoring device 2 of the present embodiment includes a river monitoring camera 3 that monitors a river, a first region in which the image pickup state changes in response to a change in the water surface of the river in a river image captured by the river monitoring camera 3, and the said. The area detection unit 4f that detects the second area, which is different from the first area and whose imaging state does not change even if the water surface of the river changes, and the subjects depicted in the first area and the second area are detected. The subject detection unit 4g, the abnormality determination unit 4h for determining the presence or absence of an abnormality in the river image based on the state of the subject depicted in at least one of the first region and the second region, and the abnormality determination unit are used for the river image. It is provided with an abnormality notification unit 4i for notifying the abnormality when it is determined that there is an abnormality.

According to the above configuration, it is determined that there is an abnormality in the river image based on the state of the subject depicted in the first region or the second region of the river image captured by the river monitoring camera 3, and the abnormality is determined. Is notified, so the administrator can be urged to take appropriate measures.
Further, in the present embodiment, the abnormality determination unit 4h determines whether or not there is an abnormality in the river image based on the size or change of the first subject other than the water surface of the river detected in the first region by the subject detection unit 4g. do. As a result, it is possible to determine that there is an abnormality in the river image that interferes with the monitoring (imaging) of the water surface of the river and gives a sense of discomfort to the viewer, and to notify the abnormality.

Further, in the present embodiment, the abnormality determination unit 4h determines the presence or absence of an abnormality in the river image based on the size or change of the second subject other than the structure detected in the second region by the subject detection unit 4g. .. This interferes with the monitoring of the boundary between the water surface of the river and the structures around the water surface, determines that there is an abnormality in the river image that makes the viewer feel uncomfortable, and notifies the abnormality. can do.

Further, in the present embodiment, the abnormality determination unit 4h is not detected by the subject detection unit 4g at the initial stage among the subjects continuously detected by the subject detection unit 4g in the first region or the second region for a predetermined time or longer. When the size of the subject is equal to or larger than a predetermined value, or when the subject changes so as to become larger, it is determined that there is an abnormality in the river imaging. As a result, anomalies in the river image that suddenly appeared in the river image and had a size that interfered with the monitoring of the water surface of the river and the boundary between the water surface and the surrounding structures were captured. It is possible to determine that there is and notify the abnormality.

Further, in one aspect of the present invention, the abnormality determination unit 4h captures a river when the subject is in a predetermined position which interferes with the monitoring of the river, or when the subject makes a change not related to the change in the water surface of the river. Judge that there is an abnormality in. As a result, it is possible to determine that there is an abnormality in the river image in which a subject that changes so as to interfere with the monitoring of the water surface of the river is reflected, and notify the abnormality.

Further, in the present embodiment, the abnormality determination unit 4h determines the presence or absence of an abnormality in the river image based on the image information including at least one of the brightness, hue, saturation, and brightness of the river image. As a result, it is possible to determine that the river image has an abnormality in brightness or color that tends to give a sense of discomfort to the viewer, and notify the abnormality.
Further, in one aspect of the present invention, the region detection unit 4f detects the first region and the second region, the subject detection unit 4g detects the subject, and the abnormality determination unit 4h determines the abnormality of the river image in a predetermined cycle. Is executed by. As a result, the area detection unit 4f, the subject detection unit 4g, and the abnormality determination unit 4h thin out the river image to some extent to perform processing, so that the processing load of each unit 4f, 4g, and 4h is reduced, and the abnormality notification frequency is also reduced. Can be made to.

Further, in the present embodiment, the abnormality notification unit 4i transfers the abnormality information indicating that the river image has an abnormality and the data of the river image having the abnormality to the terminal device (portable terminal) 6 owned by the administrator. Send. As a result, the administrator can confirm the abnormality information and the river image with the abnormality by the terminal device 6, and take more appropriate measures.
Further, in the present embodiment, the river monitoring device 2 includes an image storage unit 4b that stores the data of the river image captured by the river monitoring camera 3 and an image display unit 5c that displays the river image based on the data. Be prepared. As a result, the river image captured by the river monitoring camera 3 can be saved and confirmed by displaying the river image at a desired timing.

The river monitoring system 1 of the present embodiment includes a river monitoring device 2 and a terminal device 6 owned by an administrator and capable of communicating with the river monitoring device 2. According to this configuration, the river monitoring device 2 determines that there is an abnormality in the river image captured by the river monitoring camera 3, and the terminal device 6 is notified of the abnormality. Therefore, the administrator should take appropriate measures. Can be prompted.
Further, in the present embodiment, the river monitoring device 2 includes a river monitoring camera 3 for monitoring the river and a river image management device 4 having an image storage unit 4b for storing the data of the river image captured by the river monitoring camera 3. The river image management device 4 receives the river image data from the river monitoring camera 3 and includes the display device 5 having an image display unit 5c for displaying the river image based on the river image data. The data is stored in the image storage unit 4b and the data is transmitted to the display device 5. As a result, the river image captured by the river monitoring camera 3 can be stored in the river image management device 4, and the river image can be confirmed by the display device at a desired timing.

Further, in the present embodiment, the river image management device 4 has a first region in which the imaging state changes according to a change in the water surface of the river in the river image, and a region different from the first region and the water surface of the river. The area detection unit 4f that detects the second region where the image pickup state does not change even if the image changes, the subject detection unit 4g that detects the subject depicted in the first region and the second region, and the first region and the second region. An abnormality determination unit 4h that determines whether or not there is an abnormality in the river image based on the state of the subject depicted in at least one of them, and an abnormality determination unit that determines that there is an abnormality in the river image, notifies the abnormality. It has at least one of the abnormality notification unit 4i.

As a result, based on the state of the subject depicted in the first region or the second region of the river image stored in the river image management device 4, it is determined that the river image has an abnormality, and the abnormality is notified. Therefore, it is possible to encourage the administrator to take appropriate measures.
Although the present invention has been described above, it should be considered that the embodiments disclosed this time are exemplary in all respects and are not restrictive. The scope of the present invention is shown by the scope of claims rather than the above description, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

1 River monitoring system 2 River monitoring device 3 River monitoring camera 4 River image management device 4b Storage unit (image storage unit)
4f Area detection unit 4g Subject detection unit 4h Abnormality judgment unit 4i Abnormality notification unit 5 Display device 5c Display unit (image display unit)
6 Mobile terminal (terminal device)
11-14, 16 Structures 15, 17-19 1st subject, 2nd subject G0 Initial river image G1 to G4 River image R1 1st area R2 2nd area W River water surface

The technical means of the present invention for solving the above technical problems is characterized by the following points.
The river monitoring device according to one aspect of the present invention includes a river monitoring camera that monitors a river, a first region in which the imaging state of a river image captured by the river monitoring camera changes according to a change in the water surface of the river, and the said. A region detection unit that detects a second region that is different from the first region and whose imaging state does not change even if the water surface of the river changes, and a subject that detects subjects depicted in the first region and the second region. A detection unit, an abnormality determination unit that determines whether or not there is an abnormality in the river image that interferes with river monitoring by the river monitoring camera , based on the state of the subject depicted in at least one of the first region and the second region. When the abnormality determination unit determines that there is an abnormality in the river image, the abnormality notification unit is provided to notify the abnormality, and the abnormality determination unit is other than the water surface of the river detected in the first region by the subject detection unit. The presence or absence of an abnormality in the river image is determined based on the size or change of the subject, and the abnormality of the river image is determined based on the size or change of the subject other than the structure detected in the second region by the subject detection unit. Judge the presence or absence .

Further, in one aspect of the present invention, the river image management device is a first region in which the imaging state changes according to a change in the water surface of the river in the river image, and a region different from the first region of the river. A region detection unit that detects a second region whose imaging state does not change even if the water surface changes, a subject detection unit that detects subjects depicted in the first region and the second region, and a first region and a second region. Based on the condition of the subject depicted on at least one of them, the abnormality judgment unit that determines whether or not there is an abnormality in the river image that interferes with the river monitoring by the river monitoring camera, and the abnormality judgment unit determines that there is an abnormality in the river image. In this case, it has at least one of the abnormality notification unit for notifying the abnormality, and the abnormality determination unit is the size of the subject other than the water surface of the river detected in the first region by the subject detection unit. Based on the change, the presence or absence of abnormality in the river image is determined, and the presence or absence of abnormality in the river image is determined based on the size or change of the subject other than the structure detected in the second region by the subject detection unit .

Claims (12)

A river monitoring camera that monitors rivers and
In the river image captured by the river monitoring camera, even if the first region where the imaging state changes according to the change in the water surface of the river and the region different from the first region and the water surface of the river changes. A region detection unit that detects the second region where the imaging state does not change,
A subject detection unit that detects a subject depicted in the first region and the second region, and a subject detection unit.
An abnormality determination unit for determining the presence or absence of an abnormality in the river image based on the state of the subject depicted in at least one of the first region and the second region.
A river monitoring device including an abnormality notification unit for notifying the abnormality when the abnormality determination unit determines that the river image has an abnormality. The first aspect of claim 1, wherein the abnormality determination unit determines the presence or absence of an abnormality in the river image based on the size or change of a subject other than the water surface of the river detected by the subject detection unit in the first region. River monitoring device. The abnormality determination unit is described in claim 1 or 2 for determining the presence or absence of an abnormality in the river image based on the size or change of a subject other than the structure detected by the subject detection unit in the second region. River monitoring device. The abnormality determination unit is the size of the subject that was not detected by the subject detection unit at the initial stage among the subjects that were continuously detected by the subject detection unit in the first region or the second region for a predetermined time or longer. The river monitoring device according to any one of claims 1 to 3, wherein it is determined that there is an abnormality in the river imaging when the value is equal to or more than a predetermined value, or when the subject changes so as to become larger. The abnormality determination unit has an abnormality in the river imaging when the subject is in a predetermined position which interferes with the monitoring of the river, or when the subject makes a change not related to the change in the water surface of the river. The river monitoring device according to claim 4. Any of claims 1 to 5, wherein the abnormality determination unit determines the presence or absence of an abnormality in the river image based on image information including at least one of the brightness, hue, saturation, and brightness of the river image. The river monitoring device according to item 1. A claim in which the detection of the first region and the second region by the region detection unit, the detection of the subject by the subject detection unit, and the abnormality determination of the river image by the abnormality determination unit are executed in a predetermined cycle. The river monitoring device according to any one of 1 to 6. The abnormality notification unit is any one of claims 1 to 7 that transmits the abnormality information indicating that the river image has an abnormality and the data of the river image having the abnormality to the terminal device owned by the administrator. The river monitoring device described in the section. An image storage unit that stores data of the river image captured by the river monitoring camera, and an image storage unit.
The river monitoring device according to any one of claims 1 to 8, further comprising an image display unit that displays the river image based on the river image data. The river monitoring device according to any one of claims 1 to 9, and the river monitoring device.
A river monitoring system including a terminal device owned by an administrator and capable of communicating with the river monitoring device. The river monitoring device is
A river monitoring camera that monitors rivers and
A river image management device having an image storage unit for storing data of river images captured by the river monitoring camera, and a river image management device.
A display device having an image display unit for displaying the river image based on the river image data, and the like.
The river monitoring according to claim 10, wherein the river image management device receives data of the river image from the river monitoring camera, stores the data in the image storage unit, and transmits the data to the display device. system. The river image management device is
In the river image, a first region in which the imaging state changes according to a change in the water surface of the river and a region different from the first region in which the imaging state does not change even if the water surface of the river changes. An area detection unit that detects two areas, and an area detection unit
A subject detection unit that detects a subject depicted in the first region and the second region, and a subject detection unit.
An abnormality determination unit for determining the presence or absence of an abnormality in the river image based on the state of the subject depicted in at least one of the first region and the second region.
The river monitoring system according to claim 11, further comprising at least one of an abnormality notification unit for notifying the abnormality when the abnormality determination unit determines that the river image has an abnormality.

Images