JP2020033792A - Device, method, system and program for detecting flowing snow ice - Google Patents

Abstract

To accurately calculate the amount of generated flowing snow ice and a duration time from occurrence.SOLUTION: A device 13 for detecting flowing snow ice comprises: a lighting control section 23 controlling lighting by a light source in a visible area or a light source in an invisible area to a water surface with necessity of detecting flowing snow ice; a camera control section 22 controlling capture by an area camera 11, of an area lit by the lighting; an image processing section 28 performing prescribed image processing to the captured image data and extracting a part corresponding to the flowing snow ice; a detected result determination section 29 calculating a capture ratio of the flowing snow ice to the whole image or a selected or separated area on the basis of data obtained by the image processing and calculating a duration time of ratio excess in a case where the capture ratio of the flowing snow ice is higher than a predetermined threshold value; and a transmission control section 30 controlling transmission of the duration time of ratio excess obtained by processing of the detected result determination section 29 to other information processing unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a drift ice / snow detector, a drift ice / snow detection method, a drift ice / snow detection system, and a program.

  Drift ice and snow (also called snow jam) flowing in rivers and headraces in winter in cold regions is a relatively soft lump of ice originating from snow that floats on the water surface such as waterways and water storage tanks, and contains a large amount of water. . For this reason, it spreads to some extent on the surface of the water, and at the same time, there are many portions that exist below the surface of the water more than ordinary ice. For this reason, drift ice and snow may be condensed and clogged in the headrace, and in some cases, the flow of the headrace may be stopped by the drift ice and snow. Also, for example, in a hydroelectric power plant, if such water containing drift ice and snow flows through the dam channel as water for power generation, it may cause a water intake failure, and in some cases, damage the power generation equipment. There is a risk that it will. Therefore, when drift ice and snow occur, it is necessary to accurately detect the drift ice and snow.

  Conventionally, in order to perform stable snow jam detection while preventing flow loss in waterways, line illumination or fan-shaped laser beams that perform illumination converged from above the water surface are used, and scattered light due to snow jam is detected by a line camera or area. A snow jam is detected by imaging with a camera and detecting a scattering area size by detecting a scattering area size, or by irradiating convergent light directed from the inside of a water channel to a water surface and imaging the light transmitted and scattered by the snow jam with the camera. There is a technology to detect (For example, Patent Document 1).

JP 2001-64952 A

  The conventional technique described in Patent Document 1 described above detects drift ice and snow. However, when drift ice and snow are detected, it is not always necessary to immediately remove the drift ice and snow. The duration of the event determines whether a removal operation is required. Patent Document 1 described above describes detecting an occurrence of drift ice and snow by obtaining an image of drift ice and snow, and automatically operating a drift ice and snow removal device when the drift ice and snow occurs. That is, by using the technology described in Patent Document 1 described above, it is possible to detect the occurrence of drift ice and snow using an image. It is not easy to accurately grasp the duration from the occurrence.

  Therefore, the present invention solves the above-mentioned problems, that is, a drift ice / snow detection device and a drift ice / snow detector which can accurately calculate the amount of ice and snow flowing in a river or a headrace and the duration of the ice ice and snow after the occurrence. It is an object to provide a detection method, a drift ice / snow detection system, and a program.

  In order to solve the above-described problems, a first aspect of the drift ice / snow detection device of the present invention includes an illumination control unit that controls illumination by a light source in a visible region or a light source in an invisible region with respect to a water surface where detection of drift ice / snow is required. Imaging control means for controlling imaging by an imaging device of an area irradiated with illumination controlled by illumination control means, and performing predetermined image processing on image data obtained by controlling imaging by the imaging control means. Image processing means for extracting a portion corresponding to drift ice and snow, and an imaging ratio of drift ice and snow with respect to the entire image or a selected or separated area based on data obtained by performing image processing by the image processing means. A detection result determining means for calculating and calculating a ratio excess duration when the imaging ratio of drift ice and snow is higher than a predetermined threshold value; and a ratio excess connection obtained by the processing of the detection result determining means. Time, and a sending control means for controlling the transmission to another information processing apparatus.

  According to another aspect of the drift ice / snow detection device of the present invention, the image processing means discriminates and separates a stationary portion and a moving portion based on a synthesis result of image data obtained within a predetermined time, Is determined to be a frozen part, and the stationary part / moving part separation processing of determining that the moving part is drift ice / snow is included.

  According to another aspect of the drift ice / snow detection device of the present invention, the image processing means may be configured such that the illumination controlled by the illumination control means is a light source in a visible region such as a white light source or a light source in an invisible region such as infrared light. In the image data that is captured and supplied, it is determined whether drift ice or snow is determined based on the difference in the reflectance of the reflected light or the difference in the absorptance of the absorbed light. By performing the processing, a portion determined to be drift ice or snow is extracted.

  According to another aspect of the drift ice / snow detection device of the present invention, the image processing means refers to a synthesis result of image data obtained within a predetermined time and determines a portion detected as an instantaneous change portion as noise. And a noise reduction processing means for eliminating the noise reduction processing.

  Another aspect of the drift ice / snow detection device of the present invention is that when detecting drift ice / snow, the detection location is daytime or night, or sunshine indicating whether it is the time of irradiation of the morning sun or sunset. A sunshine information acquisition unit for acquiring information, wherein the image processing unit performs predetermined image processing on the image data obtained by the imaging control unit based on the sunshine information obtained by the sunshine information acquisition unit. It is characterized by.

  Further, another aspect of the drift ice / snow detection device of the present invention further includes a weather information acquisition unit that acquires weather information such as weather, sunrise, sunset time, temperature, snowfall amount, and the like of a detection location of drift ice / snow and weather forecast information. Wherein the image processing means performs predetermined image processing on the image data obtained by the imaging control means based on the weather information and the weather forecast information obtained by the weather information obtaining means.

  Further, another aspect of the drift ice / snow detection device of the present invention further includes a river information acquisition unit that acquires river information such as a flow rate or a flow rate of a river or the like flowing into the detection location of the drift ice or snow, or the detection location, The processing means performs predetermined image processing on the image data obtained by the imaging control means based on the river information obtained by the river information obtaining means.

  According to another aspect of the drift ice / snow detection device of the present invention, the image processing means performs exclusion replacement out of the brightness range, smoothing filter, level adjustment, gray scale conversion on the image data obtained by the imaging control means. , RGB / HSV conversion, tone adjustment, histogram adjustment, binarization processing, and noise reduction processing.

  Further, in another aspect of the drift ice / snow detection device according to the present invention, the transmission control means includes at least one of image data obtained by the imaging control means and image data subjected to image processing by the image processing means. Is transmitted to another information processing device together with the ratio excess duration time.

  Further, one aspect of the drift ice / snow detection method of the present invention is a lighting control step of controlling lighting of a water surface requiring detection of drift ice / snow by a light source in a visible region or a light source in an invisible region, and control by processing of the lighting control step. An imaging control step of controlling imaging by the imaging device of the area irradiated with the illuminated light, and performing predetermined image processing on image data obtained by controlling the imaging by the processing of the imaging control step, to perform drift ice and snow Based on the image processing step of extracting a portion corresponding to and the data obtained by performing the image processing in the processing of the image processing step, the imaging ratio of drift ice and snow to the entire image or a selected or separated area is calculated. A detection result determining step of calculating a ratio excess duration when the imaging ratio of drift ice and snow is higher than a predetermined threshold; and a processing of the detection result determining step. Ratios exceeded the duration obtained by, characterized in that it comprises a transmission control step of controlling the transmission to another information processing apparatus.

  One aspect of the program according to the present invention includes an illumination control step of controlling illumination of a water surface in which drift ice and snow need to be detected by a light source in a visible region or a light source in an invisible region, and illumination controlled by the processing of the illumination control step. An image capturing control step of controlling the image capturing of the area irradiated with the image by the image capturing apparatus, and predetermined image processing is performed on the image data obtained by controlling the image capturing by the processing of the image capturing control step, to deal with drift ice and snow. Based on the image processing step of extracting a part and the data obtained by performing the image processing by the processing of the image processing step, the imaging ratio of the drift ice and snow to the entire image or the selected or separated area is calculated, and the drift ice and snow are calculated. A detection result determining step of calculating a ratio excess duration when the imaging ratio of the image is higher than a predetermined threshold; and Ri resulting ratio of excess duration, characterized in that to execute a processing including a transmission control step of controlling the transmission to another information processing apparatus to the computer.

  Further, one aspect of the drift ice / snow detection system of the present invention includes illumination for irradiating light using a light source in a visible region or a light source in an invisible region with respect to a water surface where detection of drift ice and snow is required, and irradiation with light by the illumination. Imaging device that captures an image of a captured area, drift ice / snow detection device that detects drift ice / snow based on image data captured by the imaging device, and communication that transmits a detection result obtained by the drift ice / snow detection device to another information processing device. An apparatus for detecting drift ice and snow, an illumination control means for controlling the illumination, an imaging control means for controlling the imaging by the imaging apparatus, and a predetermined image data obtained by controlling the imaging by the imaging control means. Image processing means for extracting a portion corresponding to drift ice and snow by performing image processing; and, based on data obtained by performing image processing by the image processing means, the entire image, or selected or separated A detection result determining unit that calculates an imaging ratio of drift ice and snow with respect to the region and calculates a ratio excess duration when the imaging ratio of drift ice and snow is higher than a predetermined threshold value, and a communication device, which is obtained by processing of the detection result determining unit. Transmission control means for controlling transmission of the ratio excess duration to another information processing apparatus.

  According to the present invention, it is possible to accurately calculate the amount of drift ice and snow and the duration time since the occurrence.

It is a figure showing the outline of drift ice snow detection system 1. FIG. 2 is a diagram for describing imaging of drift ice and snow by an area camera 11. FIG. 2 is a diagram for describing imaging of drift ice and snow by an area camera 11. FIG. 4 is a diagram for describing irradiation of light to an imaging location by illumination 12. FIG. 3 is a functional block diagram illustrating functions of an image processing unit 28. FIG. 3 is a diagram illustrating an example of image data captured by an area camera 11. FIG. 5 is a diagram for describing an image processing range a specified by a user. FIG. 4 is a diagram illustrating an example of an image in which a relatively small component of drift ice and snow is hidden by snowfall. It is a figure which shows an example when the component of snowfall is detected as drift ice and snow. It is a figure which shows an example of the image from which the component of snowfall was removed and only the component of drift ice and snow was extracted. It is a figure showing an example when only components of drift ice and snow are extracted. It is a figure which shows an example of the image containing the part of freezing and snowfall, and the drift ice snow part. It is a figure which shows an example at the time of performing general image processing with respect to the image containing the part of freezing or snow, and the part of drift ice and snow. It is a figure showing an example when only a part b corresponding to drift ice and snow which is a moving part is extracted. 4 is a flowchart for describing a process executed by the drift ice detection system 1; 5 is a flowchart for describing image processing. It is a flowchart for demonstrating the separation process of a stationary part and a moving part.

  Hereinafter, a drift ice / snow detection device, a drift ice / snow detection method, a drift ice / snow detection system, and a program according to an embodiment of the present invention will be described with reference to FIGS. 1 to 17.

  An ice drifting snow detection system 1 according to one embodiment of the present invention will be described with reference to FIG.

  The drift ice / snow detection system 1 includes an area camera 11, a lighting 12, a drift ice / snow detection device 13, and a communication device 14.

  The area camera 11 is provided singly or in plurals in the vicinity of the place where the detection of the drift ice and snow is required, and based on the control of the drift ice and snow detector 13 described later, the water surface at the position where the detection of the drift ice and snow is required. Or a continuous still image is taken and supplied to the drift ice / snow detector 13. The area camera 11 corresponds to an imaging device or an imaging unit.

  The illumination 12 irradiates light to a water surface position imaged by the area camera 11 based on the control of the drift ice / snow detection device 13 described later. The lighting 12 is provided singly or plurally in the vicinity of the imaging surface of the area camera 11. The light source of the illumination 12 may be a light source in a visible region such as a white light source or a light source in an invisible region such as infrared light, and is switched and used under the control of the drift ice detection device 13. It may be something.

  The area camera 11 and the illuminating unit 12 may be a waterway that needs to detect drift ice and snow, such as a waterway to a hydroelectric power plant in a cold region, an outlet from a waterway to a dam lake, a waterway to a dam lake, or a reservoir. The area camera 11 captures an image of the water surface of the camera, and the illumination 12 is installed so that light can be emitted to the image capturing position.

  With reference to FIGS. 2 to 4, a description will be given of the imaging of drift ice and snow by the area camera 11 and the irradiation of light to the imaging location by the illumination 12.

  As shown in FIGS. 2 and 3, the area camera 11 is installed such that the water surface at a position where drift ice and snow are to be detected, such as a water channel or a reservoir, is an imaging range. The area camera 11 is installed at a position above a water surface such as a water channel or a reservoir, where a sufficient imaging range can be obtained even when the water level increases.

  Further, as shown in FIG. 4, the illumination 12 is installed so as to be able to irradiate the water surface including the imaging range of the area camera 11 described with reference to FIGS. The illumination 12 is installed at a position above a water surface such as a water channel or a reservoir, where a sufficient irradiation range can be obtained even when water is increased.

  Returning to FIG. 1, the drift ice / snow detecting device 13 detects the drift ice / snow by acquiring an image taken by the area camera 11 and performing image processing. In addition, when the drift ice / snow detection device 13 detects the drift ice / snow as a result of detection of the drift ice / snow, if the imaging ratio is higher than a predetermined threshold, the drift ice / snow detecting device 13 calculates the elapsed time since the detection of the drift ice / snow having a predetermined threshold or more. The information is transmitted to an external information processing device via the device 14.

  The drift ice / snow detector 13 includes an operation input unit 21, a camera control unit 22, a lighting control unit 23, a river information acquisition unit 24, a weather information acquisition unit 25, a sunshine information acquisition unit 26, a detection processing control unit 27, and an image processing unit 28. , A detection result determination unit 29 and a communication control unit 30.

  The operation input unit 21 includes, for example, an input device such as a keyboard and a touch panel, or a receiving unit that acquires information from another device. 28 and the detection result determination unit 29. Specifically, for example, the operation input unit 21 includes, within the imaging range of the area camera 11, a portion that is known in advance that there is no need to detect drift ice or snow, such as a river shore or a wall of a dam lake. When only a part of the image captured by the area camera 11 is used for detection of drift ice or snow, for example, when it is present, an operation input indicating the detection range is received. In addition, for example, when the control of the area camera 11 and the lighting 12 is manually performed, the operation input unit 21 receives an operation input for controlling them. The operation input unit 21 receives an input of control information in image processing executed in an image processing unit 28 described later, for example. In addition, the operation input unit 21 receives input of information such as a threshold value of an imaging ratio for an entire image corresponding to drift ice and snow or a selected or separated area in a detection result determination unit 29 described later. The operation input unit 21 supplies the information to the corresponding units of the detection processing control unit 27, the image processing unit 28, or the detection result determination unit 29.

  The camera control unit 22 controls imaging by the area camera 11 based on the control of the detection processing control unit 27 described later. Specifically, the camera control unit 22 controls the focus and the exposure of the area camera 11 based on the control of the detection processing control unit 27 described later. When the area camera 11 is installed so that the imaging direction can be changed, the camera control unit 22 controls the imaging location of the area camera 11 based on the control of the detection processing control unit 27 described later. May be made. The camera control unit 22 acquires an image captured by the area camera 11 and supplies the image to the image processing unit 28 and the communication control unit 30. The camera control unit 22 corresponds to an imaging control unit.

  The illumination control unit 23 controls the illumination 12 to irradiate light to the imaging location based on the control of the detection processing control unit 27 described later. Specifically, the illumination control unit 23 is configured to be able to change the intensity, color, and the like of the illumination light of the illumination 12, and that the illumination 12 has a light source in a visible region such as a white light source, When it is possible to switch between light sources in an invisible region such as infrared light, these are controlled based on the control of the detection processing control unit 27 described later.

  The river information obtaining unit 24 obtains, via a predetermined network such as the Internet, river information such as the amount of flow or flow velocity of a river or the like flowing into the detection location of drift ice or snow, or the detection location, and a detection processing control unit. 27.

  The weather information acquisition unit 25 acquires and detects weather information such as weather, sunrise, sunset time, temperature, snowfall amount, and weather forecast information of a drift ice / snow detection location via a predetermined network such as the Internet, for example. It is supplied to the processing control unit 27.

  The sunshine information obtaining unit 26 obtains sunshine information indicating whether the detection location is daytime or nighttime, or the time when the sunrise or sunset irradiates when detecting drift ice and snow, and performs detection processing control. To the unit 27. The sunshine information acquisition unit 26 may have a timer or the like therein and calculate sunshine information by itself, or may acquire sunshine information via a predetermined network such as the Internet. Good.

  The detection processing control section 27 controls the operation of each section of the drift ice detection device 13. Further, the detection processing control unit 27 can control each unit of the drift ice detection device 13 based on a command from the operation input unit 21. The detection processing control unit 27 includes information on the amount of water and the flow velocity of the river and the like supplied from the river information acquisition unit 24, and the weather, the temperature, and the weather of the detection location of the drift ice and snow supplied from the weather information acquisition unit 25. Information and weather forecast information, and, based on the sunshine information supplied from the sunshine information acquisition unit 26, the current environment of the location of the detection of drift ice and snow, specifically, the brightness and the position of the water surface of the location of detection of drift ice and snow, When there is sunshine, the illumination control unit 23 is controlled by detecting a change in the color of the reflected light of drift ice and snow due to the direction of sunshine, sunrise, sunset, and the like, and irradiates the imaging location with light suitable for imaging. At the same time, the camera control unit 22 is controlled to control the focus and the exposure of the area camera 11. For example, when the illumination 12 can change the color of the illumination light or the irradiation amount of the light, the detection processing control unit 27 controls the illumination control unit 23 based on the current brightness of the detection location of drift ice and snow, and It is preferable that the color, irradiation amount, and the like of light can be controlled. In addition, when the light source of the illumination 12 can be switched between a light source in a visible region such as a white light source and a light source in an invisible region such as infrared light, the detection processing control unit 27 It is preferable that the section 22 be controlled so that the imaging condition of the area camera 11 matches the state of the illumination 12. When the camera control unit 22 can handle only the control of the area camera 11, the control of the illumination 12 by the illumination control unit 23 does not need to be particularly performed.

  Further, the detection processing control unit 27 includes information such as a water amount and a flow velocity of a river or the like supplied from the river information acquisition unit 24, a weather, a temperature, a snowfall amount, and the like at a detection location of drift ice and snow supplied from the weather information acquisition unit 25. Weather information and weather forecast information, and the current environment of the drift ice / snow detection location obtained based on the sunshine information supplied from the sunshine information acquisition unit 26, and the illumination control unit 23 and the camera control unit executed thereby. Based on the contents of control to the control unit 22, the image processing by the image processing unit 28 described later is controlled.

  In addition, the detection processing control unit 27 includes, in the imaging range of the area camera 11, a portion that is known in advance that there is no need to detect drift ice or snow, such as a river shore or a wall of a dam lake. For example, when only a part of the image captured by the area camera 11 is used for detecting drift ice and snow based on the operation input by the user supplied from the operation input unit 21, the detection range is set to the image processing unit 28. Supply information to indicate. Further, when receiving from the operation input unit 21 a threshold value used for image processing and an operation input for designating the type of image processing, the detection processing control unit 27 causes the image processing to be executed based on these operation inputs. Next, the image processing unit 28 is controlled. In addition, the detection processing control unit 27 detects information such as the threshold of the imaging ratio for the entire image of the part corresponding to drift ice and snow or the selected or separated part, which is supplied from the operation input unit 21. This is supplied to the determination unit 29 to control the detection result determination process. Further, for example, when the control of the area camera 11 and the lighting 12 is manually performed, the detection processing control unit 27 sends the camera control unit 22 or the lighting control unit 23 based on the operation input supplied from the operation input unit 21. Control.

  The image processing unit 28 performs image processing on image data captured by the area camera 11 and supplied from the camera control unit 22 based on the control of the detection processing control unit 27, and determines that the image is drift ice or snow from the captured image. The extracted part is extracted, and the extraction result is supplied to the detection result determination unit 29. The image processing unit 28 performs imaging based on whether the illumination 12 is a light source in a visible region such as a white light source or a light source in an invisible region such as infrared light based on the control of the detection processing control unit 27. In the image data supplied and supplied, it is determined whether the drift ice or snow is determined based on the difference in the reflectance of the reflected light or the difference in the absorptance of the absorbed light, and image processing suitable for the determination is performed. By selecting and applying, a portion determined to be drift ice and snow is extracted, and the extraction result is supplied to the detection result determination unit 29.

  FIG. 5 is a functional block diagram illustrating functions of the image processing unit 28.

  The image processing unit 28 includes an image information acquisition unit 41, a specified table storage unit 42, a specified table processing unit 43, an image synthesis processing unit 44, a noise reduction processing unit 45, a front image storage unit 46, a stationary unit / moving unit separation process. And a contour extraction processing unit 48.

  The image information acquisition unit 41 acquires image data captured by the area camera 11 supplied from the camera control unit 22 and supplies the acquired image data to the designated table processing unit 43. FIG. 6 shows an example of image data acquired by the image information acquisition unit 41 and captured by the area camera 11.

  The designated table storage unit 42 stores a plurality of image processing tables in which the contents, parameters, and the like of the image processing executed by the designated table processing unit 43 are described. The image processing table stored in the designated table storage unit 42 includes, for example, exclusion replacement outside the brightness range, smoothing filter, level adjustment, gray scale conversion, RGB / HSV conversion, tone adjustment, histogram adjustment, and binarization. Processing and general noise reduction processing are included. It goes without saying that the designated table storage unit 42 may store other image processing tables.

  The designated table processing unit 43 performs image processing on the image data supplied from the image information acquisition unit 41 using the image processing table stored in the designated table storage unit 42 based on the control of the detection processing control unit 27. And supplies it to the image synthesis processing unit 44 and the noise reduction processing unit 45. The selection of the image processing table stored in the designated table storage unit 42 and the selection of parameters and thresholds in the image processing are performed at the time of imaging, for example, weather conditions, the state of sunshine such as sunrise or sunset, and the state of rivers. Is determined based on the control of the detection processing control unit 27 according to the state of the imaging position in. In addition, the designated table processing unit 43 determines whether the illumination 12 is a light source in a visible region such as a white light source or a light source in an invisible region such as infrared light based on the control of the detection processing control unit 27. To determine whether the drift ice or snow to be detected is determined by the difference in the reflectance of the reflected light or the difference in the absorptance of the absorbed light, and selects and performs image processing suitable for the determination. .

  In addition, the designated table processing unit 43 can execute image processing only within the range designated by the user, as needed, based on the control of the detection processing control unit 27. For example, as shown in FIG. 7, for example, a portion where it is known in advance that there is no need to detect drift ice or snow, such as snow cover on a river shore or a wall portion of a dam lake, within the imaging range of the area camera 11. Is present, the specified table processing unit 43 is controlled by the detection processing control unit 27 to execute image processing only in the range a specified by the user. Using the stored image processing table, image processing is performed only on the portion shown in FIG. 7A, and supplied to the image synthesis processing unit 44 and the noise reduction processing unit 45.

  The image synthesizing unit 44 performs a synthesizing process on the image data supplied from the designated table processing unit 43 for a predetermined time, and supplies the image data to a noise reduction processing unit 45 described later. Among the reflections other than drift ice and snow, in particular, snowfall and rainfall are imaged by the area camera 11 with the same color and brightness as drift ice and snow, so that it is difficult to distinguish from drift ice and snow. For example, as shown in FIG. 8, when an image in which a relatively small component of drift ice and snow is hidden by snowfall is supplied to the designated table processing unit 43, an image obtained by the processing of the designated table processing unit 43 In the case of, as shown in FIG. 9, there is a possibility that a snowfall portion may be detected in a wide range in the same manner as drift ice and snow. For example, when the image synthesizing processing unit 44 performs the synthesizing process on the image data supplied from the designated table processing unit 43 and shown in FIG. A portion corresponding to snowfall or rainfall, which is a portion that changes instantaneously, can be obtained as a continuous luminance component. The synthesized image generated by the image synthesis processing unit 44 is frequently used in an area where drift ice and snow occur, and is subjected to noise reduction processing in order to remove noise components such as snowfall and rainfall which are difficult to delete by general noise reduction processing. This is used in the unit 45.

  The noise reduction processing unit 45 compares the image data supplied from the designated table processing unit 43 with the composite image data supplied from the image compositing processing unit 44, and among the reflections other than drift ice and snow, A part such as snowfall or rainfall is detected, and these components are deleted and supplied to the front image storage unit 46 and the stationary / moving unit separation processing unit 47. Snowfall and rainfall flow at a higher speed than drift ice and snow from the sky to the water surface under the influence of the current wind direction. That is, in the combined image data supplied from the image combining processing unit 44, the snowfall and the rainfall have a longer continuous brightness than the single image data before the image data of the predetermined time is combined by the image combining processing unit 44. It can be obtained as a component. For example, when the wind direction changes frequently, such as in a snowstorm, the direction of snowfall or rainfall is not always constant, but can be extracted as a continuous luminance component. That is, the noise reduction processing unit 45 compares the image data supplied from the designated table processing unit 43 with the synthesized image data supplied from the image synthesis processing unit 44 to instantaneously instantaneously randomize in any direction. Snowfall or rainfall can be detected as a changing part. For example, when the image data supplied to the designated table processing unit 43 is the image of FIG. 8 and the single image data obtained by the processing of the designated table processing unit 43 is the image of FIG. 9, the noise reduction processing unit 45 Using the image data obtained by synthesizing the image data for a predetermined time by the image synthesizing processing unit 44, only the component of snowfall or rainfall, which is a part that changes instantaneously, is detected. By removing the rainfall component, it is possible to remove the snowfall component and extract only the drift ice component as shown in FIG.

  Similarly, for example, the image data captured by the area camera 11 is shown in FIG. 6, and as described with reference to FIG. 7, control is performed so that image processing is executed only within the range designated by the user. In this case, the components of snowfall included in the image of FIG. 6 are removed by the processing of the noise reduction processing unit 45, so that only the components of drift ice and snow are extracted as shown in FIG. It is supplied to the storage unit 46 and the stationary unit / moving unit separation processing unit 47.

  The preceding image storage unit 46 stores the image data supplied from the noise reduction processing unit 45 for a certain period. For example, as shown in FIG. 12, when drift ice and snow solidify and freeze, or when the snow part spreads over the water surface of a river or a dam lake, the ice part and the snow part and the drift ice snow part are excluded from the brightness range. It is difficult to completely distinguish the image shown in FIG. 13 obtained by general image processing such as replacement, smoothing filter, level adjustment, gray scale conversion, or binarization. However, in the image processing unit 28, it is necessary to correctly distinguish a frozen or snowy portion from a drifting ice and snow portion and detect only the drifting ice and snow portion. The pre-image storage unit 46 is used to separate and separate a frozen or snow-covered portion, which is a stationary portion, and a drift ice / snow portion, which is a moving portion, in the processing of a stationary / moving portion separation processing unit 47 described later. This is for temporarily storing the preceding image to be obtained.

  The stationary part / moving part separation processing unit 47 executes a combining process of the foreground image data stored in the foreground image storage unit 46 within a predetermined time. Then, the stationary part / moving part separation processing unit 47 executes the difference enhancement processing of the composite image, and based on the comparison result of the difference enhancement result and the image data supplied from the noise reduction processing unit 45, based on the comparison result. The moving unit is separated. As described with reference to FIG. 12, in a still image, a frozen or snow-covered portion and a drifting ice-snow portion are excluded from exclusion replacement outside a brightness range, a smoothing filter, a level adjustment, a gray scale conversion, or a binarization. It is difficult to make a complete distinction in the processing of. On the other hand, when the preceding images within a predetermined time are combined, different synthesis results can be obtained from the frozen or snow-covered portion, which is a stationary portion, and the drift ice-snow portion moving with the water flow. The stationary part / moving part separation processing part 47 executes the synthesis processing of the preceding image data within a predetermined time and the difference emphasis processing of the synthesized image, and compares the difference emphasis result with the image data supplied from the noise reduction processing part 45. As shown in FIG. 14, only the portion b corresponding to drift ice and snow, which is a moving portion, is extracted, and the extraction result and image data indicating the extraction result are supplied to the contour extraction processing unit 48 as shown in FIG.

  The contour extraction processing unit 48 executes a process of extracting a contour of a portion corresponding to drift ice and snow extracted by the stationary part / moving part separation processing unit 47, and extracts the extracted contour and image data of the extraction result as a detection result. It is supplied to the judgment unit 29.

  Returning to FIG. 1 again, the detection result judging unit 29 of the drift ice / snow detecting device 13 determines the threshold of the imaging ratio of the drift ice / snow supplied from the detection processing control unit 27 and the extracted value supplied from the contour extraction processing unit 48. The image data of the contour and the extraction result is acquired. Based on the information supplied from the contour extraction processing unit 48 of the image processing unit 28, the detection result determination unit 29 determines whether the entire image, the area selected by the user, or the above-described stationary / moving unit separation processing unit 47 An imaging ratio of drift ice and snow for an area other than the part recognized and separated as the stationary part is calculated. Compare the imaging ratio of drift ice and snow to the whole image, the region selected by the user, or the region other than the region recognized and separated by the above-described stationary part / moving part separation processing unit 47 as the stationary part. Whether to do so is determined based on the control of the detection processing control unit 27 that has received an operation input by the user.

  Then, the detection result determination unit 29 determines whether the calculated imaging ratio of drift ice and snow exceeds the threshold of the imaging ratio of drift ice and snow supplied from the detection processing control unit 27. When the imaging ratio of drift ice and snow exceeds the threshold of the imaging ratio, the detection result determination unit 29 calculates the ratio excess duration. Then, the detection result determination unit 29 supplies the communication control unit 30 with the calculated imaging ratio of drift ice and snow and the ratio excess duration, and the image data of the extraction result supplied from the contour extraction processing unit 48.

  The communication control unit 30 of the drift ice / snow detecting device 13 includes the imaging ratio of the drift ice / snow, the ratio excess duration time, the image data of the extraction result, and the camera control unit supplied from the detection result determining unit 29 of the drift ice / snow detecting device 13. The captured image data supplied from 22 is supplied to the communication device 14 and the transmission of such information by the communication device 14 is controlled.

  The communication device 14 acquires the imaging ratio of the drift ice and snow supplied from the drift ice snow detection device 13 and the ratio excess duration, the image data of the extraction result, and the captured image data, and based on the control of the communication control unit 30, For example, the information is transmitted to an information processing device owned by the user via a predetermined network such as the Internet.

  As described above, the drift ice / snow detector 13 includes the illumination control unit 23 that controls the illumination of the water surface where the detection of drift ice / snow is required by the light source in the visible region or the light source in the invisible region, and the area camera of the area where the illumination is irradiated. 11, an image processing unit 28 that performs predetermined image processing on the captured image data to extract a portion corresponding to drift ice and snow, and a camera control unit 22 that performs image processing. Based on the data, a detection result determining unit that calculates an imaging ratio of drift ice and snow with respect to the entire image or a selected or separated area, and calculates a ratio excess duration when the imaging ratio of drift ice and snow is higher than a predetermined threshold. 29, and a transmission control unit 30 for controlling transmission of the ratio excess duration obtained by the processing of the detection result determination unit 29 to another information processing apparatus, so that the amount of drift ice and snow is generated. It is possible to accurately calculate the duration from occurring.

  In addition, the drift ice snow detection system 1 includes an illumination 12 that irradiates a water surface that needs to detect drift ice and snow with a light source in a visible region or a light source in an invisible region, and an area irradiated with the light by the illumination 12. Camera 11 that captures an image of snow, a drift ice / snow detector 13 that detects drift ice / snow based on image data captured by the area camera 11, and a detection result by the drift ice / snow detector 13 is transmitted to another information processing device. An illumination control unit 23 that includes a communication device 14, wherein the drift ice / snow detecting device 13 controls illumination of a water surface requiring detection of drift ice / snow by a light source in a visible region or a light source in an invisible region, and an area irradiated with the illumination. A camera control unit 22 for controlling the imaging by the area camera 11 and an image for performing predetermined image processing on the captured image data to extract a portion corresponding to drift ice and snow Calculation unit 28, based on the data obtained by the image processing, calculates the imaging ratio of drift ice and snow with respect to the entire image or a selected or separated area, and when the imaging ratio of drift ice and snow is higher than a predetermined threshold And a transmission control unit 30 that controls the transmission of the ratio excess duration obtained by the processing of the detection result determination unit 29 to another information processing apparatus. Therefore, it is possible to accurately calculate the amount of drift ice and snow and the duration time since the occurrence.

  Next, processing executed by the drift ice detection system 1 of FIG. 1 will be described with reference to the flowcharts of FIGS.

  With reference to FIG. 15, the processing executed by the drift ice detection system 1 will be described.

  In step S1, the operation input unit 21 receives a user's operation input, the river information acquisition unit 24 receives river information, the weather information acquisition unit 25 receives weather information and weather forecast information, and the sunshine information acquisition unit 26 receives sunshine information. Acquired and supplied to the detection processing control unit 27.

  In step S2, the detection processing control unit 27 causes the illumination control unit 23 to control the illumination 12 based on the various information acquired in step S1. The illumination 12 irradiates a predetermined position with light based on the control of the illumination control unit 23.

  In step S3, the detection processing control unit 27 controls the camera control unit 22 to control the imaging by the area camera 11 based on the various information acquired in step S1, and acquires the image information captured by the area camera 11. , The image processing unit 28 and the communication control unit 30.

  In step S4, image processing described below with reference to the flowchart of FIG. 16 is executed.

  In step S5, a separation process of the stationary unit and the moving unit, which will be described later with reference to the flowchart of FIG. 17, is performed.

  In step S6, the outline extraction processing unit 48 of the image processing unit 28 detects the outline of drift ice and snow from the image data generated by the processing of steps S4 and S5, and extracts the extracted outline and the image data of the extraction result. , To the detection result determination unit 29.

  In step S7, the detection result determination unit 29 calculates the surface area of the drift ice and snow based on the contour of the drift ice and snow, and calculates the imaging ratio of the drift ice and snow with respect to the entire image or the selected or separated area.

  In step S8, the detection result determination unit 29 calculates the continuation time of the state in which the area of the drift ice and snow exceeds the threshold value, and calculates the imaging ratio of the drift ice and snow and the ratio excess duration, and the contour extraction processing unit. The image data of the extraction result supplied from 48 is supplied to the communication control unit 30.

  In step S <b> 9, the communication control unit 30 supplies the area of the drift ice and snow supplied from the detection result determination unit 29 of the drift ice and snow detection device 13, that is, the imaging ratio of the drift ice and snow to the entire image or the selected or separated area. The continuation time when the area of the drift ice and snow exceeds the threshold, the processed image data, and the captured image data supplied from the camera control unit 22 are supplied to the communication device 14, and the communication device 14 Control the transmission of information. The communication device 14 receives the supply of the information, transmits the information to the information processing device owned by the user via a predetermined network such as the Internet, for example, and ends the process.

  Next, the image processing executed in step S4 in FIG. 15 will be described with reference to the flowchart in FIG.

  In step S21, the image information acquisition unit 41 of the image processing unit 28 acquires the image data supplied from the camera control unit 22 and captured by, for example, the area camera 11 described with reference to FIG. It is supplied to the table processing unit 43. The designated table processing unit 43 acquires a predetermined image processing table from the designated table storage unit 42 based on the control of the detection processing control unit 27.

  In step S22, the designated table processing unit 43 executes image processing based on the image processing table acquired from the designated table storage unit 42 under the control of the detection processing control unit 27. At least a part of the image processing executed by the designated table processing unit 43 is based on whether the illumination 12 is a light source in a visible region such as a white light source or a light source in an invisible region such as infrared light. In the data, drift ice and snow are determined depending on whether the difference is determined by the difference in the reflectance of the reflected light or the difference in the absorptance of the absorbed light. The image processing table stored in the specified table storage unit 42 includes, for example, exclusion replacement outside the brightness range, smoothing filter, level adjustment, gray scale conversion, RGB / HSV conversion, tone adjustment, histogram adjustment, and binarization. Processing and general noise reduction processing are included. It goes without saying that the designated table storage unit 42 may store other image processing tables.

  In step S23, the designated table processing unit 43 determines whether or not the processing of the designated image processing table has been completed. If it is determined in step S23 that the processing of the designated image processing table has not been completed, the processing returns to step S22, and the subsequent processing is repeated.

  If it is determined in step S23 that the processing of the designated image processing table has been completed, the designated table processing unit 43 supplies the processed image data to the image synthesis processing unit 44 and the noise reduction processing unit 45. In addition, the designated table processing unit 43 can execute image processing only within the range designated by the user, as needed, based on the control of the detection processing control unit 27.

  In step S <b> 24, the image synthesis processing unit 44 performs an image synthesis process on the image data supplied from the designated table processing unit 43 within a predetermined time, and supplies the image data to the noise reduction processing unit 45.

  In step S25, the noise reduction processing unit 45 compares the image data supplied from the designated table processing unit 43 with the composite image data supplied from the image compositing processing unit 44, to thereby reduce the reflection other than drift ice and snow. Among them, in particular, a portion such as snowfall or rainfall is detected, these components are subjected to noise reduction as noise, and supplied to the pre-image storage unit 46 and the stationary / moving unit separation processing unit 47.

  In step S26, the front image storage unit 46 stores the noise-reduced image in the front image storage unit 46, and the process proceeds to step S5 in FIG. For example, as described with reference to FIG. 12, when drift ice and snow solidify and freeze, or when the snow part spreads over the water surface of a river or a dam lake, the freeze and snow part and the drift ice snow part are outside the brightness range. It is difficult to completely distinguish the image shown in FIG. 13 obtained by general image processing such as exclusion replacement, smoothing filter, level adjustment, gray scale conversion, or binarization. However, in the image processing unit 28, it is necessary to correctly distinguish a frozen or snowy portion from a drifting ice and snow portion and detect only the drifting ice and snow portion. The pre-image storage unit 46 is used to separate and separate a frozen or snow-covered portion, which is a stationary portion, and a drift ice / snow portion, which is a moving portion, in the processing of a stationary / moving portion separation processing unit 47 described later. This is for temporarily storing the preceding image to be obtained.

  Next, with reference to the flowchart of FIG. 17, the separation process of the stationary unit and the moving unit, which is performed in step S5 of FIG. 15, will be described.

  In step S41, the stationary part / moving part separation processing unit 47 reads out images within a predetermined time of the image data stored in the front image storage unit 46, and executes a synthesis process of these images.

  In step S42, the stationary part / moving part separation processing unit 47 executes a difference emphasis process on the composite image generated in step S41.

  In step S43, the stationary part / moving part separation processing unit 47 performs comparison with the image data supplied from the noise reduction processing unit 45 based on the difference enhancement result obtained in step S41, and based on the result. And a separation process of the stationary unit and the moving unit. When the foreground images within the predetermined time are combined, a different combination result can be obtained from the frozen or snow-covered portion, which is a stationary portion, and the drift ice-snow portion moving with the water flow. The stationary part / moving part separation processing part 47 executes the combining processing of the preceding image data within a predetermined time and the difference emphasis processing of the combined image, and as described with reference to FIG. Only the corresponding portion is extracted, and the extraction result and the image data of the extraction result are supplied to the contour extraction processing unit 48, and the process proceeds to step S6 in FIG.

  With these processes, in detecting the drift ice and snow flowing in the river or the headrace in the cold snowy area, the amount of drift ice and snow and the duration since the occurrence can be accurately calculated.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer built into dedicated hardware or installing various programs. It is installed from a program recording medium into a possible general-purpose personal computer, for example.

  Note that the program executed by the computer may be a program in which processing is performed in chronological order in the order described in this specification, or may be performed in parallel or at a necessary timing such as when a call is made. It may be a program that performs processing.

  Embodiments of the present invention are not limited to the above-described embodiments, and various changes can be made without departing from the spirit of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Drift ice snow detection system, 11 ... Area camera, 12 ... Lighting, 13 ... Drift ice snow detection device, 14 ... Communication device, 21 ... Operation input part, 22 ... Camera control part, 23 ... Lighting control part, 28 ... River information Acquisition units 24, 25: weather information acquisition unit, 26: sunshine information acquisition unit, 27: detection processing control unit, 28: image processing unit, 29: detection result determination unit, 30: communication control unit, 41: image information acquisition unit 42, a designated table storage unit, 43, a designated table processing unit, 44, an image synthesis processing unit, 45, a noise reduction processing unit, 46, a preceding image storage unit, 47, a stationary / moving unit separation processing unit, 48, Contour extraction processing unit

Claims (12)

Light control means for controlling the illumination by the light source in the visible region or the light source in the invisible region with respect to the water surface where the detection of drift ice and snow is required,
Imaging control means for controlling imaging by an imaging device of an area irradiated with the illumination controlled by the illumination control means,
Image processing means for performing predetermined image processing on image data obtained by controlling the imaging by the imaging control means, and extracting a portion corresponding to the drift ice and snow,
Based on the data obtained by performing the image processing by the image processing unit, the entire image, or the imaging ratio of the drift ice and snow with respect to the selected or separated region is calculated, and the imaging ratio of the drift ice and snow is calculated from a predetermined threshold value. A detection result judging means for calculating a ratio excess duration time when it is high;
And a transmission control unit for controlling transmission of the ratio excess duration obtained by the processing of the detection result determination unit to another information processing device. The drift ice snow detection device according to claim 1,
The image processing means determines and separates a stationary part and a moving part based on a synthesis result of image data obtained within a predetermined time, determines that the stationary part is a frozen part, and An apparatus for detecting drift ice and snow, comprising a process of separating a stationary part and a moving part that determines that the drift ice is snow. The drift ice snow detection device according to claim 1 or 2,
The image processing unit is imaged and supplied based on whether the illumination controlled by the illumination control unit is a light source in a visible region such as a white light source or a light source in an invisible region such as infrared light. In the image data, it is determined whether the drift ice or snow is determined based on the difference in the reflectance of the reflected light or the difference in the absorptance of the absorbed light. A drift ice / snow detector that extracts a portion. The drift ice and snow detection device according to any one of claims 1 to 3,
The image processing unit includes a noise reduction processing unit that refers to a synthesis result of image data obtained within a predetermined time, determines a portion detected as an instantaneous change portion as noise, and excludes the portion as noise. Drift ice and snow detector. The drift ice and snow detection device according to any one of claims 1 to 4,
At the time of detection of drift ice and snow, the detection location is day or night, or, further comprises a sunshine information obtaining means for obtaining sunshine information indicating whether it is the time of irradiation of the morning sun or sunset,
The image processing means performs predetermined image processing on the image data obtained by the imaging control means based on the sunshine information obtained by the sunshine information obtaining means. apparatus. It is the drift ice snow detection device according to any one of claims 1 to 5,
Weather information acquisition means for acquiring weather information such as weather of a drift ice snow, sunrise, sunset time, temperature, weather information such as snowfall and weather forecast information,
The image processing unit performs predetermined image processing on the image data obtained by the imaging control unit based on the weather information and the weather forecast information obtained by the weather information obtaining unit. Drift ice and snow detector. The drift ice and snow detection device according to any one of claims 1 to 6,
Drift ice detection point, or, further comprising a river information acquisition means for acquiring river information such as water volume and flow velocity of the river flowing into the detection point,
The image processing means performs predetermined image processing on the image data obtained by the imaging control means based on the river information obtained by the river information obtaining means. apparatus. The drift ice and snow detection device according to any one of claims 1 to 7,
The image processing means, for the image data obtained by the imaging control means, exclusion replacement outside the brightness range, smoothing filter, level adjustment, gray scale conversion, RGB / HSV conversion, tone adjustment, histogram adjustment, A drift ice / snow detection device that performs at least one of a binarization process and a noise reduction process. The drift ice snow detection device according to any one of claims 1 to 8,
The transmission control means, the image data obtained by the imaging control means, and at least one of the image data subjected to image processing by the image processing means, together with the ratio excess duration, the other A drift ice / snow detector that controls processing to be transmitted to an information processing device. An illumination control step for controlling the illumination by a light source in the visible region or a light source in the invisible region on the surface of the water where the detection of drift ice and snow is required,
An imaging control step of controlling imaging by an imaging device of an area irradiated with the illumination controlled by the processing of the illumination control step;
An image processing step of performing a predetermined image processing on image data obtained by controlling the imaging by the processing of the imaging control step, and extracting a portion corresponding to the drift ice and snow,
Based on the data obtained by performing the image processing in the image processing step, the imaging ratio of the drift ice and snow with respect to the entire image or the selected or separated area is calculated, and the imaging ratio of the drift ice and snow is determined to be a predetermined value. A detection result determining step of calculating a ratio excess duration when the value is higher than the threshold value;
A transmission control step of controlling transmission of the ratio excess duration obtained by the processing of the detection result determination step to another information processing apparatus. An illumination control step for controlling the illumination by a light source in the visible region or a light source in the invisible region on the surface of the water where the detection of drift ice and snow is required,
An imaging control step of controlling imaging by an imaging device of an area irradiated with the illumination controlled by the processing of the illumination control step;
An image processing step of performing a predetermined image processing on image data obtained by controlling the imaging by the processing of the imaging control step, and extracting a portion corresponding to the drift ice and snow,
Based on the data obtained by performing the image processing in the image processing step, the imaging ratio of the drift ice and snow with respect to the entire image or the selected or separated area is calculated, and the imaging ratio of the drift ice and snow is determined to be a predetermined value. A detection result determining step of calculating a ratio excess duration when the value is higher than the threshold value;
A program causing a computer to execute a process including a transmission control step of controlling transmission of the ratio excess duration obtained by the detection result determination step to another information processing apparatus. Lighting for illuminating the surface of the water where the detection of drift ice or snow is required using a light source in the visible region or a light source in the invisible region,
An imaging device for imaging an area irradiated with light by the illumination,
A drift ice / snow detector that detects the drift ice / snow based on image data captured by the imaging device;
A communication device that transmits a detection result by the drift ice snow detection device to another information processing device,
The drift ice snow detection device,
Lighting control means for controlling the lighting,
Imaging control means for controlling imaging by the imaging device;
Image processing means for performing predetermined image processing on image data obtained by controlling the imaging by the imaging control means, and extracting a portion corresponding to the drift ice and snow,
Based on the data obtained by performing the image processing by the image processing unit, the imaging ratio of the drift ice and snow with respect to the entire image or the selected or separated region is calculated, and the imaging ratio of the drift ice and snow is calculated from a predetermined threshold value. A detection result judging means for calculating a ratio excess duration time when it is high;
A transmission control unit for controlling transmission of the ratio excess duration obtained by the processing of the detection result determining unit to the other information processing device by the communication device.