JPH11191191A - Waterway inflow/outflow object monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a waterway inflow/outflow object monitoring device with which detection/inflow and outflow degree measurement of jellyfish to flow in/out of a waterway can be performed through a simple image processing method. SOLUTION: Applied image data are sent through processings in the order of a mean value reducing processing part 4, a spatial filtering processing part 5 and a binarizing processing part 6, wherein a jellyfish-like part with remarkable differenced in gradation value from surrounding pixels is emphasized, a binary image having the relevant part of a white pixel is generated, and jellyfish candidates are extracted by an object candidate detecting processing part 7. At an inter-image differential processing part 10, a difference between two images of different acquisition time is obtained, that difference is applied through a differential image reducing processing part 11 to a detection-exception area setting part 12, and part of reflected sunlight having a large differential value is obtained. Based on the outputs from both the processing parts 7 and 12, a detection exclusion processing part 8 extracts a jellyfish candidate located outside the area of reflected light and when it is matched with fixed conditions, a prescribed alarm is issued by operating an alarm processing part 9.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a waterway inflow / outflow monitoring device, and more specifically, to monitoring objects flowing into and out of a waterway (mainly jellyfish flowing into a cooling water intake channel). Also, the present invention relates to an apparatus for detecting a target object by image processing from a video of a monitoring camera and measuring the flow-in / out degree.

[0002]

BACKGROUND OF THE INVENTION When a large number of jellyfish enter a waterway, the waterway is clogged and normal water intake processing at an intake port or the like cannot be performed. Therefore, conventionally, a waterway is imaged with a monitoring camera, and the imaged video signal is output to a monitor. Then, an observer monitors the monitor and, if the jellyfish has invaded, works to remove the jellyfish.

[0003] In the conventional monitoring method described above, however, it is necessary to constantly monitor the monitor in order to detect the occurrence of jellyfish without delay, but in actuality, the monitor must perform other processing and work. You only have to watch the monitor at regular intervals. As a result, the jellyfish removal operation may be delayed. Further, what can be seen from the monitor is the number of jellyfishes at that time, and it is not known whether the number of jellyfishes is increasing or decreasing. Furthermore, the speed of the increase or decrease is not known. As a result, it is difficult to determine whether or not it is necessary to perform the jellyfish removal work, and to determine whether or not it is necessary to perform the work immediately.

Therefore, predetermined image processing is performed on image information captured by the surveillance camera to extract a jellyfish portion, to count the number of jellyfishes occurring on the screen, and to determine the change in the number of jellyfishes with the lapse of time. By detecting the history, it is conceivable to make a correct decision on the presence or absence of jellyfish removal work and its urgency. Then, the following two methods can be considered as those utilizing general image processing.

** First method A moving object or a state change is extracted as an image change region by a difference process between images, and whether or not jelly is determined based on the characteristics (size, shape, number, etc.) of the extracted change region. The number of jellyfishes present in the image can be obtained by judging and finally counting the judgements.

That is, the difference between the image data obtained by A / D converting the gray value of the image obtained from the surveillance TV camera into a digital image and the image data obtained several frames before (the difference between frames) Take). Then, since the background portion that has not moved is removed by calculating the difference, the moving jellyfish or the like is eventually extracted.

Therefore, by performing binarization processing on the gray value of each pixel of the obtained difference image by comparing it with a set threshold value, a change area and a non-change area are discriminated from each other. Extract a jellyfish part. Finally, it is determined from the characteristics (size, shape, etc.) of the obtained change area whether or not the jellyfish is jellyfish.

Further, instead of the inter-frame difference, a method of obtaining a difference image between the background image and the input image (background difference) by previously imaging the background portion may be used.

However, in this method, if the background portion is not fixed when it is converted into a grayscale image, a part of the background portion is also extracted as a difference image, and there is a risk of erroneous detection. In particular, since the background image of the monitoring target region of the present invention is a liquid surface such as a sea surface or a water surface, the background image is constantly changing, so that the above-described problem is conspicuous.

Furthermore, since the object to be detected is a jellyfish floating on the sea or water surface, its moving speed is not constant.
Sometimes they stagnate in the same place. Then, when calculating the inter-frame difference, it is premised that the object moves by a fixed amount in the time between the two images, so that the moving speed of the jellyfish, which is the object, is slow, When the positions partially overlap with each other, the difference value of the portion is suppressed to a low value, and a shape loss at the time of binarization occurs, which may not be able to be extracted.

** Second method A method of extracting a detection target area by binarizing the obtained image itself and applying a method of judging the state based on the characteristics (size, shape, number, etc.) of the area is applied. It is possible to do.

That is, an image obtained from a surveillance TV camera is A / D converted to a digital image, and
A binarization process is performed on the grayscale value of each pixel of the obtained digital image to separate the grayscale value into a detection area and a non-detection area by comparing with a set threshold value. Finally, the characteristics (size, size, etc.) of the detection target area obtained from the binary image
From the shape, etc.), judge whether the jellyfish is or not and count the number of the jellyfish.

By the way, in order to perform accurate discrimination by this method, it is important to determine a threshold value for binarization. However, the background image of the monitoring target area of the present invention is
Since the image is on the water surface, there is a large gradient or unevenness in the gradation value in one screen. As a result, if the entire screen is binarized with one threshold, an area that cannot be binarized properly occurs, and jellyfish may not be extracted correctly.

In the case where the gradation value is uneven in one entire screen as described above, it is conceivable to apply the method by a method of automatically setting a threshold value according to a portion of an image. However, when the gray level of an image fluctuates with time, such as the sea surface / water surface, it is necessary to appropriately set a threshold value accordingly, and it is obtained in order to cope with the time fluctuation of the image. For each image, the process of setting the threshold value according to the part of the image must be performed,
The processing becomes enormous and is not suitable for real-time processing for monitoring.

In both the first and second methods, since one binarization is performed on a pixel-by-pixel basis, there is a case where one detection target is divided and extracted, and subsequent integration processing is not performed well. In addition, there is a problem that the measurement of the number of objects becomes inaccurate.

Therefore, in order to solve the above problems,
The present applicant previously disclosed in Japanese Patent Application No. Hei 8-224677 a waterway inflow / inflow monitoring device capable of detecting jellyfish flowing into and out of a waterway and measuring the degree of inflow and outflow by a simple image processing method. Proposed. Briefly describing such a prior invention, it has a block configuration as shown in FIG. The illustrated block diagram is not the same as the drawing attached to the specification of the prior application, but is different for convenience in comparison with the present invention, but is substantially the same.

As shown in FIG. 1, a gray scale image (shade image data) obtained from the surveillance TV camera 1 is converted to an A / D converter 2.
Is converted into a digital image. Next, in the frame integration / averaging processing unit 3, in order to reduce a noise component in a signal that interferes with detection of an object in a subsequent process and fine unevenness of an image due to a wave of a liquid surface, the frame integration / averaging processing unit 3 performs Perform averaging. That is, the grayscale values of the same coordinates are integrated and averaged using the sequentially input images.

The image data obtained from the frame integration / averaging processing section 3 is subjected to an average value reduction processing in a predetermined block size in an average value reduction processing section 4. The average value reduction process divides an image by a predetermined block size, and uses the average value of the pixel grayscale values in the block as a representative value of the block. If the block size is N × N, the number of data Is compressed to 1 / N ² .

Subsequently, the image data obtained from the average value reduction processing section 4 is subjected to a spatial filter processing in a spatial filter processing section 5. That is, a pixel having a large difference in gray value from surrounding pixels is emphasized. Therefore, the jellyfish part rises.

In the image binarization processing section 6, data obtained by excluding a preset non-detection area from the image data obtained from the spatial filter processing section 5 is set to a threshold value of 2
By binarizing (eg, detecting pixels 1 and non-detecting pixels 0), the image is distinguished into a detection area and a non-detection area.

The object detection processing section 7 'uses the image binary data obtained from the image binarization processing section 6 to measure the number of detected objects in the image. That is, the number of jellyfishes can be measured by performing a labeling process on the binary image data. Then, in the alarm processing unit 9 ', the value of the number of detections by the object detection processing unit 7' and its time change,
The degree of inflow of the detected object is determined, and an alarm is issued accordingly.

In the above-mentioned invention of the prior application, a portion having a higher or lower gray value than the background liquid surface is extracted as an object, and the number thereof is counted to determine the degree of inflow. Therefore, the inflow (jellyfish) can be detected by a relatively simple image processing algorithm, and the initial purpose of automatically monitoring can be achieved. However, when further research is performed, the liquid surface shines and becomes white due to the reflection of sunlight on the liquid surface. Also, the liquid surface is affected by the flow of the wind or water, and generates ripples. Then, it has been found that the liquid surface on which the ripples are generated has minute irregularities, so that it may be irregularly reflected and become whitish.

Therefore, in the above-mentioned invention of the prior application, there is a possibility that a portion of reflected light which appears relatively white when captured as a grayscale image is also extracted as a detection pixel. Since a distinction such as jellyfish / reflected light is not detected and detected, there is a possibility that even if there is reflected light or the like, it may be counted as jellyfish. Then, for example, if the object to be detected is jellyfish, and the detected object includes an object that detects reflected light, the detection accuracy is reduced, and if the amount is large, a false alarm is caused.

The present invention has been made in view of the above-mentioned background, and has as its object to solve the above-mentioned problems and to detect an object (mainly jellyfish) flowing into and out of a water channel from an image of a surveillance camera. And the measurement of the inflow / outflow degree can be realized by a simple image processing method, and furthermore, erroneous detection due to reflection of liquid surface such as sunlight can be reduced, and an inflow / inflow substance monitor capable of reliably detecting an object to be detected. An apparatus is provided.

[0025]

In order to achieve the above object, a waterway inflow / outflow monitoring device according to the present invention sequentially receives image data based on a video image obtained by imaging a waterway to be monitored. Frame integration and averaging processing means for obtaining an averaged image from the image data of a predetermined number of times, and a block size set for the averaged image output from the frame integration and averaging processing means, Average value reduction processing means for generating a reduced image by obtaining a gray value of the block based on the gray value;
Spatial filter processing means for performing predetermined spatial filter processing on the image data subjected to the average value reduction processing so as to emphasize a processing target pixel having a large difference in gray value from surrounding neighboring pixels, and the spatial filter processing means It is assumed that an image binarization processing means is provided for binarizing image data output from the image processing unit with a set threshold value and discriminating between a detection area and a non-detection area. Further, a means for detecting a portion of the image data based on the video image obtained by imaging the waterway to be monitored, in which the time change of the grayscale value is equal to or more than a certain value (in the embodiment, the image difference processing unit 10 and a difference image reduction processing unit 11), a detection exclusion area setting unit for obtaining the portion detected by the detection unit and a peripheral portion thereof, and setting a detection exclusion region, and the set detection exclusion region. Detection means for detecting a detection target based on the binary image obtained by the image binarization processing means (in the embodiment, “target candidate detection processing unit 7, detection exclusion processing unit 8”, (Corresponding to the "image synthesis processing unit 20, the object detection processing unit 21") and alarm means for outputting an alarm based on the detection result of the detection means (claim 1).

The means for detecting a portion where the time change of the gray value is equal to or more than a certain value includes a difference processing means for performing an inter-image difference process, and can detect a portion where the difference value is equal to or more than a certain value. Claim 2).

The frame integration and averaging processing means includes a first frame memory for storing given image data and a second frame memory for storing integrated image data obtained by integrating the given image. And a third frame memory for storing an averaged image obtained by dividing the integrated image data stored in the second frame memory by the number of times of integration, wherein the difference processing means is based on the video image. A fourth frame memory for storing image data; an image data stored in the first frame memory;
It can be configured to include means for performing a difference process based on the image data stored in the frame memory, and a fifth frame memory for storing a difference image obtained by the difference process (claim 3).

As another method, the frame integration and averaging means includes a first frame memory for storing given image data, and an integrated image data obtained by integrating the given image. A second frame memory for storing the image data, and a third frame memory for storing an averaged image obtained by dividing the integrated image data stored in the second frame memory by the number of times of integration. Means for performing a difference process based on the image data stored in the first frame memory and the image data stored in the third frame memory; and a frame memory for storing a difference image obtained by the difference process. (Claim 4).

Here, the water channel includes not only a cooling water intake channel, but also a natural river or an artificial water channel having a relatively small width through which water flows, and a relatively wide portion such as a dam or a pond. Including. Then, the detection target is, in the embodiment,
Although the jellyfish is illustrated, the invention is not limited to this, and a jellyfish that blocks a water channel (for example, garbage such as vinyl) may be used.

According to this configuration, since a predetermined spatial filter process is performed by taking the configuration of the preconditions, an image having a gradient or unevenness of the gray value becomes a relatively flat image. Processing becomes easier. This is true even in the case where the gray level always fluctuates like the liquid surface serving as the background image in the present invention, and furthermore, there is a characteristic of image data in which the gray level in each part is not uniform as a whole and is slightly different. Since the shade value of the jellyfish to be detected is larger (brighter) than that of the background portion, the difference in the shade value at the boundary is large. Therefore, the object to be detected emerges because it is emphasized by performing the spatial filter processing. Therefore, it is possible to reliably discriminate the detection target from the background by the threshold processing.

Further, performing the average value reduction processing as preprocessing of the spatial filter processing not only reduces the processing load by reducing the data amount, but also reduces the possibility that the detection target is divided. The counting of the objects becomes easy.

According to the configuration of the preconditions, an object having the same density (gray level) as the object to be detected can be accurately extracted, but a non-detection object having a relatively similar gray level is also included. May be present. The reflected light from the liquid level of sunlight is detected as a non-detection target object despite being detected.

Therefore, in the present invention, attention is paid to the fact that detection in a portion of the input image in which the gray level changes greatly in a very short time is highly likely to be erroneously detected due to reflection of sunlight on the liquid surface, and this is excluded from the detection. It is characterized by adding processing.

Therefore, by executing a means for detecting a portion where the time change of the gray level value is equal to or more than a certain value or an image difference processing means, the area shining white due to the liquid level reflection of sunlight is extracted. be able to. By setting the detection exclusion area based on the area, even if there is an object having a high gray value in the detection exclusion area, it can be determined that the object is likely to be reflected light, and is removed from the counting of the detection target. . As a result, the target object to be detected can be accurately detected, and an alarm output can be correctly output.

In the third and fourth aspects of the present invention, two frame memories are usually required to store two images to be subjected to difference processing in order to perform the difference processing between images. By using one or two of the frame memories mounted in the frame integration and averaging processing means, the number of frame memories used can be reduced. As a result, it is possible to reduce the size of the system, reduce the number of parts, and reduce costs due to the simplification of the configuration.

[0036]

FIG. 2 shows a first embodiment of a waterway inflow and outflow monitoring device according to the present invention. As shown in the figure, first, a grayscale image obtained from the surveillance TV camera 1 is given to an A / D converter 2 where it is A / D converted and converted into a digital image. Then, the digital image is provided to the frame integration / averaging processing unit 3 in the next stage.

The frame integration / averaging processing unit 3 integrates the grayscale values of the pixels of the same coordinates with respect to the n consecutively input image data and obtains the average value. The average gray value is set as the gray value of the pixel of the image data serving as a reference for the subsequent image processing. This allows
The influence of the noise component in the signal and the fine unevenness of the image due to the wave of the liquid surface are rarely kept on the same coordinates, so that the influence is reduced. This makes it possible to remove noise components that cause erroneous determination such as noise. Further, an average value reduction processing unit 4 is connected to the subsequent stage of the frame integration / averaging processing unit 3, and performs average value reduction processing on the integrated / averaged image data with a set block size. This average value reduction processing is performed by first setting a predetermined N ×
The image is divided into M (N = M is also possible) block sizes, and the average value of the grayscale values of the pixels present in the divided blocks is obtained. Then, the obtained average gray value is used as a representative value of the block.

By performing such processing, for example, if the block size is N × N (N = M), the number of data is 1 /
It can be compressed to N ^2. By compressing in this way, the image is blurred, for example,
The difference in shape and density is less likely to appear, and the number of data to be processed is reduced, so that more accurate judgment can be made in a short time.

As a specific size, since the plane shape of the jellyfish is substantially circular, it is preferable to set N = M. In addition, the length of N pixels and the radius of the jellyfish are substantially the same. It is preferable to set so that Then, in this example, the blocks are divided into 8 × 8 blocks.

The block size at the time of division may be fixed in advance as described above, or a separate block size setting unit may be provided and set based on a control signal from the block size setting unit. You may do so.
The setting may not be performed every time in the detection process, but may be performed periodically or irregularly by manual setting or automatic setting.

Further, the image data generated by the average value reduction processing section 4 is supplied to a spatial filter processing section 5, where a predetermined spatial filter processing is performed so that a jellyfish portion of the detection object emerges. As a specific processing method of the spatial filter, for example, an absolute value of a difference between each grayscale data in an n × n local region centered on a pixel to be processed and a grayscale value of the pixel to be processed is obtained. The maximum value can be selected from the absolute values of the obtained (n × n−1) differences and output as a result of the spatial filter processing.

As an example, if n = 3, FIG.
As shown in (A), the pixels forming the local region are:
, Of which the central pixel is the pixel to be processed. Therefore, | − |, | − |, | −
|,..., | − |, | − |, And the largest value among the total of eight calculation results is set as the representative value of the pixel to be processed. Note that the circled numbers in the above expression of the absolute value calculation mean the grayscale values of each pixel.

With this configuration, the representative value is high when the density difference with surrounding pixels is large, and the representative value is small when the nine pixels including the pixel to be processed have substantially the same gray value. . That is, as described above, the pixel to be processed here has been reduced in the previous stage, and has a size approximately equal to the radius of the jellyfish. Therefore, when the jellyfish exists in the processing target pixel, it exists for four pixels as shown by hatching in FIG.
Therefore, in the illustrated example, the density difference between the five pixels that are not hatched is large. On the other hand, in a portion where there is no jellyfish, the local region is the sea surface or the water surface, so that the density difference between each pixel is small. Therefore, when there is at least a jellyfish in the processing target pixel, the representative value of the processing target pixel increases.

FIG. 4 shows an example of the processing result when the spatial filtering is performed by the above method. In the figure, the place where the gray value (the “representative value” mentioned above) is high is the candidate place where the jellyfish exists.

As another spatial filtering method, focusing on the fact that the gray value of the jellyfish to be detected is higher (brighter) than the gray value of the background liquid level, The purpose is to extract high pixels.

That is, nx centering on the pixel to be processed
The maximum gray value and the minimum gray value are extracted from the gray value data of each pixel existing in the n rectangular area. Then, the maximum grayscale value and the minimum grayscale value are respectively subtracted from the grayscale value of the processing target pixel, and the obtained two subtraction values are added. When the obtained addition value is positive, the value is used as the filter output value, and when the obtained addition value is negative, 0 is used as the filter output value.

The specific processing for the image data can be obtained by the following equation.

[0048]

(Equation 1) In a processing example in the case of a 3 × 3 local area, when a processing target pixel has a detection target, as shown in FIG. 5, the gray value of the processing target pixel becomes the maximum gray value. Therefore, the difference is “0”, and the difference from the minimum shading value is also a large positive value. Therefore, the final addition result becomes equal to the difference from the minimum shading value and becomes a large positive value.

As described above, in the present embodiment, the jellyfish occupies four pixels, but when at least the pixel to be processed has a jellyfish portion, the result is as described above. .

Further, when there is no jellyfish of the object to be detected in the region to be processed (or, of course, there is no pixel to be processed), since the difference between the pixels is small, the gray value of the pixel to be processed, the maximum gray value, and the minimum value The difference from the gray value becomes smaller. In addition, when the pixel to be processed is not the maximum grayscale value, the value obtained by subtracting the maximum grayscale value from the pixel to be processed is negative, so that the final addition result is suppressed by that much. Further, as in the example of FIG. 6, when the gray value of the processing target pixel is closer to the minimum gray value than the maximum gray value, the addition result becomes negative and the filter output becomes 0.

FIG. 7 shows an example of a partial area obtained by executing the above-mentioned method for the same average value reduction processing image used when obtaining FIG. 4. Was. As is clear from the comparison between FIG. 4 and FIG. 7, the portion where the jellyfish exists is steeper, and the value of the background portion is suppressed lower, so that the jellyfish can be easily detected.

The output of the spatial filter processing unit 5 is
It is provided to the image binarization processing unit 6. The image binarization processing unit 6 binarizes the image data obtained from the spatial filter processing unit 5 from the data excluding the predetermined non-detection target area with a set threshold (for example, , The detected pixel (the bright pixel corresponding to the jellyfish) is 1, and the non-detected pixel is 0). In this example, since the spatial filter processing is performed as the preprocessing, the gray value of the jellyfish portion to be detected is increased, and the gray value of the background liquid level is reduced (darkened). Was made larger, so 2
Even if the valuation threshold is set relatively rough, it is possible to reliably extract the jellyfish portion. Then, as a result of performing binarization processing on the grayscale image data resulting from the spatial filter processing as shown in FIG. 7, the result becomes as shown in FIG. It should be noted that a blank portion in the drawing is a candidate portion having jellyfish.

It should be noted that the above-mentioned non-detection target area is a part which is likely to be liable to malfunction, such as an extremely bright part, a dark part, a part having uneven density, or a boundary part between a liquid surface and a wall in the screen. It is set in advance to exclude from. That is, it is specified based on information (coordinate data and the like) regarding the non-detection target area. For the setting of the non-detection target area, for example, the non-detection target area may be directly instructed while looking at the monitor, or an extremely bright place from a gray-scale image (an image without jellyfish) or an area where the density is likely to change. For example, an area that satisfies a predetermined condition such as the above may be extracted and then automatically set.
Then, a function of setting the non-detection target region may be added, but the present invention is not necessarily provided.

Further, the output of the above-mentioned image binarization processing section 6 is given to an object candidate detection processing section 7, and the object candidate detection processing section 7 detects the detection object candidate in a binary image of one frame. Is extracted. That is, in this example, since one detection target object may be detected by a plurality of pixels, a labeling process is performed, and an area where a plurality of pixels are connected is identified as a group of areas. Then, each labeled area is detected, and the coordinate position of the target object candidate is obtained.

The areas extracted by executing the processing units indicated by the above-mentioned reference numerals 1 to 7 are pixel portions (object candidates) which can be estimated to be jellyfish based on the shading. However, the block size is set so that the object is detected in about 1 to 4 blocks for the purpose of simply finding the object candidate. Accordingly, the condition that corresponds to such a condition may be, for example, liquid reflection of sunlight or the like, in addition to the case of jellyfish to be detected. Therefore, in the present embodiment, in addition to the above-described processing device, the following function is further added to detect the liquid-reflection region of the sunlight, thereby preventing erroneous measurement.

That is, as shown in FIG. 2, the output of the A / D converter 2 and the intermediate data in the frame integration / averaging processor 3 are given to the inter-image difference processor 10, where the two images are interpolated. Is performed. By this processing, a difference value of the grayscale value of the pixel at the same position between the two images is calculated, and an image difference image including the absolute value is calculated.

The specific structure of the image difference processing unit 10 can be configured as shown in FIG. That is, as shown in FIG. 3, a first frame memory 3a for storing an input image supplied from the A / D converter 2 has an internal structure for performing the processing of the frame integration / averaging processing section 3 described above. A second frame memory 3b for storing an integrated image obtained by sequentially integrating the images input to the first frame memory 3a, and dividing the integrated image stored in the second frame memory 3b by the number of integrations N. The third frame memories 3c for storing the average images obtained in this way are connected in series, and an adder 3d for performing an integration process and a divider 3e for obtaining an average value are interposed between each frame. It can be realized by a configuration in which the

In this embodiment, the image difference processing unit 1
One fourth frame memory 10a is arranged on the input side of 0, and the input image given from the A / D converter 2 is stored in the fourth frame memory 10a. And
The output of the first frame memory 3a for storing the input image in the above-described frame integration / averaging processing unit 3 is also provided to the inter-image difference processing unit 10, and the output is stored in the first and fourth frame memories 3a and 10a. Each of the input images is supplied to a subtracter 10b, where image difference processing is performed, and the obtained difference image is stored in a fifth frame memory 10c on the output side. Then, the difference image stored in the fifth frame memory 10c on the output side is provided to the difference image reduction processing unit 11 at the next stage. That is, in the sunlight reflection area, the shading value in the same pixel fluctuates relatively largely due to ripples and the like. On the other hand, in the case of the non-reflection area, the gray value takes a substantially constant value. Therefore, it can be said that a pixel having a large difference value by performing the difference processing has a high possibility of being a sunlight reflection area.

The image data provided from each of the frame memories 3a and 10a to the subtractor 10b is, of course, image data obtained at different times. For example, the image data is stored in the fourth frame memory 10a. The image is output after being delayed by a predetermined number of times (for example, two to three frames), or by interposing delay means on a path from the predetermined first frame memory 3a and / or 10a to the subtractor 10b (not shown). it can. Then, as described later, such an inter-image difference process does not need to be performed each time an image is acquired, so that image data input at a certain timing as in the former is stored in the fourth frame memory 10a, and thereafter, This can be easily realized by performing control such that the output is performed after a predetermined period (time / frame) has elapsed.

The difference processing section 10 performs difference processing between images at fixed time intervals (generally known as inter-frame difference processing) using temporally continuous images. This is effective for detecting a temporal change of an image occurring in the image. Generally, when performing the difference processing, two frame memories for storing the target image of the difference between the images,
One frame memory for storing the image of the operation result, a total of 3
Basically, it requires two frame memories. However, in the present invention, as shown in FIG. 9, since one of the target images of the difference between the images uses the image of the frame memory of the frame integration / averaging process, only one frame memory is required for the target image. This is preferable because it can be improved and one can be omitted.

The difference image reduction processing section 11 converts the difference image data from the image difference processing section 10 into an average value reduction processing section 4.
The image is reduced with the predetermined block size used for the image processing. The purpose of the reduction processing here is to make the difference image showing the time change of the image the same size as the average reduction processing image which is the basis of the detection processing. As a specific reduction method, the same processing as that performed by the average value reduction processing unit 4 may be performed. However, the maximum value of the pixel density value in the block (the average value in the case of the average value reduction processing) is used for the block. May be used as the representative value of That is, the liquid-reflection part of sunlight appearing in the difference image often has a high density value but has a fine pattern, and when this is averaged, the difference from the jelly-existing part due to the smoothing effect is reduced. There is a possibility that it will be difficult to distinguish them. In order to prevent this, the latter method using the maximum value is more preferable.

The reduced image obtained by the difference image reduction processing section 11 is provided to the detection exclusion area setting processing section 12 at the next stage. The detection exclusion area setting processing unit 12 performs a process of extracting a reflected portion of sunlight to be excluded from detection targets from the reduced difference image. Specifically, first, the image data from the difference image reduction processing unit is binarized by a predetermined binarization threshold (for example, the detection exclusion pixel is 1 and the non-detection exclusion pixel is 0). Since the detection exclusion pixels are often sparsely extracted, expansion and integration are performed on the pixels by performing expansion processing a predetermined number of times to calculate a target detection exclusion area. This expansion process is a process of repeating a predetermined number of times that a pixel to be processed is a detection exclusion target pixel (1) and surrounding pixels are also set to a detection target exclusion target pixel (1). Therefore, if the number of repetitions is one, up to eight neighboring pixels (3 × 3 local areas) will be a detection exclusion area, and if the number of repetitions is two, a 5 × 5 local area around the detection exclusion pixels will be detected. It becomes an exclusion area.
Of course, it may be repeated three or more times.

Then, the detection exclusion area setting processing section 12
The detection exclusion area data obtained in step (1) and the result obtained by the object detection processing section 7 described above are provided to the detection exclusion processing section 8. The detection exclusion processing unit 8 executes processing for excluding, from among the object candidates by the object detection processing unit 7, those present in the detection exclusion area from the detection exclusion area setting processing unit 12. That is, when the position coordinates of the object candidate are in the detection exclusion area, they are removed. Thereby, erroneous detection due to reflection of sunlight on the liquid surface, which cannot be distinguished in the basic detection processing in the object detection processing unit 7, can be excluded, and the detection accuracy is improved.

Since the number of jellyfishes can be counted with high accuracy in the detection exclusion processing unit 8 as described above, the alarm processing unit 9 detects the number of jellyfishes based on the value of the number of detections from the detection exclusion processing unit 8 and its time change. Is determined and an alarm is issued accordingly.

That is, assuming that the input image is as shown in FIG. 10 and the non-detection area is as shown in FIG. 11, the image binarization processing section 6 outputs a binary image as shown in FIG. Is output. The white part in the figure is the detection target candidate part. The region R in FIG. 10 is a region based on the reflected light, but in FIG. 12, the detection target candidate portion is extracted in a portion corresponding to the region.

On the other hand, after performing the difference processing between the images, the averaging and expansion processing are performed, so that the detection exclusion area setting processing section 12
The detection exclusion area output from is the blacked-out part in FIG. 13, and as is clear from comparison between FIG. 10 and FIG.
The region R based on the reflected light is set as a detection exclusion region.
Therefore, even if the detection target candidate exists in this detection exclusion area, it is ignored. Therefore, when the portion corresponding to the detection exclusion area in the binary image shown in FIG. 12 is replaced with black pixels, the result becomes as shown in FIG. Then, the white pixel portion in FIG. 14 is the portion where the jellyfish which is the detection target exists. Therefore, the number of jellyfish can be counted based on the white pixel area. In addition,
FIG. 14 is shown to explain the basic concept of counting,
Needless to say, there is no need to obtain the actual binary image and count based on it. That is, as described above, the counting can be performed based on the coordinate values obtained by the target object detection processing unit 7 and the detection exclusion area shown in FIG. Of course, as in the other embodiments described later, it is of course also possible to specifically obtain a binary image and count based on the binary image.

FIGS. 15 and 16 show a second embodiment of the present invention. In the present embodiment, the difference between the images is 2
By using the frame integration and averaging image and the final input frame to the frame integration and averaging process for the target images, the target image difference processing can be performed without adding a frame memory for the image difference target image. Can be done.

Specifically, the output from the first frame memory 3a in the frame integration / averaging processing unit 3 and the third frame memory 3c, that is, the frame integration / averaging processing unit 3 Output (averaged image)
And a built-in subtractor 10b executes an inter-image difference process. The obtained difference data is provided to the fifth frame memory 10c, and is provided to the next-stage difference image reduction processing unit 11.

That is, the frame integrated / averaged image is
This is an image obtained by smoothing the temporal change of the image density value, and the difference image between this image and the final frame tends to have a lower difference value than a simple inter-frame difference image. However, since the gray level change itself due to sunlight reflection is large, even with this method, it is possible to obtain a difference value that can be distinguished from the change due to inflow and outflow. Therefore, according to the present embodiment, the number of frame memories can be further reduced by one in comparison with the first embodiment.

FIG. 17 shows a third embodiment of the present invention. This embodiment is based on the first embodiment described above, and is different from the method of obtaining a detection target based on a target candidate and a detection exclusion area. That is, the output of the image binarization processing unit 6 and the output of the detection exclusion area setting processing unit 12 are given to the image synthesis processing unit 20. This image synthesis processing unit 2
0 takes the logical sum of the two given images, and if at least one pixel is a black pixel, makes the pixel black. As a result, in the binary image output from the image binarization processing unit 6, the pixel corresponding to the detection exclusion area becomes a black pixel. Therefore, if the two binary images given to the image synthesis processing unit 20 are respectively shown in FIGS.
FIG. 14 shows the image composition processing unit 20.
That is, the image synthesis processing unit 20 performs the detection exclusion processing.

Then, the output of the image synthesizing processing section 20 is supplied to the object detecting processing section 21, where the given binary image is subjected to labeling processing to integrate continuous white pixels, and an area is set. By counting the number of the areas, the number of detection targets is obtained. When the obtained number is equal to or more than the predetermined number, the alarm processing unit 9 is operated to issue an alarm. Note that such a configuration can of course be applied to the above-described second embodiment.

In each of the embodiments described above, an example is shown in which one or two frame memories for storing the image to be processed in the inter-image difference processing section are omitted, but the present invention is not limited to this. Instead, a frame memory may be provided independently in the inter-image difference processing unit to perform a generally performed inter-image difference process.

[0073]

As described above, in the waterway inflow / outflow material monitoring apparatus according to the present invention, by performing the spatial filter processing, the image in which the gradient or unevenness of the grayscale value is present becomes a relatively flat image. Binarization processing can be easily performed. In addition, since the average value reduction processing is performed as the preprocessing, the amount of data to be processed can be reduced, the processing load is reduced, real-time processing becomes possible, and the block size is appropriately adjusted. By setting, the possibility that the target object is divided is reduced, and the detection target object candidate can be easily and accurately detected.

As described above, by adopting the configuration of the precondition, the object can be reliably extracted. And
The non-detection target is also included in the extraction in this manner. However, in the present invention, by providing the processing means as defined in each claim in addition to the preconditions, the area of the reflected light of sunlight can be detected, so that the detection target can be accurately extracted. In addition, the detection of objects (mainly jellyfish) flowing into and out of the water channel from the images of the surveillance cameras and the measurement of the degree of the inflow and outflow can be realized by a simple image processing method. Can be distinguished with high accuracy.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a waterway inflow and outflow monitoring device according to a prior invention, which was devised in the course of carrying out the present invention.

FIG. 2 is a block diagram showing a first embodiment of a waterway inflow / outflow substance monitoring apparatus according to the present invention.

FIG. 3 is a diagram illustrating a function of a spatial filter.

FIG. 4 is a diagram illustrating an example of a processing result of a spatial filter.

FIG. 5 is a diagram illustrating a function of a spatial filter.

FIG. 6 is a diagram illustrating a function of a spatial filter.

FIG. 7 is a diagram illustrating an example of a processing result of a spatial filter.

FIG. 8 is a diagram illustrating an example of a frame image binarization processing result;

FIG. 9 is a diagram showing a main part of the present embodiment.

FIG. 10 is a diagram illustrating an example of an input image.

FIG. 11 is a diagram illustrating an example of a non-detection target area in the input image.

12 is a diagram illustrating a binary image output from the image binarization processing unit 6 when the input image in FIG. 10 is provided.

FIG. 13 is a diagram showing an image output from the detection exclusion area setting processing unit 12.

14 is a diagram illustrating a binary image obtained by performing a detection exclusion process on the image illustrated in FIG. 12;

FIG. 15 is a block diagram showing a second embodiment of the waterway inflow / outflow substance monitoring apparatus according to the present invention.

FIG. 16 is a diagram showing a main part of the present embodiment.

FIG. 17 is a block diagram showing a third embodiment of the waterway inflow / outflow substance monitoring apparatus according to the present invention.

[Explanation of symbols]

 Reference Signs List 1 camera 2 A / D converter 3 frame integration / averaging processor 4 average reduction processor 5 spatial filter processor 6 image binarization processor 7 target candidate detection processor 8 detection exclusion processor 9 alarm processor DESCRIPTION OF SYMBOLS 10 Image difference processing part 11 Difference image reduction processing part 12 Detection exclusion area setting processing part 20 Image synthesis processing part 21 Object detection processing part

Claims (4)

[Claims] A frame integration and averaging means for sequentially receiving image data based on a video image obtained by imaging a waterway to be monitored and obtaining an averaged image from a predetermined number of image data; Average value reduction processing means for generating a reduced image by dividing an averaged image output from the processing means into a set block size, and obtaining a grayscale value of the block based on grayscale values of pixels in the block And a spatial filter processing means for performing predetermined spatial filter processing on the image data subjected to the average value reduction processing so as to emphasize a processing target pixel having a large difference in gray level between neighboring pixels, and the spatial filter. Image binarization processing means for binarizing image data output from the processing means with a set threshold value to distinguish between a detection area and a non-detection area; Means for detecting a portion in which the time change of the grayscale value is equal to or greater than a predetermined value in image data based on a video image obtained by imaging the waterway to be monitored, and the portion detected by the detecting means and the portion Detection exclusion area setting means for obtaining a peripheral portion and setting a detection exclusion area; detecting a detection target based on the set detection exclusion area and the binary image obtained by the image binarization processing means A waterway inflow / outflow monitoring device, comprising: a detection unit; and an alarm unit that outputs an alarm based on a detection result of the detection unit. 2. The image processing apparatus according to claim 1, wherein the means for detecting a portion where the time change of the gray value is equal to or more than a predetermined value includes a difference processing unit for performing an inter-image difference process, and detects the portion where the difference value is equal to or more than a certain value. 2. The waterway inflow and outflow monitoring device according to 1. 3. The frame integration and averaging processing means includes: a first frame memory for storing given image data; and a second frame memory for storing integrated image data obtained by integrating the given image. And a third frame memory for storing an averaged image obtained by dividing the integrated image data stored in the second frame memory by the number of times of integration, wherein the difference processing means is based on the video image. A fourth frame memory for storing the image data; a means for performing a difference process based on the image data stored in the first frame memory and the image data stored in the fourth frame memory; The inflow / outflow substance monitoring apparatus according to claim 2, further comprising a fifth frame memory that stores the obtained difference image. 4. A first frame memory for storing given image data, and a second frame memory for storing integrated image data obtained by integrating the given images. And a third frame memory for storing an averaged image obtained by dividing the integrated image data stored in the second frame memory by the number of times of integration, wherein the difference processing means is provided in the first frame memory. Means for performing a difference process based on the image data stored in the third frame memory and the image data stored in the third frame memory, and a frame memory for storing a difference image obtained by the difference process. The inflow / outflow material monitoring device according to claim 2, wherein