JPH11211845A - Rainfall/snowfall detecting method and its device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a rainfall/snowfall detecting method capable of automatically detecting rainfall or snowfall based on the image photographed by a photographing means such as a TV camera. SOLUTION: A rainfall/snowfall object region is photographed by a TV camera 101, adjacent frame images are stored in frame image memories 202a, 202b by switching a multiplexer 201 in turn, the sequential differential calculation of the frame images is made by a differential process section 203, and the same moving body is recognized from the photographed image by a moving body judging process section 204 based on the differential result. An area threshold value process is applied by an area threshold value process section 205, only the rainfall/snowfall component is extracted from the moving body, the density square sum or area ratio of the rainfall/snowfall component is calculated by a rainfall/snowfall intensity calculation section 3, and it is outputted as the detected result of the rainfall or snowfall.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a rainfall and snowfall detection method and a rainfall snowfall detection device for automatically detecting rainfall or snowfall from an image photographed by photographing means such as a TV camera.

[0002]

2. Description of the Related Art Conventionally, rainfall or snowfall has been detected by monitoring a monitor screen taken by a monitoring camera such as a TV camera. However, when using a surveillance camera, it is necessary to keep an eye on the monitor screen, which increases the burden on the observer and makes it difficult to make accurate decisions.

[0003]

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problem, and it is possible to automatically detect rainfall or snowfall based on an image photographed by photographing means such as a TV camera. An object of the present invention is to provide a rainfall snow detection method and a rainfall snow detection device.

[0004]

According to a first aspect of the present invention, there is provided a detection method comprising: performing a sequential difference operation between two temporally adjacent images of a rain / snow detection target area; Only image information relating to a moving object is detected from the image, and area threshold processing is performed on the detected moving object image, thereby extracting only the rainfall snow component in the moving object image. With such a configuration, rainfall and snowfall components can be automatically extracted from the captured image. Further, since the area threshold processing is performed, even if a traveling vehicle is present in the photographing screen, it can be excluded and only the rainfall snow component can be extracted.

According to a second aspect of the present invention, only a rainfall snow component is extracted from a photographed image by performing a sequential difference operation between two temporally adjacent images photographing the rainfall snow detection target area. It is. With such a configuration, similar to the detection method of claim 1,
Rainfall or snowfall components can be automatically extracted from the captured image. The detection method according to the second aspect is obtained by omitting area threshold processing from the detection method according to the first aspect, and separates a moving object having a large area, such as a traveling vehicle, existing in a shooting screen. Cannot be excluded. Therefore, it is not suitable for detecting rainfall and snow on a road or the like where a traveling vehicle exists, and is suitable for detecting rainfall and snow in a place where the traveling vehicle or the like does not appear at all.

According to a third aspect of the present invention, there is provided a detection method, wherein a dynamic difference operation is performed between a background image of a rain / snow detection target area and a photographic image obtained by photographing the rain / snow detection target area, so that only image information on a moving object is obtained from the photographic image. Is detected, and only the rainfall snow component is extracted from the moving object image by performing area threshold processing on the detected moving object image. With such a configuration, rainfall and snowfall components can be automatically extracted from the captured image. Further, since the area threshold processing is performed, even if a traveling vehicle is present in the photographing screen, it can be excluded and only the rainfall snow component can be extracted.

According to a fourth aspect of the present invention, only a rain / snow component is extracted from a photographed image by performing a dynamic difference operation between a background image of the rain / snow detection target region and a photographed image of the rain / snow detection target region. It is something to do. With such a configuration, the rainfall or snowfall component can be automatically extracted from the captured image, as in the detection method of the third aspect. The detection method according to the fourth aspect is obtained by omitting the area threshold processing from the detection method according to the third aspect, and separating a moving object having a large area, such as a traveling vehicle, existing in a shooting screen. Cannot be excluded. Therefore, it is not suitable for detecting rainfall and snow on a road or the like where a traveling vehicle exists, and is suitable for detecting rainfall and snow in a place where the traveling vehicle or the like does not appear at all.

According to a fifth aspect of the present invention, there is provided a photographing device for photographing a rainfall / snow detection target region, and a sequential difference operation between two images temporally adjacent to each other or a dynamic difference operation between a photographed image and a background image. Thereafter, image processing means for executing area threshold processing to extract only the rainfall snow component, and calculating the sum of the squares of the concentrations or the area ratio of the extracted rainfall snow component, and outputting this as a rainfall snow detection result And a snow intensity calculation unit. By configuring as such an apparatus, it is possible to accurately calculate the intensity of only the rainfall snow component.

[0009]

Embodiments of the present invention will be described below with reference to the drawings. [First Embodiment] FIG. 1 shows a first embodiment of a rain / snow detector according to the method of the present invention. The first embodiment shows an example in which a technique called a sequential difference method is used for extracting a rainfall snow component. Therefore,
Before describing the first embodiment, the sequential difference method will be described first.

[0010] The sequential difference method is a method of extracting only a moving object from an image by calculating a difference between two temporally adjacent images. FIG. 2 shows the principle of the sequential difference method. FIG. 7A shows a photographed image f (T) at a certain time T, and FIG.
+ T (t is a photographing interval; for example, in the case of a TV camera, 1/30 second of a frame interval) is a photographed image f (T + t), and when the photographed image of (B) is subtracted from the photographed image of (A), When the object M in the image is a moving object, as shown in the difference image of FIG. 11C, both ends of the object have a positive (+) polarity corresponding to the moving distance. A region m and a region m ′ having a negative (−) polarity appear as a pair,
Thus, it is possible to determine that the object is a moving object and extract the object. FIG. 11D shows the difference density values along the line L in FIG. Such a moving object extraction method is referred to as a sequential difference method.

FIG. 2 shows the case of the forward difference Δf {(T) − (T + t)} in which the subsequent image is subtracted from the temporally previous image. Conversely, FIG. Backward difference Δf ｛(T + t) − by subtracting the previous image from the image
In the case of (T)｝, the same operation can be performed only by reversing the positive and negative polarities with respect to the moving direction.

Next, a detailed description will be given of the rainfall / snow detector of FIG. 1 utilizing the sequential difference method. FIG.
Reference numeral 1 denotes a photographing means for photographing the detection target area of rainfall and snow. The TV camera 101 and a high-frequency component of a video signal photographed by the TV camera 101 are removed to reduce noise and prevent aliasing at the time of sampling. Low-pass filter (LP
F) 102 and AD for converting a video signal into a digital signal
And a converter 103. 2 is TV camera 1
Image processing means for extracting only the rainfall snow component which is a moving object from the temporally adjacent captured images taken at 01 based on the sequential difference method, and 3 calculates the intensity of rainfall or snowfall at that time from the extracted rainfall snow component. It is a rainfall snow intensity calculation unit.

The image processing means 2 includes a multiplexer 201 for switching a switch left and right in units of frame images.
And two frame image memories 202a and 202b for alternately storing captured frame images, and a difference processing unit 20 for performing a difference operation between two temporally adjacent frame images stored in the frame image memories 202a and 202b.
3 and based on this sequential difference result (+)
A moving object discrimination processing unit 204 that recognizes the same moving object based on the polarity of (−), and a vehicle or pedestrian with a large area by performing area threshold processing on the recognized moving object that has been subjected to absolute value processing. And an area threshold value processing unit 205 that extracts only snowflakes and raindrops having a small area except for the above.

In the case of detecting rainfall snow on a road where vehicles and pedestrians come and go, traveling vehicles and pedestrians as well as snowflakes and raindrops are extracted as moving objects. Although the value processing unit 205 is required, if the moving object shown on the screen is only snowflakes or raindrops in a place where vehicles and people do not pass at all, the area threshold processing unit 205 can be omitted.

Although the moving body discrimination processing section 204 in the above embodiment is provided for the purpose of recognizing the same moving body and improving the detection accuracy, the polarity itself of (+) (-) is the moving body. Since only the intensity of rainfall snow is detected, the absolute value processing as a pre-process of the area threshold processing is executed by the area threshold processing unit 205. do it. Furthermore, the image processing means 2 and the rain / snow intensity calculation unit 3 may be constituted by a hardware circuit, but are usually constituted by a software circuit using a microcomputer.

Next, the processing operation of the rain / snow detector of FIG. 1 will be described with reference to the flowchart of FIG. T
The video signal of the rain / snow detection target area photographed by the V camera 101 is cut into high-frequency components by a low-pass filter 102, then converted into a digital signal by an AD converter 103, and one frame (1/30 second) by a multiplexer.
Each time they are alternately stored in the two frame image memories 202a and 202b (step S1 in FIG. 3).

The difference processing unit 203 performs the sequential difference calculation shown in FIG. 2 using two temporally adjacent frame images f (T) and f (T + t) stored in the frame image memories 202a and 202b. Then, a moving component having positive and negative polarities in the image is extracted (step S2).
Then, the moving body determination processing unit 404
The moving object is recognized based on the presence or absence of the positive and negative moving components obtained in step 03, and only the same moving object is extracted from the captured image (step S2). FIG. 4 shows a specific image example of the extraction result of the moving object. FIG. 4 shows an example in which a road in a snowfall state is photographed, and a white lump having a large area in the center of the screen is a vehicle running on the road, and is randomly scattered throughout the screen. Each chunk consisting of small white spots is a snowflake.

As described above, when detecting the rain / snow condition in a place where vehicles and people do not pass at all, a running vehicle or a pedestrian does not appear on the screen, but the shooting location is a road or the like. In the case of, a vehicle running on the screen is also extracted as a moving object as shown in FIG. Therefore, in such a case, the next area threshold value processing unit 205 determines that a moving object having a larger area than a certain size is not a snowflake or a raindrop, and excludes these moving objects from the screen. That is, the area threshold value processing unit 205 determines whether the area of each of the white chunks present in the moving object extracted image of FIG. 4 is equal to or larger than a preset threshold value (number of pixels). If the area is equal to or larger than the predetermined area, it is determined that the white mass is not a rainfall snow component but a traveling vehicle or a pedestrian, and is excluded from the screen (step S).
3).

As a method of separating a moving object as a pre-process for executing the area threshold processing, a method based on a labeling process proposed in Japanese Patent Application No. 8-293406 previously filed by the present applicant has been proposed. Etc. may be used. That is, labeling means that for a binarized image obtained by separating a grayscale image into two regions, an extraction target and a background, with an appropriate threshold value, an extraction target region (continuous component) to which “1” is normally given is given. This is an operation of assigning different labels to the pixels that are those elements, and the maximum number value of labeling is that the total number of areas, the number of pixels in the area indicates the area of the area, and the labeling processing is that This is a process of performing labeling on an object by using a propagation method or the like that sequentially propagates information according to a propagation rule defined for a pixel.

The specific value of the area threshold for separating the rainfall snow component having a small area from the vehicle portion having a large area is determined by an appropriate value depending on the photographing place, photographing angle of view, photographing conditions and the like. Needless to say, FIG. 5 shows the result of performing the area threshold processing with the area threshold (number of pixels) = 500 on the moving object extracted image of FIG. 4. As is clear from a comparison between FIG. 5 and FIG.
In FIG. 5 after the area threshold processing, a moving object having a large area such as a vehicle is excluded, and it can be confirmed that only the rainfall snow component is extracted.

The rain / snow component image extracted as described above is sent to the rain / snow intensity calculation unit 3 to calculate the sum of the squares (power) of the density of the pixels of each of the rain / snow components contained in the rain / snow component image. The calculation result is output as the current rainfall intensity or snowfall intensity (step S4).

Photographing the actual snowy field 60
FIG. 6 shows the result of detection of snowfall by the above apparatus using the image data of minutes. In the figure, the horizontal axis represents time, and the vertical axis represents the sum of squares of the concentration of the snowfall component (power). It can be seen that the snowfall component captured within the angle of view of the TV camera fluctuates greatly with time, and that the power fluctuates greatly at times of weak snowfall and strong snowfall. Although the power value is large in some places even at the time when there is no snowfall, this is because a small area such as a vehicle wheel or a window frame cannot be distinguished from a snowfall component. As is clear from FIG. 6, the detection result according to the present invention well represents the actual snowfall state, and its effectiveness was confirmed.

In the example shown in FIG. 1, a frame image is used as a temporally adjacent photographed image. However, an odd field image and an even field image constituting one frame may be used. In addition, the sum of the squares of the concentration is calculated as the detection result of rainfall or snowfall, but the area ratio of the detected rainfall snow component to the entire screen may be calculated and output. Further, as described above, the area threshold value processing unit 205 may be omitted when detecting a rain / snow condition in a place where no vehicles or people pass.

[Second Embodiment] FIG. 7 shows a rainfall and snow detecting apparatus according to a second embodiment of the present invention. The second embodiment shows an example in which a technique called a dynamic difference method is used for extracting a rainfall snow component. Therefore, before describing the second embodiment,
First, the dynamic difference method will be described.

The dynamic difference method is a method of extracting only a moving object from an image by taking a difference between a background image showing only a stationary object and a photographed image. FIG. 8 shows the principle of this dynamic difference method. FIG. 9A shows a photographed image at a certain time, and FIG. 10B shows a background image. When the background image of FIG. 10B is subtracted from the photographed image of FIG. As shown in the difference image, only the moving object existing in the captured image can be extracted.
Such a method of extracting a moving object is called a dynamic difference method. In FIG. 8, the background image is subtracted from the photographed image. Conversely, even if the photographed image is subtracted from the background image, the same operation can be performed only by changing the polarity of the result of the subtraction.

Next, the rainfall / snow detector of FIG. 7 utilizing the dynamic difference method will be described in detail. In addition,
The same parts as those in the first embodiment (FIG. 1) are denoted by the same reference numerals.

In FIG. 7, reference numeral 101 denotes a TV camera, 10
2 is a low-pass filter (LPF), 103 is an AD converter, 2 is an image processing means for extracting only a rainfall snow component which is a moving object based on the dynamic difference method, and 3 is a rainfall snow intensity calculation unit.

In the case of the second embodiment, the image processing means 2 includes a photographed image memory 206a for sequentially storing frame images photographed by the TV camera 101, and a memory for storing changes in weather conditions and changes in luminance during the day and night. A background image memory 206b for storing a background image whose brightness state is followed in accordance therewith, and a difference operation between the photographed image stored in the photographed image memory 206a and the background image stored in the background image memory 206b are performed. The difference processing unit 203 performs area threshold processing on the moving object in the image extracted by the dynamic difference processing, and extracts only small snowflakes and raindrops excluding vehicles and pedestrians having a large area. And an area threshold value processing unit 205. In addition,
As described above, when detecting a rain / snow condition in a place where no vehicles or people pass, the area threshold processing unit 2
05 can be omitted.

Next, the processing operation of the rain / snow detector of FIG. 7 will be described with reference to the flowchart of FIG. Note that the flowchart of the first embodiment described above (FIG. 3)
The same processing parts as those described above are denoted by the same reference numerals.

In the case of the second embodiment, a background image consisting only of a background component having the same luminance as that at the time of photographing is always stored in the background image memory 206b. As a method of automatically updating the background following the change of the background, a method proposed by the present applicant in Japanese Patent Publication No. 7-114494 may be used. That is, by calculating the difference between corresponding pixels of a photographed image photographed at regular time intervals of about several tens of millimeters to several seconds and a background image stored in advance, the amount of change in the luminance value of both is calculated, and the amount of change is calculated. If it is determined that is a change amount due to weather conditions or the like, the change amount of the luminance value is weighted, and then added back to the background image, and the added image is used as a new background image. How to

Next, after the high frequency component of the video signal of the rain / snow detection area photographed by the TV camera 101 is cut by the low-pass filter 102, it is converted into a digital signal by the AD converter 103, and is converted into one frame (1 frame). / 30 seconds) is stored in the captured image memory 206a (step S1 in FIG. 9).

The difference processing unit 203 is provided in the photographed image memory 20
6a and a background image memory 206b
Then, a dynamic difference calculation as shown in FIG. 8 is performed using the background image stored in the image data, and only the moving object is extracted from the image (step S2a). Then, as in the first embodiment, the area threshold processing unit 205 performs the area threshold processing described above, and determines that the moving object having a larger area than a certain size is not a snowflake or a raindrop. Then, these moving objects are excluded from the screen (step S3). As described above, the area threshold processing may be omitted when detecting a rain / snow condition in a place where vehicles and people do not pass at all.

The rain / snow component image extracted as described above is sent to the rain / snow intensity calculation unit 3 to calculate the sum of squared power (power) and the area ratio of each of the rain / snow components contained in the rain / snow component image. The calculation result is output as the rainfall intensity or the snowfall intensity at that time (step S4).

[0034]

As described above, claims 1 and 3
According to the invention, rainfall and snowfall components can be automatically extracted from the photographed image, so that it is not necessary for the observer to keep an eye on the monitor as in the related art, and the burden on the observer can be reduced. It can accurately judge rainfall and snowfall without individual differences, and also performs area threshold processing, so even if there is a running vehicle in the shooting screen, it is excluded and only the rainfall snowfall component is extracted It is possible to accurately detect a rainfall snow condition on a road on which the vehicle travels.

According to the second and fourth aspects of the present invention, since the detection method of the first and third aspects is simplified and the area threshold processing is omitted, the rainfall in a place where the traveling vehicle or the like is not photographed at all can be obtained. It can be used for snow detection to show its excellent effect.

Further, according to the fifth aspect of the present invention, a photographing means for photographing a rain / snow detection target area, a sequential difference calculation between two images temporally adjacent to each other, or a dynamic difference calculation between a photographed image and a background image. After performing the area threshold processing, the image processing means for extracting only the rainfall snow component and calculating the sum of the squares of the concentration or the area ratio of the extracted rainfall snow component, and use this as a rainfall snow detection result Since it has a rainfall snow intensity calculation unit that outputs, even if there is a running vehicle in the shooting screen, it can be excluded and only the rainfall snow component can be extracted, and therefore, on the road where the vehicle runs This has the effect that the intensity of only the rainfall snow component in the rainfall snow state can be accurately calculated.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment.

FIG. 2 is a diagram illustrating the principle of a sequential difference method employed in the first embodiment.

FIG. 3 is a flowchart of a processing operation according to the first embodiment.

FIG. 4 is a diagram illustrating an example of an image of a moving object extracted by a sequential difference method.

5 is a diagram showing an example of an image in which only the rainfall snow component is extracted by performing the area threshold processing on the image of FIG. 4;

FIG. 6 is a diagram illustrating an example of calculating a sum of density squares according to the first embodiment.

FIG. 7 is a block diagram showing a second embodiment.

FIG. 8 is a diagram illustrating the principle of the dynamic difference method employed in the second embodiment.

FIG. 9 is a flowchart of a processing operation according to the second embodiment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 imaging means 101 TV camera 102 low-pass filter (LPF) 103 AD converter 2 image processing means 3 rain / snow intensity calculation unit 201 multiplexer 202 a, b frame image memory 203 difference processing unit 204 moving object determination processing unit 205 area threshold processing Unit 206a photographed image memory 206b background image memory

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor, Isao Horiba 50-2 Shinbayashi, Higashisakai-cho, Kariya City, Aichi Prefecture

Claims (5)

[Claims] 1. A method for detecting only image information relating to a moving object from a photographed image by performing a sequential difference operation between two temporally adjacent images photographing the rainfall snow detection target area, and A method for detecting rainfall snow, wherein only a rainfall snow component in a moving object image is extracted by performing area threshold processing. 2. A rain / snow detection method characterized by extracting only a rain / snow component from a photographed image by performing a sequential difference operation between two temporally adjacent images obtained by photographing a rain / snow detection target area. . 3. A dynamic difference operation between a background image of a rainfall snow detection target area and a photographed image obtained by photographing the rainfall snow detection target area detects only image information relating to a moving object from the photographed image. A rainfall snow detection method characterized by extracting only a rainfall snow component from a moving body image by performing area threshold processing on the obtained moving body image. 4. The method according to claim 1, wherein only a rainfall snow component is extracted from the photographed image by performing a dynamic difference operation between a background image of the rainfall snow detection target region and a photographed image photographing the rainfall snow detection target region. Rainfall snow detection method. 5. A photographing means for photographing a rainfall / snow detection target area, a sequential difference operation between two images temporally adjacent to each other or a dynamic difference operation between a photographed image and a background image, and then an area threshold value. Image processing means for executing processing and extracting only the rainfall snow component, a rainfall snow intensity calculation unit that calculates the sum of squared concentrations or area ratio of the extracted rainfall snow component, and outputs this as a rainfall snow detection result, A rainfall snow detection device comprising: