JPH0767101A - Method and device for monitoring abnormality - Google Patents

Abstract

PURPOSE:To precisely detect the abnormality of an equipment with a visual state change by using the picked-up image information of the equipment. CONSTITUTION:The edge part of a picked-up image in a monitor target area is extracted by an edge extraction part 3, and this edge part is defined as a mask image. Next, an arithmetic part 5 calculates the image of difference between this picked-up image and the image of the monitor target area picked up after the lapse of time from the photographing of the picked-up image, and the differential image is masked with the mask image by a logic arithmetic part 7. The differential image provided as a result is binarized with a binarizing threshold value by the logic arithmetic part 7, the binary image is provided, and an abnormality deciding part 8 decides the presence/absence of abnormality from the size of the area of the binary image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an abnormality monitoring method and apparatus for visually monitoring component equipment such as a power plant using picked-up image information, and more particularly to a method for detecting abnormality in a short time. An abnormality monitoring method and its apparatus.

[0002]

2. Description of the Related Art As a conventional technique of a monitoring device, there is, for example, one disclosed in Japanese Patent Laid-Open No. 2-171897. This prior art is for monitoring an intruder or the like. First, in the setting mode, the reference images are generated by accumulating the edge images of each of a plurality of images taken at a certain time (that is, there is no intruder. The edge image of the image obtained when only the illuminating light beam changes in the state is stored.), The edge image is obtained from the image captured in the next detection mode, and this edge image is compared with the reference image to obtain a new image. We are trying to determine the presence or absence of an intruder based on the presence / absence of an edge image that appears in.

In the prior art disclosed in Japanese Patent Laid-Open No. Sho 60-7593, the difference between two images picked up with time is calculated, and when the number of pixels of the difference is equal to or larger than a set value, it is judged that an abnormality has occurred. ing.

[0004]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
It is desired to automate the monitoring of the constituent devices. For example, in the case of monitoring the flange of the pipe, if an abnormality occurs in the flange, water droplets, steam leakage, etc. will occur from the abnormal portion. Therefore, it is possible to monitor by automatically detecting the presence or absence of these fluid leaks. However, the above-mentioned conventional technique cannot be directly applied to the monitoring of such water drops.

In monitoring a power plant or the like, a robot is equipped with a monitoring camera, and a single monitoring camera monitors several tens of monitoring target areas. This plant monitoring is described in Japanese Patent Laid-Open No. 2-171897. In the case of applying the above-mentioned conventional technique, it becomes necessary to previously generate a reference image for each monitoring target area. That is, when the reference image is generated in advance,
When monitoring a certain area to be monitored, it becomes necessary to move the monitoring robot to a predetermined position, which makes control of the monitoring robot difficult. Further, since the illumination light of the plant is not always constant, a situation may arise in which a plurality of reference images must be generated for one monitoring target area. In particular, in a nuclear power plant, when the reactor enters the operating state, maintenance and inspection of the monitoring robot cannot be performed until the next periodic inspection for about one year. Therefore, it is practically impossible to apply this conventional technique to the monitoring of a power plant.

Further, if the presence / absence of an abnormality is judged from the edge image newly appearing in the accumulated image of the edge images, there is a problem that the occurrence of the abnormality is likely to be overlooked. This is because, in the above example, the judgment is made based on the presence / absence of the edge image of the image of a considerably small water drop. Regarding this point, it can be said that it is more reliable in plant monitoring to obtain it from the difference between two images with time, as in the conventional technique described in Japanese Patent Laid-Open No. 60-7593. However, when the difference image between the two images is obtained, not only the image of the water drop is extracted, but various images appear in the difference image, and the water drop image is buried in the difference image, and the presence or absence of the water drop is detected. It is difficult for the device to judge automatically.

Further, the image processing requires binarization processing of the picked-up image, and the detection accuracy varies depending on the method of setting the threshold value for the binarization. We have not resolved the threshold issue.

It is an object of the present invention to provide a plant abnormality monitoring method and apparatus capable of automatically detecting a minute change without generating a reference image in advance.

[0009]

The above-mentioned object is to extract an edge portion of a captured image of a monitoring target area, and to capture the captured image and a captured image of the monitoring target area captured at a time after the capturing of the captured image. When the difference image is obtained, the difference image is obtained using the edge portion as a mask image, and the difference image is set to a threshold value of 2
This is achieved by performing binarization to obtain a binarized image and determining the presence or absence of abnormality from the binarized image.

The binarization threshold value is automatically set in the difference processing for obtaining the difference image as follows. First, the relationship between the average density of an image and the density difference in the difference processing is experimentally obtained in advance. Next, before performing the difference processing of the two captured images obtained in time series, the average density of the monitored image is obtained from the grayscale image of the captured monitoring target area. By using the average density of this image, the binarization threshold value is automatically set based on the relationship between the average density of the image and the density difference which is obtained in advance.

Preferably, the binary images thus obtained are accumulated, and the area of the accumulated image is obtained to determine the abnormality.

[0012]

The difference image between the two images obtained in time series contains various noises. The noise is generated due to, for example, the characteristics of the CCD, which is the image pickup means, or a subtle change in illumination light, but according to experiments by the inventors of the present invention, it was found that the noise is concentrated on the edge portion of the imaged image. Therefore, in the present invention, most of the noise can be removed by obtaining the edge portion and using this edge portion as a mask image.

When an abnormality occurs, the image density changes due to the abnormality. This change is related to the average density of the captured image. For example, when a water drop is leaking, the image of the water drop is increased compared to the original normal image, and the average density of the image is increased accordingly. Therefore, by changing the binarization threshold value based on the relationship between the average density of the captured image and the density difference, it is possible to obtain a binarized image that is excellent in the presence or absence of abnormality.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. First, the outline of the present invention will be described with reference to FIGS. FIG. 2 is a diagram for explaining the principle of the difference processing method for detecting an abnormality with a short time constant of state change such as water leakage. This utilizes the fact that a water drop changes its position at the next moment, and by performing the difference processing between two images captured in time series, only the pixels with a difference between the images are extracted and an abnormality is detected. To do. Here, an example in which the water droplet 21 leaks from the flange 23 connected to the pipe 22 in the plant will be described.

First, the input image 20a to be monitored at time t is captured. This input image 20a has a flange 2
An image of the water droplet 21a leaked from No. 3 is captured. Next, the input image 20b to be monitored at time t + Δt is captured. The water droplet 21b in the input image 20b has fallen in the vertical direction during Δt, and the water droplet 21a in the input image 20a.
The image is captured at a position different from. Therefore, the difference processing between the two images 20a and 20b is performed, and the binarization processing is further performed. By such processing, the difference image 20c is obtained,
Pixels having a difference between the images, that is, the water droplet 21c can be detected. However, not only the water droplet 21c but also the noise 24 is detected in the difference image 20c at the same time. This noise is generated from pixels in the input image where the density changes abruptly, that is, from the edge portion of the image. The reason for this is considered to be due to the characteristics of the image sensor of the ITV camera. In FIG. 2C, the noise is illustrated as a noise having almost the same shape as the edge of the flange, but in reality, the noise does not appear as a continuous line corresponding to the edge,
It is a discontinuous and considerably disturbed noise in the form of a broken line, which at first glance cannot be recognized as the edge of the flange. The reason for this noise generation is considered to be due to the characteristics of the image sensor of the ITV camera. As a conventional technique for removing this noise, there is, for example, one described in Japanese Patent Laid-Open No. 2-205998. In this conventional technique, in order to eliminate the influence of noise, a window is provided in an area where noise does not enter on the screen and the monitoring area is limited. For this reason, there is a problem that even if the image pickup area of the camera is wide, what can actually be monitored is limited to a narrow window.

FIG. 3 is a diagram for explaining the outline of the present invention. First, the input image 20d to be monitored is captured. Next, the edge portion of the image of the input image 20d is extracted to generate the mask image 20e in which only the edge portion is extracted. The difference image 20c shown in FIG. 2 is masked with reference to this mask image 20e, that is, in this mask processing, the difference processing is not performed around the pixels only in the edge portion of the image. Further, 2 is applied to the difference image subjected to the mask processing.
Perform value conversion processing. As a result, the difference image 20f in which only the water droplet 21c is detected is obtained. By such processing, without limiting the monitoring area in the difference processing between time-series images,
It is possible to monitor the abnormality in the entire imaging range of the surveillance camera.

Next, an automatic setting method of the binarization threshold value in the difference processing according to the present invention will be described. In the difference processing, even if the difference between images of the same monitoring target is calculated, the density of the difference image does not become zero and has a significant value. This is because of a quantization error associated with A / D (Analog to Digital) conversion.

FIG. 4 is experimental data showing the relationship between the average density of an image and the maximum value of the density difference in the difference processing, in which the difference processing is repeated 10 times and the maximum value of the density difference is measured 3 times. Here, “normal time” (white circle) is the result when there is no water droplet, and “abnormal time” (black circle) is the result when the water droplet is dropped. In this way, the density difference 25 in the normal state and the density difference 26 in the abnormal state also linearly increase depending on the average brightness of the image, that is, the average density of the image. In this case, it can be seen that there is a significant difference in density of about 20 at the time of occurrence of abnormality and at the time of normality. Therefore, the binarization threshold value 26 is set to the level of the density difference of 25 or more in the normal state depending on the average density of the image.
By setting, it is possible to automatically detect an abnormal event. Here, since the density difference linearly increases depending on the average density of the image, the binarization threshold value is represented by a linear expression of the average density of the image.

The binarization threshold value T is expressed by the following equation 1 when the average density I of the image is used.

[0020]

## EQU1 ## T = A.I + B where A and B are constants.

By thus setting the binarization threshold value from the average density of the image, the binarization process in the difference process can be automated. The relationship between the average density of the image and the density difference in this difference processing is obtained in advance and stored in the memory.

Next, the processing algorithm of the first embodiment of the present invention will be described for the case of an abnormality with a short time constant of state change such as water leakage. FIG. 5 is a flowchart showing a difference processing procedure of time series images according to the present embodiment. First, the relationship between the average density of an image and the density difference in the difference processing is obtained in advance, and a constant of a linear expression representing the relationship between the binarization threshold value and the average density is stored in the memory (step 11A). Next, a mask image is generated by extracting the edge portion of the image from the grayscale image of the monitoring device imaged before the execution of monitoring (step 11B). In the next step 11C, the average density of the image is obtained from the grayscale image of the captured monitoring device,
The binarization threshold value for this monitoring image is set from the relationship between the binarization threshold value obtained in step 11A and the average density.

In the next step 11D, difference processing between images is performed. In this difference processing, first, the image g (t) to be monitored at time t is captured, and then the image g (t + Δt) to be monitored at time t + Δt is captured. Then, the difference processing between the captured images is performed. The difference image obtained by this difference processing is masked with reference to the mask image generated in step 11B (step 11
E), the difference image obtained was set in step 11C 2
A binarized image is obtained by binarizing the binarized threshold (step 11F). In this embodiment, the binary images obtained here are sequentially accumulated to obtain an accumulated image (step 1
1G).

Next, it is judged whether or not to continue this series of processing, and if it is continued, this difference processing is further executed, and if it is not continued, the next step is executed (step 11).
H). The area of the cumulative image thus obtained is calculated (step 11I), and finally it is determined whether this area is greater than or equal to a predetermined value (step 11J). As a result of this judgment,
If the area of the cumulative image is equal to or larger than a predetermined value, it is determined that there is an abnormality (step 11K), and if it is less than the predetermined value, there is no abnormality (step 11L). By the above processing, it becomes possible to accurately perform abnormality monitoring when an abnormality having a short time constant of state change is targeted.

FIG. 1 is a block diagram of an abnormality monitoring device for performing the above-mentioned monitoring. The monitoring image input from the ITV camera 1 is input to the edge extraction unit 3 and the calculation unit 5 via the input unit 2. The edge extraction unit 3 performs edge extraction processing on the monitoring image, generates a mask image, and stores the mask image in the image memory 4. The calculation unit 5 executes the grayscale image acquisition and difference processing, and outputs the calculation result to the logical calculation unit 7. The logical operation unit 7 carries out a masking process on the difference image with reference to the mask image, a binarization process of the masked difference image, and an accumulation process of the binary image. The abnormality determination unit 8 calculates the area of the cumulative image and also determines whether or not this area is equal to or larger than a predetermined value to detect the presence or absence of abnormality. These calculation results are displayed on the display unit 10 via the output unit 9. Here, the ITV camera is used as the image pickup means, but an infrared camera for picking up a thermal image of the device can also be used.

6 and 7 are explanatory views of an abnormality monitoring method according to the second embodiment of the present invention. In the present embodiment, the monitoring image is divided into a plurality of areas, the binarization threshold value in the difference processing is set for each area, and the abnormality is detected more accurately. FIG. 6 shows an example in which the surveillance image is divided into a plurality of areas. Here, for simplicity, the image is divided into four areas. The input image 20 is divided into four regions 20A to 20D as shown in FIG. 6, the average density of the image is obtained for each divided region, and a binarization threshold value for each region is set from this. FIG. 7 is a flowchart showing the procedure for setting the binarization threshold value, and the processing other than the setting of the binarization threshold value is the same as the processing in FIG. In FIG.
First, the average densities Ia, Ib, Ic, Id for each divided area are calculated (step 13A). Next, the binary thresholds Ta, Tb, Tc, Td for each region are calculated by the following equation 2 using the constants A and B of a linear expression expressing the relationship between the binary threshold and the average concentration which are obtained in advance.
~ Number 5

[0027]

[Equation 2] Ta = A · Ia + B

[0028]

[Equation 3] Tb = A · Ib + B

[0029]

[Formula 4] Tc = A · Ic + B

[0030]

## EQU5 ## Obtained by Td = A.Id + B (step 13B).

Using the binarized threshold value thus obtained, difference processing is performed for each area. According to this embodiment,
Since the binarization threshold is set for each region and the binarization process is performed, it is possible to detect an abnormality with a small change in density.

Next, a third embodiment of the present invention will be described. This embodiment is applied when the location of the abnormality is known in advance on the screen of the monitor image. That is, the window is set and the monitoring area is limited. This makes it possible to shorten the time required for the difference processing. FIG. 8 is a diagram for explaining the outline of the third embodiment of the present invention. First, the window 28 is set in the input image 20d to be monitored to limit the monitoring area. By this processing, only the image inside the frame of the window 28 is the processing target. Next, the edge extraction processing of the input image 20d to be monitored is performed to generate the mask image 20e in the window 28. Since the subsequent processing is the same as that of the first embodiment, the description will be omitted. Finally, the difference image 20f is obtained, and the water droplet 2 in the window 28
1c is detected as abnormal.

Next, a fourth embodiment of the present invention will be described with reference to FIG.
This will be described with reference to FIG. The present embodiment is applied when an abnormality with a long time constant such as a state change such as a puddle or an oil puddle is targeted, and a difference process between a monitoring target image in a normal state and an image in monitoring is performed. The abnormality is detected by performing. FIG. 9 is a diagram for explaining the outline of the fourth embodiment. Here, an example will be described in which an abnormality is detected when a puddle is generated on the floor surface (not shown) due to water leakage from the flange portion in the plant.

First, an image 30a of the monitoring target in a normal state is taken before the monitoring is executed. Next, the edge portion of the image of the normal image 30a is extracted to generate the mask image 30b in which only the edge portion is extracted. When monitoring is executed, the image to be monitored is captured. A puddle 31 that was not present in the normal-time image 30a is captured in the monitoring-time image 30c. The difference processing between the monitor-time image 30c and the normal-time image 30a is performed, and the result of this difference processing is the mask image 30.
The mask process is performed with reference to b, and then the binarization process is performed. As a result of this processing, the difference image 30d in which only the abnormality due to the puddle 31 is detected is obtained.

FIG. 10 is a flowchart describing only the difference processing between images in the processing procedure of the fourth embodiment, and the other processing is the same as the processing of the first embodiment. In FIG. 10, a normal time image g stored in advance
The edge portion of n is extracted to generate a mask image gm (step 12A). Next, the image gin at the time of monitoring is taken in, the average density in this image is obtained, and the binarization threshold value is set from this (step 12B). A difference process is performed between the captured monitoring time image gin and the normal time image gn stored in advance (step 12C), and the difference result is masked with reference to the mask image gm (step 12).
D). This difference result is binarized using the binarization threshold value obtained in step 12B to obtain a binarized difference image (step 12E). Next, the area of the difference image obtained here is calculated (step 12F). The subsequent processing is the same as the algorithm of the first embodiment shown in FIG. 5, and the presence or absence of abnormality is determined from the area of the difference image. According to the present embodiment, it is also effective for anomalies with a long time constant of state change such as a puddle.

FIG. 11 is a block diagram of a robot for installing the above-mentioned abnormality monitoring device in a plant. Plant 50
Inside the rails for the abnormality monitoring device of each monitoring device to run.
Le 51 is laid. The ITV camera 1 has a drive mechanism 52.
The ITV camera 1 mounted on the vehicle and moving on the rail 51 and stopped at a predetermined position captures the state of the monitoring device 54 as an image. The image captured by the ITV camera 1 is sent to the abnormality monitoring device 53 installed outside the plant 50,
Presence / absence of state change, that is, presence / absence of abnormality is detected. In addition,
In the plant abnormality monitoring apparatus according to the present embodiment, an abnormal event with a short time constant for state changes such as water dripping and steam leakage, and an abnormal event with a long time constant for state changes such as water pools and oil puddle,
It also applies to abnormal events such as dropping or moving of various equipment.

FIG. 12 is a diagram showing a display example of the display device installed in the abnormality monitoring device 53 described with reference to FIG. When the ITV camera 1 stops at the target position and the operator gives an instruction to start the abnormality monitoring, an image captured by the ITV camera 1 is displayed on the screen (FIG. 12).
(A)). Next, the abnormality monitoring device displays the mask image extracted from this image on the screen (FIG. 12 (b)). The abnormality monitoring device performs difference processing between the first captured image that generated the mask image and the second captured image that is captured next, and obtains a binarized image from the difference image using the binarized threshold value. Display the binarized image on the screen. When it is determined from this image that there is an abnormality, "abnormality occurrence" is displayed on this screen.

[0038]

According to the present invention, an abnormality can be accurately monitored without setting a window on the screen and limiting the monitoring area, and a wide area can be monitored at one time.
Further, since the binarization threshold value for the monitoring image is automatically set based on the relationship between the average density of the image and the density difference obtained in advance in the difference processing, the process of setting the binarization threshold value for each monitoring target device in advance is unnecessary. Becomes As a result, the time required to set the monitoring parameters can be shortened.

[Brief description of drawings]

FIG. 1 is a block configuration diagram of an abnormality monitoring device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a difference processing method for time-series images.

FIG. 3 is a diagram illustrating an outline of the present invention.

FIG. 4 is a diagram illustrating a method of automatically setting a binarization threshold value in the difference processing according to the present invention.

FIG. 5 is a flowchart showing a processing procedure of an abnormality monitoring method according to the first embodiment of the present invention.

FIG. 6 is an explanatory diagram of an abnormality monitoring method according to a second embodiment of the present invention.

FIG. 7 is a flowchart of a main part of the processing procedure of the abnormality monitoring method according to the second embodiment of the present invention.

FIG. 8 is an explanatory diagram of an abnormality monitoring method according to a third embodiment of the present invention.

FIG. 9 is an explanatory diagram of an abnormality monitoring method according to a fourth embodiment of the present invention.

FIG. 10 is a flowchart of a main part of the processing procedure of the abnormality monitoring method according to the fourth embodiment of the present invention.

FIG. 11 is a configuration diagram in which the abnormality monitoring device of the present invention is applied to plant monitoring.

12 is a diagram showing a screen display example of a display of the abnormality monitoring device shown in FIG.

[Explanation of symbols]

1 ... ITV camera, 3 ... Edge extraction unit, 4, 6 ... Image memory, 5 ... Calculation unit, 7 ... Logical calculation unit, 8 ... Abnormality determination unit.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Keiji Tanaka 3-1-1, Saiwaicho, Hitachi-shi, Ibaraki Hitachi Ltd. Hitachi factory

Claims (10)

[Claims] 1. A difference process between a plurality of images obtained by time-sequentially capturing the state of a device and binarizing the difference image obtained by the difference process to detect the presence or absence of abnormality of the device. In the abnormality monitoring method, an edge portion is extracted from a captured image to form a mask image, and the difference image obtained by masking the difference image with the mask image is binarized and binarized. An abnormality monitoring method characterized by determining abnormality of the device from a binarized image. 2. The abnormality monitoring method according to claim 1, wherein the binarization threshold value in the binarization process is obtained from the relationship between the average density and the density difference of the image of the device captured in advance. 3. The abnormality monitoring method according to claim 1, wherein the image masking process is a combination of a process of limiting a monitoring region and a process of masking only an edge portion of the image. 4. The abnormality monitoring method according to claim 1, wherein the image from which the edge portion is extracted is one of a plurality of images obtained by time-sequentially capturing the state of the device. 5. The abnormality monitoring according to claim 1, wherein the image used for the difference processing is an image obtained by capturing an image of a device in a normal state and a corresponding image obtained by capturing the state of the device during monitoring. Method. 6. The binarization threshold in the binarization process according to claim 2, wherein the image of the state of the device is divided into a plurality of areas, and the average density of the image of each area is obtained. And the abnormality monitoring method. 7. An image pickup means for picking up the state of the device in time series, a difference processing means for performing a difference process between a plurality of images picked up by the image pickup means, and a difference image obtained by the difference process. In an abnormality monitoring device including a binarization processing unit that performs binarization processing and an abnormality determination unit that detects the presence or absence of abnormality of the device from the binarized image obtained by the binarization processing, an edge is detected from the captured image. An anomaly characterized by comprising: means for extracting a portion to make a mask image; and means for binarizing the difference image obtained by masking the difference image with the mask image by the binarization processing means. Monitoring equipment. 8. An image pickup means for time-sequentially picking up the state of the monitored device and a relationship between the average density and the density difference of the picked-up image obtained by previously picking up the monitored device by the image pickup means are stored. A storage unit, an edge extraction unit that extracts an edge portion of an image of the monitoring target device captured by the imaging unit in time series, and a plurality of images of the monitoring target device captured by the imaging unit in time series. Means for performing difference processing, mask means for obtaining the remaining image from which the edge portion has been removed from the difference image obtained by the difference processing, average density of the picked-up image for which the difference processing is performed, and the stored in the storage means. A binarization threshold that sets a binarization threshold based on the relationship and binarizes the image output from the masking unit based on the binarization threshold, and the monitoring target from the binarized image by the binarization Check the device for abnormalities. An abnormality monitoring device comprising: an abnormality determining means for determining. 9. An abnormality monitoring apparatus for imaging a monitored device with an imaging means, determining whether or not there is an abnormality in the monitored device from the captured image, and outputting a determination result, wherein the monitoring target by the imaging means during monitoring. The image of the device is displayed on the screen, the image of the edge portion obtained from the image is displayed on the screen,
It is characterized by further comprising display means for displaying on the screen a binarized image obtained by binarizing an image of a result obtained by masking a difference image of a plurality of images time-sequentially captured by the imaging means with the edge portion. Anomaly monitoring device. 10. The abnormality monitoring device according to claim 7, further comprising robot means that carries the image pickup means and moves.