JPH1042274A - Abnormality monitoring method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an abnormality diagnostic method capable of attaining an appropriate processing result and correct judgement even in the case that the environmental change of illumination conditions and the intrusion of the shadow of an object, etc., is present. SOLUTION: After monitoring pictures inputted from a monitoring object area by a television camera or the like are smoothed and noise is eliminated, the histogram of a picture element number for respective luminance values is obtained, based on the luminance values for respective picture elements in the pictures and thus, the cumulative curve of the picture element number for the respective luminance values is obtained. An inflection point for the cumulative curve, the luminance of the inflection point and the average luminance of source pictures are obtained and the presence/absence of the environmental change are judged depending on whether or not the absolute value of the difference of the inflection point luminance and the average luminance is larger than the index value of the environmental change. Then, in the case of judging that the environmental change is present, the monitoring picture is luminance divided into two areas, difference pictures with a reference picture are obtained for the respective divided areas and respectively binarized, the ratio of the white or black of processing pictures is compared with a reference value and the presence/ absence of abnormality are judged. On the other hand, in the case of judging that no change of an environment is present, the difference picture of the reference picture and the monitoring picture is obtained and binarized, the ratio of white or black after a processing is compared with the reference value and the presence/absence of the abnormality are judged.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for monitoring abnormalities, and more particularly to a method and an apparatus for monitoring abnormalities in a site such as a boiler power generation facility and a chemical plant facility, in which a change in lighting conditions or the like in a monitoring area occurs. The present invention relates to an abnormality monitoring method and apparatus capable of properly complementing an on-site patrol of an operator in an error case.

[0002]

2. Description of the Related Art Generally, in large plants such as power plants and chemical factories, operation is performed by remote operation and central operation using an operating device for operation and various instruments for monitoring. The conventional on-site abnormality monitoring method used at that time is that a raw signal (image / sound) transmitted from a television camera or a microphone is reproduced as it is on a monitor television or a speaker, and the operator monitors this. Was. Such monitoring has a heavy burden on the operator and requires skill of the operator since the operator always determines whether or not the abnormality is abnormal.

Further, with the recent development of image processing technology and image processing apparatuses, images from a television camera are mechanically processed by an image processing apparatus to be useful for determining the presence or absence of an abnormality on site and early detection of an abnormality. The number of plants that reduce the power of patrols that patrol and monitor the site is increasing. The introduction of surveillance systems aiming at the practical use of so-called alternative systems for human visual functions has begun.

[0004] As shown in FIG. 7, a monitoring robot 6 is installed at a plant site, and a television having the monitoring robot 6 mounted on the monitoring robot 6 with image data and sound data near a monitoring target at a predetermined monitoring route or monitoring point. The data is collected by the camera 7 and the microphone 8, and the data is generally transmitted to the abnormality determination device 9 installed in the central control room via the communication control device 12.

[0005] In the patrol inspection by the on-site patrol, it is said that the rate of discovering various fluid leaks (leakage) from plant equipment by the patrol's vision and hearing is about 90% of the total number of occurrences. Automated decision making by sound processing and its reliability are important issues. As a basic method of finding an abnormality by image processing, as in the algorithm shown in FIG. 8, (a) an image of each monitoring point in a normal state is previously captured and stored (this is referred to as a reference image ( Memory (2))),
(B) Next, the presence or absence of abnormality in the monitored object, and the difference between the image (memory (1)) captured by the television camera mounted on the monitoring robot during monitoring and the reference image (this is referred to as a difference image (memory (1)) -Memory (2))) as the base data, the environmental conditions of the optical system at the monitoring point, and the degree of abnormal phenomena (this is actually
Abnormal conditions cannot be generated in piping, etc., so simulate oil leaks, etc.). The optimal threshold is set to clearly distinguish the difference between. That is, in the difference image, a portion having a large difference in luminance due to the threshold is white, and a portion having a small difference in luminance is black (this is referred to as a binarization process). The histogram processing for calculating the accumulated value of the number of pixels is performed, and it is determined whether or not there is an abnormality according to the number of white pixels.

[0006] However, the validity of the threshold value of the luminance at the time of the above-mentioned binarization process becomes a problem. The main issues are as follows. (1) Significant changes in illuminance due to changes in sunlight, etc.
Due to disturbances such as irregular positions, sizes, and shadows in the field of view, and changes in lighting conditions such as fluorescent lights even indoors, the result of the binarization processing changes dramatically. (2) At first, the subject in the field of view at the time of reading and storing as the reference image is displaced or the degree of positional reproducibility when the monitoring robot stops at the position of the monitoring point depends on the difference image. The luminance value changes, and the number of white pixels after the binarization process increases. (3) Therefore, conditions for the so-called image processing disturbance can be set in advance, and even if these disturbances can be avoided, a reference image is stored for each monitoring item or monitoring point. It is expensive and requires a lot of memory.

[0009] Further, an example is shown in which a water flow as shown in FIG. 9, for example, a flow 1 of a river is monitored by a television camera, and a flow of oil flowing into the river is detected. A1 is a normal image,
A2 is stored in a memory and is used as a reference image. B1 is an image at the time of monitoring the flow of the oil 17.
C is a difference image between A2 and B1, which is subjected to a binarization process, and becomes D1, D2, and D3 according to the threshold value.

[0008] If D1 is too low, D2 is
Indicates that the threshold is appropriate, and D3 indicates that the threshold is too high. Since the brightness of the surface of the river changes every moment, the difference image changes greatly. As shown in FIG. 9, the number of white pixels as a result of the difference image / binarization processing is
Typical examples are shown at D1, D2, and D3 in the figure. In order to extract only the oil of the abnormal event, the threshold value at the time of the above-mentioned binarization process is determined to determine the ratio of white, that is, the normal state. The image change from the reference image fluctuates from 2 to 35%, and it is difficult to determine the final determination threshold value.

FIG. 10 shows an example in which the flow of an oil is detected by a television camera in the flow of a river having an environmental change such as a change in sunshine. A1 is a normal image, and A2 is a reference image stored in a memory. B1 is an image at the time of monitoring in which the shadow 2 exists and the oil 17 flows. C
Is a difference image between A2 and B1. The difference image is binarized, and becomes D4 and D5 depending on the threshold value.

D4 is D5 if the threshold is too low.
Indicates that the threshold is too high. As shown in FIG. 10, since the brightness of the river surface changes every moment due to a shadow or the like, the difference image fluctuates greatly. FIG.
As shown in the figure, the number of white pixels as a result of the difference image / binarization process is a typical example of D4 and D5 in the figure. When the threshold value is determined in this case, the ratio of white, that is, the variation from the reference image increases, and it is difficult to determine the final determination threshold value.

On the other hand, as a method other than the difference image / binarization processing method, a method in which a normal case without oil leakage and an abnormal case with oil leakage are learned by a neural network can be applied. If the image is far apart from the learning image, the output value of the determination from the neural network may be ambiguous. This is in the field of neural networks
It has a difficult task as a generalization ability, and its effectiveness is limited, and its prediction is difficult at present.

[0012]

In the above-mentioned prior art, there are insufficient measures against lighting conditions for the monitoring target or the monitoring target area and environmental changes such as intrusion of a shadow of the object, and an erroneous processing result is not obtained. Occurred in some cases.
It is an object of the present invention to provide an abnormality monitoring method and apparatus capable of correctly determining a processing result and a correct determination even when there is a change in an environmental condition such as a lighting condition or a shadow of an object. is there.

[0013]

The invention claimed in this application to achieve the above object is as follows. (1) In an abnormality diagnosis method for detecting the presence or absence of an abnormality based on a difference image between a reference image in a normal state and a monitoring image during monitoring of a monitoring target area, a cumulative curve of the number of pixels for each luminance of the monitoring image is calculated. Based on the absolute value of the difference between the luminance of the inflection point for the cumulative curve and the average luminance of the original image, it is determined whether there is any environmental change such as a change in illuminance of the monitored area or intrusion of a shadow. In some cases, the monitoring image is divided into a plurality of areas according to the degree of environmental change,
A difference image from the reference image is obtained for each of the divided areas, and based on the difference image, the presence or absence of abnormality in the monitoring target area is determined.If there is no environmental change, an abnormality is determined based on the difference image between the reference image and the monitoring image. An abnormality monitoring method characterized by determining the presence or absence of a fault.

(2) In an abnormality monitoring method for detecting the presence or absence of an abnormality based on a difference image between a reference image input in a normal state and a monitoring image input during monitoring of a monitoring target area, each brightness of the monitoring image is The step of calculating the number of pixels of, the step of calculating the cumulative value of the number of pixels for each luminance based on this, the step of calculating the cumulative curve of the number of pixels for each luminance based on the calculated cumulative value, The luminance at the inflection point and the average luminance of the original image are obtained. If the absolute value of the difference between the two luminances is greater than the threshold value of the index value of the environmental change, there is an environmental change. Determining a difference image between the reference image and the monitor image when there is no environmental change, binarizing the difference image and comparing it with a reference value to determine whether there is an abnormality; When there is For each luminance value obtained for the monitoring image and a histogram of the number of pixels corresponding to the luminance value, the monitoring image is divided into two regions at a division luminance that maximizes the inter-class variance when divided. A difference image from the reference image is obtained for each of the divided regions, and each is binarized and compared with a reference value.
A step of determining that there is an abnormality if any of them exceeds a reference value.

(3) In an abnormality monitoring apparatus for detecting the presence or absence of an abnormality based on a difference image between a reference image in a normal state and a monitoring image at the time of monitoring, the number of pixels for each luminance of the monitoring image Means for calculating the cumulative value of the number of pixels for each luminance, means for calculating the cumulative curve of the number of pixels for each luminance based on the calculated cumulative value, and an inflection point for the cumulative curve. Means for determining the average luminance of the luminance and the original image and determining the presence or absence of an environmental change in the monitoring target area based on the absolute value of the difference between the two luminances, based on the difference image between the reference image and the monitored image if there is no environmental change Means for judging the presence or absence of an abnormality; means for dividing the monitoring image into two areas according to luminance so as to maximize variance between classes when there is an environmental change; Is Obtaining a difference image between the reference image for each area, the abnormality monitoring device characterized by comprising a means for determining the presence or absence of an abnormality in the monitored area based on this.

In the present invention, the detection resolution of the image processing for the subsequent judgment is improved by dividing the image to be monitored into a plurality of regions with respect to the disturbance exerted on the image processing and processing the image. In addition, erroneous determination in the monitoring system can be significantly reduced.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to FIG. FIG. 1 shows an abnormality diagnosis algorithm by image processing according to the present invention. An image is captured by a camera (camera input). One screen is composed of a large number of pixels. For example, on a television, the vertical direction is 256
Columns, horizontal direction are 256 rows, that is, 256 × 256 = 65
It is composed of 536 pixels, and one pixel outputs a luminance signal divided into a plurality of stages.

Normally, the luminance signal is 25 from 0 to 255.
There are six stages. Next, a smoothing process is performed in accordance with the degree or magnitude of noise in the image, and preprocessing for subsequent image processing operations is performed. Smoothing processing refers to performing, for each pixel, a process of replacing the brightness of the pixel with the average brightness obtained by dividing the sum of each brightness of surrounding pixels by the number of pixels, thereby reducing noise. Remove. The next step is histogram processing. FIG. 2 shows an example of creating a histogram. That is, the image (A) shown in the upper part of FIG.
Has luminance in the range of 0 to 5 for each of the 25 pixels in each of the vertical and horizontal 5 columns. Based on this, as shown in the lower part of FIG. 2, the luminance is plotted on the horizontal axis and the number of pixels is plotted on the vertical axis. A histogram of the image of the smoothing process indicating the number of pixels for each luminance is obtained.

That is, the histogram is obtained by calculating the number (frequency) of pixels having the luminance value in the image for each luminance value. Generally, it is represented by a bar graph in which the horizontal axis represents the luminance value and the vertical axis represents the number of pixels (frequency). The histogram is a collection of information on what luminance values the image has from the pixels.

The next step is the step
A cumulative curve is obtained from the histogram. The cumulative curve is His
It is obtained by accumulating the number of pixels for each luminance of the
You. That is, as shown in the drawing of FIG.
₁, I_Two, I_Three, I_Four, I _FiveThe number of pixels of each
p₁, P_Two, P_Three, P_Four, P_FiveThe original histogram that is
From the luminance i as shown in FIG._TwoIs the accumulated value of p_Two
+ P₁And the luminance i_ThreeIs the accumulated value of p_ThreeTo p₁+ P_TwoTo
The line that the cumulative value obtained in this way is drawn
It becomes a cumulative curve. FIG. 3 shows an example of the cumulative curve.

In the next step, the inflection point of the cumulative curve and the average luminance of the original image are obtained. Based on the distance between the average luminance and the inflection point, it is determined whether or not there is a change in the environmental condition due to illumination such as a shadow or illuminance. FIG. 4 shows the relationship between the average luminance and the inflection point. That is, the cumulative curve of the luminance value versus the number of pixels of the rectangular area surrounded by the dashed line of the image without shadow at the upper left of FIG. 4 is shown at the lower left, and the luminance of the area within the dashed line of the image with shadow at the upper right is shown in FIG. The cumulative curve of value versus number of pixels is shown in the lower right. FIG.
, 1 is a river to be monitored, 2 is a shadow, 3 is an inflection point of a cumulative curve, 4 is an average luminance, and 5 is an index value of an environmental change described later. Average luminance 4 is defined as follows.

Calculation of average luminance (s)

[0023]

(Equation 1)

M: luminance p: number of pixels of luminance m z: total number of pixels L: maximum luminance value The inflection point is defined as follows.

Detection of inflection point (d) First derivative

[0026]

DI (m) = I (m + 1) −I (m) (Equation 2) m: luminance I (m): cumulative number of pixels of luminance m dI (m): first derivative of luminance m 2 Next derivative d^TwoI (m) = dI (m + 1) −dI (m) (Equation 3) d^TwoI (m): Second derivative of luminance m Inflection point search Cumulative curve gradient d_T(M) greater than a certain "threshold"
And the second derivative d ^TwoAt the minimum value of I (m)
Is the inflection point.

DI (m)> δ,... (Equation 4) δ: Threshold of the first derivative D = min (d ² I (m)) (Equation 5) *: d ² I ( A search is made for m (luminance) that minimizes m), that is, the luminance of the inflection point D. Δ in (Equation 4) is determined depending on the object to be monitored.

The relationship between the luminance at the inflection point D and the average luminance (s) (absolute value g of the deviation amount) is shown below as an index value for judging the presence or absence of a change in environmental conditions due to illumination such as shadows and illuminance. . g = | (s) − (D (m)) | (Equation 6) g: index value of environmental change (s): average luminance (D (m)): luminance of inflection point • no environmental change Case g <α (Equation 7) α: Threshold value of environmental change index value ・ If there is environmental change g> α (Equation 8) α: Threshold value of environmental change index value α It is determined depending on the object to be monitored.

Here, when g <α, that is, when there is no environmental change, the entire area of the captured image is set as the target area. In the next step, region division is performed when g> α, that is, when there is an environmental change. In the area division, a threshold for optimal division in a statistical sense is determined from a histogram of an image, and the area is divided based on the threshold. FIG. 5 shows a flow of the threshold value determination.

The principle of determining the threshold is defined below. ^{_{σ B 2 = ω 1 (M}} 1 -M T) 2 + ω 2 (M 2 -M T) 2 ...... ( Equation ^{_{9) σ W 2 = ω 1}} σ 1 2 + ω 2 σ 2 2 ...... ( Equation 10) However, omega ₁ (): number of pixels omega ₂ class 1 (): number of pixels class 2 M _1: average of class 1 brightness M _2: average of class 2 luminance M _T: average brightness of all pixels sigma _1: class 1 variance σ ₂ : class 2 variance. Classes 1 and 2 refer to pixel groups classified according to luminance.

The luminance value is changed from {β: minimum value to L: maximum value}.
And the threshold value K is determined based on the luminance [β, K] and [K +
1, L] into two classes C ₀ and C ₁ .
Equation (9) shows the inter-class variance (σ _B ² ). Equation (10) shows the intra-class variance (σ _W ² ). The threshold value to be determined determines the threshold value K by maximizing the ratio of the variance of the average value of the two classes, that is, the variance between the classes and the variance of each class, that is, the variance within the class.

Here, to maximize σ _B ² / σ _W ² , σ _B ² may be maximized. That is, K: a threshold value may be obtained by changing the threshold value to maximize σ _B ² . The minimum brightness β is determined depending on the object to be monitored. β is the abnormality to be monitored, that is, the luminance abnormality of the oil leak here. It is shown that the above-described threshold K minimizes the mean square error between the obtained binarized image and the original grayscale image.

FIG. 6 shows an example of the area division by the threshold value K for the histogram of the luminance value and the number of pixels. As shown in the figure, when there are two regions to be divided, it is possible to divide a pixel having a luminance equal to or higher than the threshold value into a sunny region and a pixel having a luminance equal to or lower than the threshold value into a shadow region. Next, a binarization process is performed for each of the regions (here, the region of the sun and the region of the shadow).

FIG. 12 shows an embodiment applied to the case where oil flows in a river and a shadow is projected on the lower right of the input image in FIG. In FIG. 12, an image (monitoring image) B input at the time of monitoring is divided into a sunny area B ₁ ′ and a shadow area B ₂ ′. The difference image between the reference image A ′ and the region B ₁ ′ image is C ₁ ,
The binarized image of C ₁ is D ₁ . Next, the difference image between the reference image A ′ and the area B ₂ ′ image is C ₂ . Difference upper left half C ₂ a in the image C ₂ is relatively a luminance difference is small,
The lower right half C ₂ b has a relatively large luminance difference, but if the threshold value is properly selected and binarized, the binarized image of C ₂ becomes D ₂ .

Here, the D ₁ image has a white ratio of 0% and the D ₂ image has
The image has a white ratio of 15%. In FIG. 1, assuming that a threshold for determining the presence or absence of an abnormality is, for example, 10%,
The above case is determined to be abnormal and an alarm is issued. Each threshold value when binarizing the difference image between the normal reference image and the monitoring image is determined depending on the object to be monitored. A threshold value for the final judgment is set for the number of white pixels of the binary image for each of the divided areas. If any of the divided areas exceeds this threshold, it is determined that there is an abnormality, and If not, it is determined to be normal.

From the above, conventionally, if the difference image / binarization processing is performed in an environment where there is a partial change in sunshine or illuminance such as a shadow, the environment change area is erroneously determined to be an oil area. In the present embodiment, by performing area division and limiting the target area before performing the difference image / binarization processing,
It is possible to respond correctly without erroneous judgment.

[0037]

According to the present invention, even in the case where a change in the lighting condition in the monitoring target area or an environmental change such as the intrusion of a shadow into the area by another object occurs, the abnormality can be appropriately detected without erroneous determination. Can be detected.

[Brief description of the drawings]

FIG. 1 is a diagram showing an algorithm of an abnormality monitoring method according to an embodiment of the present invention.

FIG. 2 is a diagram showing a procedure for creating a histogram of a luminance value versus the number of pixels for an image.

FIG. 3 is an embodiment diagram showing a cumulative curve of a luminance value versus the number of pixels obtained from the histogram shown in FIG. 2;

FIG. 4 is an example diagram showing a relationship between an inflection point and an average luminance in a cumulative curve of luminance value versus number of pixels.

FIG. 5 is a flowchart of a region division of a monitoring image according to the embodiment of the present invention.

FIG. 6 is a view showing an embodiment of area division.

FIG. 7 is a diagram showing an abnormality monitoring device according to the related art.

FIG. 8 is a flowchart of an abnormality monitoring method according to the related art.

FIG. 9 is an explanatory diagram of an abnormality monitoring method according to the related art.

FIG. 10 is an explanatory diagram of an abnormality monitoring method when an environment changes according to the related art.

FIG. 11 is an explanatory diagram showing a procedure for creating a histogram of luminance versus the number of pixels of a target image according to the present invention.

FIG. 12 is an embodiment diagram showing a processing method at the time of region division of a monitoring target image.

[Explanation of symbols]

1 ... river (flow), 2 ... shadow, 3 ... inflection point, 4 ... average luminance, 5 ... g (index value of environmental change), 6 ... surveillance robot,
7 ... TV camera, 8 ... Microphone, 9 ... Abnormality judging device, 10 ... Man-machine interface, 11 ...
Robot controller, 12: Communication controller, 13: Robot traveling axis, 14: Sensor part (camera part) turning axis, 15 ...
Sensor unit vertical axis, 16 monitor.

Claims (3)

[Claims] In an abnormality diagnosis method for detecting the presence or absence of an abnormality based on a difference image between a reference image in a normal state and a monitoring image at the time of monitoring of a monitoring target area, the accumulation of the number of pixels for each luminance of the monitoring image is performed. A curve is obtained, and based on the absolute value of the difference between the luminance of the inflection point for the cumulative curve and the average luminance of the original image, it is determined whether or not there is an environmental change such as a change in illuminance of the monitored area or intrusion of a shadow in the monitored area. If there is a change, the surveillance image is divided into a plurality of areas according to the degree of environmental change, and a difference image from the reference image is obtained for each of the divided areas. And determining whether there is an abnormality based on a difference image between the reference image and the monitoring image when there is no environmental change. 2. An abnormality monitoring method for detecting the presence or absence of an abnormality based on a difference image between a reference image input in a normal state of a monitoring target area and a monitoring image input at the time of monitoring. A step of calculating the number of pixels, a step of calculating a cumulative value of the number of pixels for each of the luminances based thereon, a step of calculating a cumulative curve of the number of pixels for each luminance based on the calculated cumulative value, The luminance at the inflection point and the average luminance of the original image are obtained. If the absolute value of the difference between the two luminances is larger than the threshold value of the index value of the environmental change, there is an environmental change. Determining a difference image between the reference image and the monitoring image when there is no environmental change, binarizing the difference image and comparing it with a reference value to determine whether there is an abnormality; Sometimes monitoring Dividing the monitoring image into two regions at a divisional luminance that maximizes the inter-class variance when divided for each luminance value obtained for the image and a histogram of the number of pixels corresponding to the luminance value Then, a difference image from the reference image is obtained for each of the divided areas, and each is binarized and compared with a reference value.
A step of determining that there is an abnormality if any of them exceeds a reference value. 3. An abnormality monitoring device for detecting the presence or absence of an abnormality based on a difference image between a reference image in a normal state and a monitoring image at the time of monitoring of a monitoring target area, wherein the number of pixels for each luminance of the monitoring image is determined. Means for calculating, a means for calculating a cumulative value of the number of pixels for each of the luminances, a means for calculating a cumulative curve of the number of pixels for each of the luminances based on the calculated cumulative value, and an inflection point of the cumulative curve. Means for determining the luminance and the average luminance of the original image to determine the presence or absence of an environmental change in the monitoring target area based on the absolute value of the difference between the two luminances, and when there is no environmental change, abnormal based on the difference image between the reference image and the monitored image Means for judging presence / absence of the image, and means for dividing the monitoring image into two regions by a division luminance such that the inter-class variance is maximized when the monitoring image is divided into two parts by luminance when there is an environmental change. Area Find the difference image from the reference image for each
Means for judging the presence or absence of an abnormality in the monitoring target area based on the information.