JP2947689B2 - Perimeter monitoring image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a means for monitoring surroundings of plant equipment and the like, and more particularly, to a means for accurately performing image processing for surroundings warning even in outdoor equipment having a large difference in illuminance. Related to technology.

[0002]

2. Description of the Related Art As an apparatus for monitoring the periphery of important plant equipment or the like, for example, an industrial camera is used, which is remotely operated.
It is common to monitor video information projected on a V monitor.

[0003] However, in the case of 24-hour monitoring,
Since it is judged that it is impossible from the physiological point of view, such as the limit of human physical strength, image processing of the obtained video signal is performed, and an alarm is output only when an unusual event occurs, to call attention. By doing so, the maintenance staff only has to determine the situation while watching the video only at the time of alarm output, and the application range of such image processing is expanding.

[0004] For example, in the case of monitoring various situations indoors, the lighting conditions are determined to some extent, the output of the camera video signal is binarized, and various outputs such as addition and subtraction are performed on the binarized signal. Image processing has been performed.

In the case of monitoring various situations outdoors, the lighting conditions are not determined, and are influenced by natural conditions (for example, various conditions such as the sun, clouds, and rain can be considered). Since an appropriate contrast ratio (the contrast ratio affects the setting of a threshold necessary for binarization) cannot be obtained as a condition for binarizing a video signal, At present it is often not done properly.

[0006]

Therefore, for example,
As described in Japanese Patent Application Laid-Open No. 2-311703, "Position detection method", a work of applying a phosphor to a measurement object, irradiating an ultraviolet lamp, and improving a contrast ratio until an appropriate value is performed is performed. There was a problem that occurred.

In the meantime, the monitoring work in the important plant equipment and the like includes, for example, monitoring for the presence or absence of abnormality in the main engine such as a transformer and a line switch (for example, deformation, adhesion, breakage, etc. are considered), and It is broadly divided into monitoring the presence or absence of access to the controlled area other than the authorized person (for example, entry into good faith, illegal entry, etc.).

[0008] Therefore, the objects to be monitored are various objects having a three-dimensional shape, people, and the like, and it is necessary to appropriately grasp them.

Therefore, the present invention achieves the object of the present invention in consideration of the following matters.

First, the three primary color (R, G, B) signals are independently image-processed to provide an image of the three primary colors which are very close to human visual ability, thereby improving the discrimination sensitivity.

Secondly, since the illuminance of the surrounding environment of the monitored object changes, the registered image registered in advance for comparison with the input image is updated every time the illuminance changes, so that the registered image and the registered image are updated. The illuminance difference of the surrounding environment in the input image is reduced.

Third, in order to greatly reduce a processing algorithm for recognizing a change in the situation of a monitoring target, an image of one frame is divided into a plurality of pixels (pixels) in a matrix. further performs addressable to the group of pixels, pixels with a difference in the image data between the registered image and the input image
Performs the process of selecting only the cell group , and based on the number of pixels , an alarm is generated when a general change in the shape of the monitored object or a change in the situation such as intrusion of a foreign object into the monitored area is detected. Alert the maintenance staff by output, etc.For detailed information about the monitored object,
If the maintenance personnel visually check, the monitoring effort of the maintenance personnel is greatly reduced.

[0013]

In order to solve the above-mentioned problems, the present invention is based on an image of a monitored area taken by an image pickup means.
In the peripheral monitoring image processing device for monitoring the monitoring area
And store the reference image divided into a matrix
A reference image memory and an image captured by the imaging unit
Into a matrix corresponding to the matrix of the reference image.
Matrix dividing means for dividing, and the matrix dividing
Monitor input of images divided into matrix by means
The monitoring input image and the reference
For each matrix element associated with the image
And the pixels that constitute one of the matrix elements
Data a ₁ , a ₂ ,..., _An and the other matrix element
Using the constituent pixel data b ₁ , b ₂ ,..., B _n ,

(Equation 1) The K value is sequentially calculated by the
Comparison operation to determine the match / mismatch of response matrix elements
Means , based on the determination result of the comparison operation means,
First warning output that outputs a warning according to the decision of the general operation means
Peripheral monitoring image processing apparatus, comprising:
Provide a replacement.

[0014]

[0015]

According to the periphery monitoring image processing of the present invention, the basic
The reference image and the monitoring input image are
Divided into multiple matrix elements (pixel groups)
Of the reference image and the monitoring input image
The K value representing the correlation between each corresponding matrix element
Each K value is calculated based on this K value.
To detect each image change of the trix element,
The processing algorithm is simplified more than before.

Further , an image pickup means (capturing means) for picking up a color image
Camera), the reference image memory
Stores a reference image of each color component as the reference image.
And the matrix dividing means sets the image pickup means
Each color component image separated from the
And the reference image updating means executes the matrix
Each color component divided into a matrix by the
Image for each color component,
The reference image of each color component is updated, and the comparison operation is performed.
Means for each of the corresponding matrices for each color component.
The K value of the matrix element is calculated sequentially and the corresponding matrix
If the match / mismatch between elements is determined sequentially,
Unable to detect when using image capturing means for capturing images
Detecting matches / mismatches between corresponding matrix elements
Can be. In general, from a color camera, NTSC
Signal and RGB signal are output.
Can use RGB signals close to the human visual ability
desirable. In this case, the comparison operation means may further include a color component.
Match / mismatch between corresponding matrix elements determined for each
Of each corresponding matrix element
The total calculating means for determining the match / mismatch between provided, wherein
The first warning means is configured to determine the total
If a warning is output, abnormalities that occur in the monitored area
Detects with near-visual accuracy and calls the watchman's attention
be able to. Therefore, maintenance personnel should be
To monitor the surveillance area.
The visual burden is greatly reduced. Here, the comparison operation
Means for each corresponding matrix element determined for each color component
The majority of the match / mismatch results is "2 out of
3 ”logic (at least two of the three color components do not match
If so, it is determined that they do not match) .

Also, a peripheral monitoring image processing apparatus according to the present invention
, The illuminance detecting means for detecting an illuminance at the monitoring region
Is determined according to the illuminance value detected by the illuminance detection means.
At the timing, the matrix is divided by the matrix dividing means.
Imports the image divided into
A reference image for updating a reference image in the reference image memory.
If the update means is provided, it is detected by the illuminance detection means.
(E.g., morning, noon)
Around, in the evening, etc.), the imaging means (industrial camera, etc.)
The reference image for comparison with the monitoring input image is
Automatically updates to a reference image suitable for the current illumination level
Monitoring without being affected by changes in illuminance in the monitored area.
A change that has occurred in the viewing area can be detected.

Further, a peripheral monitoring image processing apparatus according to the present invention
In the decision that caused the first warning means to output the warning
A video storage means for storing the monitoring input image used,
When the number of matching matrix elements is greater than or equal to a predetermined value
May be provided with a second warning means for outputting a warning.

The operation will be described in more detail.

The illuminance value detected by the illuminometer has exceeded a threshold value
If it falls or falls below, the image taken by the camera
Each color component image separated from
Divide into matrices by the division unit (one frame
Number of pixel groups), and each pixel group is assigned “X-Y number”.
Landing ”. The image information of each of these color component images is
Information is shared with the XY address information of each pixel group.
Then, it is stored in the reference image memory as a reference image.

That is, the reference image memory always has
The base of each color component suitable for the current illuminance value of the surrounding environment of the viewing area
The quasi-image is registered by being divided into a matrix .

Also, images captured by the camera are sequentially captured.
Each color component image separated from that image (hereinafter, each color component image)
Input monitoring images) are
Divide into matrices by division unit. And each pic
"X-Y addressing" is performed on the xell group.

In the comparison operation unit, each color
For each component, between the input monitoring image and the reference image,
For each of the associated pixel groups,
The K value is executed by (1).

[0024]

(Equation 1) Here, a ₁ , a ₂ ..., _An are the pixel groups of the reference image.
The density values of the included pixels, b ₁ , b ₂ ...., b
_n is the number of pixels contained in the pixels of the input monitoring image.
This is the density value of the cell.

When K = 1, the input monitor image
Pixel groups corresponding to the input monitoring image and the reference image are completely the same when K <1 .
The corresponding pixel groups do not match. Therefore, the comparison operation
Knit, if K <1 , some change in the monitored area
(Deformation or disappearance of the monitored object, invasion of foreign matter, etc.) is determined.

[0026]

In the comparison operation unit, all the color components
Is determined as a match / mismatch between each corresponding pixel group.
Then, in the general operation unit, all colors
Majority decision (“2 out of
3 "), the match / mismatch between each corresponding matrix element
Determine a match. And finally decides that there is a mismatch
Of monitored objects based on the number of pixel groups
Depending on the extent of the change, you can set an alarm and
Start recording on the recorder.

[0028]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows an example of the configuration.

This configuration comprises an ITV camera 1, an image processor 2, an illuminometer 3, an illuminance pulser 4, and a video recorder 5.
Is configured.

Further, the image processor 2 includes an A / D converter 9, a matrix division unit (MTX), a reference image memory 12, a comparison operation unit 13, and a total operation unit 1.
4, a keyboard 8, a CPU 6, and an input / output unit (also referred to as "I / O-U") 7.

Further, the camera measurement position information storage section may be connected to the input / output unit 7. The main storage unit is a unit that stores information on which monitoring location the input image is an input image of, and how to use the information will be described later.

The ITV camera 1 outputs a video output signal
An RGB signal is output, and the output signal is input to an A / D converter 9 provided in the image processor 2. The RGB signals are recorded on the video recorder 5 when necessary.

On the other hand, the illuminometer 3 and the illuminance pulser 4 output I / O as a P signal output from the illuminance pulser 4.
-Input to U7.

The method of generating the P signal will be described later with reference to FIG.

The control of the entire apparatus according to the present embodiment
U6 performs a synchronization signal (Sy
n) is also given to manage the entire timing chart. The A / D-converted image data for each of R, G, and B is transmitted via the memory bus 10 and
Via the matrix division unit 11, it is stored in the reference image memory 12, the comparison operation unit 13, and the total operation unit 14 as appropriate. The keyboard 8 is an input means at the time of an external operation, and is used, for example, for starting the apparatus.

FIG. 2 is an explanatory time chart for explaining the operation of the present invention. Although the registered image [input I] 15 is stored in the reference image memory 12 in advance, initially, the initially registered image is stored in the reference image memory 12 via, for example, the keyboard 8.

Next, the input comparison image [input I
I] 16 is subjected to A / D conversion processing through the same A / D converter 9, but in the comparison operation unit 13, [input I]
By comparing the [Input II] 16 with reference to the registered image, the “match” of the image data for each pixel,
Alternatively, an arithmetic process for determining “mismatch” is performed, and the determination is repeatedly performed at regular time intervals t ₁ , for example.

However, when the ambient illuminance in the visual field of the camera image changes, the illuminance pulse output device 4 generates a P signal based on the electric signal output from the illuminometer 3, and the registered image [input I] is generated by the P signal. 15 is automatically updated. As a result, the next comparison image input is subjected to comparison calculation processing using the newly updated registered image as a reference image, so that the registered image is output to the illuminometer output which is output momentarily. It has features that will be newly updated based on it.

Thus, for example, even when the background has a large contrast change, such as an outdoor surveillance image,
It has become possible to provide an effective means for accurately monitoring an object.

FIG. 3 shows how the P signal is generated based on the output of the illuminometer.

The input of the illuminance meter 3 is represented by a lux (lx) value. The output [I] when the value is from 0 to 10 (lx) and the output [I] when the value is from 10 to 1,000 (lx).
I], output at 100 to 10000 (lx) [II
I].

In order to realize such a configuration, for example, a required number of comparators 17 using operation amplifiers, resistors and the like are provided, and the threshold values for each of the comparators are set to the above [I] and [I]. II], [II
I], the output of each comparator is logically OR-operated, and the pulse output circuit 18 generates a P signal, whereby the illuminance pulser 4 can be realized.

With this configuration, a predetermined pulse output is obtained every time the illuminance changes between [I] and [III]. For example,
When the illuminance takes a value between 10 (lx) and 1000 (lx), the same pulse output is obtained, and the same output pulse is obtained for a slight change in illuminance.

In consideration of the above description, the operation of the image apparatus according to the present invention will be described with reference to FIG.

FIG. 4 illustrates a case where, for example, an R signal is input to the A / D converter 9.

"A / D-R" in the figure means an A / D converter for an R (red) signal. In this embodiment,
The configuration is such that an AGC 19 is provided. Here, the AGC is an automatic gain controller.

In this embodiment, the registered image and the input image are divided into a “16 × 16” matrix, and a total of 256 pixels are compared.

Of course, the matrix division of “16 × 16” is only an example, and generally “m × n” (m and n are integers) such as “32 × 32” is possible.

Further, in this embodiment, the monitoring is not limited to one location, and monitoring is performed at a total of 32 monitoring locations. Needless to say, the number of monitoring points is not limited to 32.

For example, in a power plant, it is necessary to sequentially monitor monitoring points dispersed in a vast site. In this case, for example, first, the transformer is monitored, and second, the circuit breaker is monitored.

In such a case, the image processing means according to the present invention may be mounted on a moving body, and the moving body may monitor predetermined monitoring points sequentially at predetermined time intervals. Conceivable.

In this embodiment, the moving body will not be described in detail. For example, there are various applications such as an autonomous robot running on a predetermined route and a moving body running on a rail provided in a plant. Can be considered.

[Input I] 100, which is an input image when the illuminometer generates the P signal, is converted by the matrix division unit 11 into one screen of 32 × 32 (matrix in FIG. 4).
Is divided into box reference 400), 16 × 16 (a 1 ~a n)
Of pixels. These total 10
Data of the 24 group of pixels, as the reference image M _1,
The information is updated and registered in the reference image memory 12. Similarly, other supervisors
For even viewing location, the registered image M _2, ...., updates M ₃₂
Register .

Similarly, reference images are registered at other monitoring locations, and finally, reference images M ₁ to M ₃₂ are registered in the reference image memory 12.

Then, as the monitoring input image, the input image [input
Force II] is input,The input image [Input II]Ma
By the same processing via the trix splitting unit 11,
AndOne screen is divided into 32 × 32, " 16x16 "
(b ₁ ~ B _n ) Pixel group matrixDivided.
In addition, regarding which monitoring location the input image is
Is determined based on the camera measurement position information storage unit.
You. For example, a counter that can count from 1 to 32
To be counted up by 1 each time the patrol goes
By configuring to reset when it reaches 32,
During a tour of 32 locations, it was decided which of the monitoring locations was the input image.
You can judge.

Now, the monitoring input image is divided into a matrix.
First, each pixel group of the monitoring input image is
Then, each performs a K operation (referred to as the operation shown in Equation 1).

Further, in the comparison operation unit 13,
When K = 1, the pixel group of the monitoring input image and the registered image
Since there is a corresponding pixel groups of M ₁ are the same image, the determination of the no condition change (determination OK) is.

When K <1, there is a mismatch between the input image and the reference image, so it is determined that a change has occurred in the contents of the reference image, which is a registered image, and the input image.

When K <1, for example, K may be input and set as a threshold value within a range of "0.80 to 0.99" (for example, it may be operated by the keyboard 8). Although it is possible, the closer K is to 1, the smaller the change, the more it is determined that they do not match.

[0061]

In this case, unmatched pixels within one screen
According to the total number of file groups (see the layout in 400 in FIG. 4) ,
For example, it is also possible to estimate the magnitude of a change in a device to be monitored (for example, damage to an insulator in a transformer). This is because the visual field range within one screen is determined in advance, and the magnitude of the change is determined based on the number of unmatched pixel groups .

Although the R signal processing has been described above, the same processing (not shown) is performed on the (G) and (B) signals.

Therefore, in the general operation unit 14,
First, using the “2 out of 3” logic,
RGB determination for the pixel group determined to be
The majority of the results decides again whether they match or not, and
An alarm (ALARM (1)) is output accordingly.

When an alarm (ALARM (1)) is output, the video recorder 5 (for example, VTR, etc.) is activated in conjunction with the output, and the video at the time of the discrimination is determined.
Later, the maintenance staff can perform more detailed analysis on the monitor based on rules of thumb.

As the warning means, warning means using sound such as alarm, warning means using light such as LED, CRT
Various things such as displaying a message on a screen are possible.

Further, when the number of mismatching pixel groups is equal to or more than a predetermined number, for example, three or more, eight or more, an alarm output (ALARM (2)) is performed by the mismatching pixel number determination circuit 22. What is necessary is just to make it the structure which calls attention of a maintenance person.

The "predetermined number" may be changed according to the purpose. For example, a certain value may be set for intrusion of a person or the like, and another value may be set for breakage of a transformer insulator. This makes it possible to estimate and classify the occurrence of a desired abnormal situation.

The same applies to other monitoring points.
Next, K operation is performed for each pixel group of the monitoring input image.
Based on the resulting number of mismatched pixels.
Output the video . As described above, according to the present invention, the following effects can be obtained.

First, when three RGB signals of video output signals from an ITV camera or the like are input, each image processing is performed in parallel, and a comprehensive judgment is made after multi-value processing. Image processing determination with higher sensitivity than before.

Second, since the reference image for image processing is automatically updated in response to a change in the illuminance of the external environment, accurate processing can be performed especially outdoors.

Third, when the number of unmatched pixels in one screen (one frame) exceeds a predetermined threshold value due to the multi-value processing, this is used as a trigger signal to generate an alarm. It is now possible to output.

That is, within the viewing angle, the divided pixels
The size of the file group does not match because it corresponds to the measured size
Based on the number of pixel groups , it is possible to estimate and classify occurrence events such as intrusion of a person and deformation of a structure (main engine).

Fourthly, together with the detection of the mismatch of the pixel group,
Since the alarm can be sounded and the video recorder can be activated, the video at the time of the determination and the subsequent video can also be recorded, so maintenance personnel only have to make a visual determination only at the time of the alarm output. A processing device can be provided.

[0075]

In the peripheral monitoring device of an important plant facility or the like, the RGB signals of the ITV camera are multivalued independently of each other, the correlation function is calculated, and the mismatched pixel group is obtained.
By outputting alarms and collecting video records based on the number of the mismatched pixel groups, it is possible to facilitate maintenance personnel monitoring and provide an image processing means for accurately determining event change. did.

[Brief description of the drawings]

FIG. 1 is a hardware configuration diagram of an embodiment according to the present invention.

FIG. 2 is an explanatory diagram of a time chart of automatic updating of a registered image.

FIG. 3 is an explanatory diagram of generation of a P signal based on an output of an illuminometer.

FIG. 4 is an explanatory diagram showing a procedure of image processing.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... ITV camera, 2 ... Image processor, 3 ... Illuminometer, 4 ...
Illuminance pulse output device, 5: video recorder, 6: CPU, 7
... Input / output unit, 8 keyboard, 9 A / D converter, 10 memory bus, 11 matrix division unit, 12 reference image memory, 13 comparison operation unit,
14: Comprehensive arithmetic unit, 17: Comparator, 18:
Pulse output circuit, 19 AGC, 21 three-input determination circuit, 22 mismatched pixel number determination circuit

Continuation of the front page (58) Field surveyed (Int.Cl. ⁶ , DB name) H04N 7/18 G01D 21/00 G06T 1/00 G08B 25/00

Claims (5)

(57) [Claims] 1. An image processing method according to claim 1 , further comprising the steps of:
Peripheral monitoring image processing apparatus for monitoring the monitoring area.
A reference image storing a reference image divided into a matrix
A memory that stores the image captured by the imaging unit as a matrix of the reference image;
Matrix to divide into matrices corresponding to
Dividing means and a matrix by the matrix dividing means
The captured images are sequentially captured as monitoring input images,
Each associated with the visual input image and the reference image
One matrix for each matrix element
Pixel data a _1, a ₂ constituting elements, ..., and a _n, other
Pixel data b ₁ , b ₂ ,.
by using the b _n, [number 1] The K value is sequentially calculated by the
Comparison operation to determine the match / mismatch of response matrix elements
Means for outputting a warning based on the determination result of the comparison operation means.
And a first warning output device. 2. A peripheral monitoring image processing apparatus according to claim 1, wherein
Therefore, a mismatch that occurs between the reference image and the monitoring input image
Unmatched pixel count detection to calculate the number of matrix elements
Output means, and the mismatched matrices detected by the mismatched pixel group number detection means.
Warning when the number of ricks elements is greater than or equal to a predetermined value
Peripheral monitoring image processing device, comprising: a second warning unit that outputs 3. The peripheral monitoring image processing according to claim 1 or 2.
An apparatus used for determining that the first warning means has output the warning.
A peripheral surveillance image processing apparatus comprising: a video storage unit for storing a monitored input image . 4. The method as claimed in claim 1, wherein said first, second, and third ones are selected from the group consisting of:
A near monitoring the image processing apparatus, when further comprises a comprehensive calculation means, the imaging means is an image shot is a color image
In addition, the reference image memory stores each color component as the reference image.
The matrix dividing unit stores the reference image of
Each color component image separated from the image is matrix
And the comparison operation means performs, for each color component,
The K value of the corresponding matrix element is sequentially calculated, and
Successively determining the match / mismatch between the trix elements,
The overall calculation means determines by the comparison calculation means for each color component.
Of matching / mismatching results between each corresponding matrix element
By matching, the correspondence between the corresponding matrix elements /
Determining a disagreement, wherein the first warning means responds to the determination of the general calculation means.
A peripheral image monitoring processor that outputs the warning . 5. The method as claimed in claim 1, wherein said first, second, third, and fourth aspects are selected.
A periphery monitoring image processing apparatus, comprising: a luminance detection means for detecting the illuminance in the monitoring area, Thailand determined in accordance with the illuminance value the illuminance detecting means detects
The matrix by the matrix dividing means.
Captures the image divided into boxes, and according to the image,
Reference image update for updating a reference image in the reference image memory
Surroundings monitoring image processing apparatus characterized by comprising: means.