JP4559043B2 - Image monitoring device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image monitoring apparatus, and in particular, image monitoring that can be used for measuring a property of a moving image at a low load, cutting out a partial image (animal body image, etc.) from a base image, and correcting an image such as brightness correction. Relates to the device.
[0002]
[Prior art]
As a conventional device of this type, for example, there is one described in JP-A-8-55220. As described in this publication, methods for extracting a moving object region in a moving image include a background difference method, a method using image features, and the like.
[0003]
In the background difference method, an image corresponding to the background is captured and accumulated in advance, the input image is compared with the accumulated background image, a difference between the two images is calculated, and a setting value with this difference is present. At the above time, it is determined that the image changes due to the moving object.
[0004]
On the other hand, the method using the feature of the image includes a feature amount extracting unit that extracts an image feature amount for each pixel or each small region from the image of each frame of the moving image, and the past frame obtained by the feature amount extracting unit. The feature quantity of the current frame is estimated in consideration of the change of the imaging environment by obtaining the tendency of the feature quantity in time series with reference to the feature quantity of the current frame, and further, the current frame obtained by the feature quantity extraction unit The feature change amount is obtained by taking the difference between the feature amount of the current frame and the estimated feature amount of the current frame, and the moving object is determined based on the feature change amount.
[0005]
[Problems to be solved by the invention]
In the former background subtraction method, since there is no compensation for changes in the entire screen or background portion, it is impossible to distinguish between changes due to moving objects and changes in background luminance. In addition, since the actual moving object and the background are diverse, when the luminance value of the moving object and the luminance value of the background are close to each other, the detection of the moving object may be erroneously detected, resulting in a decrease in reliability. There is a problem.
[0006]
On the other hand, the latter method using the feature of the image has a problem that the calculation load is heavy because the feature amount of the entire screen is measured.
[0007]
In particular, if the above method is applied to an omnidirectional camera image or 360 ° omnidirectional image having a large amount of data of one frame, there is a problem that the calculation load is further increased and the processing speed is decreased.
[0008]
The present invention has been made in view of the above-described problems of the prior art, and an object of the present invention is to enable image change detection with high accuracy and low computational load in continuous video monitoring. It is to provide a monitoring device. Another object of the present invention is to provide an image monitoring apparatus capable of correcting the luminance change of the entire video with a low calculation load.
[0009]
In order to achieve the above-described object, the present invention provides an image monitoring apparatus for detecting a change in a screen, a sampling block selection unit that samples a part of the block in the screen, and the past several frames related to the screen. For each pixel value, a signal value having a high frequency in time is obtained, and a reference image creating unit that uses the signal value as a reference image for each pixel value, and brightness correction of the screen is performed in comparison with the reference image. A screen offset correction unit to perform, and a significant / insignificant block sorting unit that performs significant and non-significant classification on the sampled block, and the screen offset correction unit is selected by the sampling block selection unit Difference absolute value sum, average of at least one of luminance signal and color difference signal with respect to the sampled block and the reference image at the same position Obtains a feature amount such as multiplication error, it carried an offset correction to give the same effect to the entire image frame when the sampling block having an equivalent feature quantity in the entire frame is present more than a certain number, the significant and non-significant block fractionation The unit compares the feature amount of the sampling block with a predetermined threshold value, classifies it into a significant block or an insignificant block according to the magnitude, and cuts out a significant region including the significant block after the screen offset correction. This is the first feature.
[0010]
According to this feature, a significant area in the screen can be obtained with a low calculation load. In addition, a significant region image can be cut out at high speed. In addition, it is possible to correct the luminance change of the video with a low calculation load.
[0013]
Further, the significant region has a second feature in that the significant region is obtained by searching for a significant block in the surrounding blocks .
[0014]
According to this feature, a significant region in the screen can be obtained with a low calculation load and can be cut out at high speed.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 4 is a conceptual diagram of an omnidirectional camera, and FIG. 5 shows an example of a screen in which image data captured by the omnidirectional camera is displayed on a display such as a personal computer.
[0016]
Although the omnidirectional camera is well known and has various types, the omnidirectional camera 1 of FIG. 4 has a hemispherical or bowl-shaped omnidirectional mirror provided on the upper surface inside the transparent acrylic cylinder 2. 3, a center needle 4 extending downward from the center of the omnidirectional mirror 3, a CCD camera 5 and a cable connector 6 provided below the center needle 4. Since this camera is well known, detailed description of its operation is omitted.
[0017]
When the omnidirectional photographing camera 1 is connected to a personal computer or the like via the cable connector 6 and an image photographed by the omnidirectional photographing camera 1 is displayed on a display, a screen 11 as shown in FIG. 5 is obtained. On the screen 11, a donut-shaped 360 ° omnidirectional image 11a that is a captured image of the omnidirectional camera 1 itself, an image 11b obtained by developing the donut-shaped 360 ° omnidirectional image 11a in a belt shape, that is, a panoramic developed image, A normal one-frame-sized image 11c obtained by cutting out a part of the image 11b is displayed. 11d is a cut-out image of the image 11c. The image 11c can be distributed to, for example, a mobile phone.
[0018]
The image obtained from the omnidirectional camera 1 is a moving image of 15 frames / second, for example, in the same manner as a normal PC camera. Since the developed image 11b is many times larger than a normal one frame size such as a PC camera image, for example, the method of determining the moving object by measuring the feature amount of the full screen as described in the prior art requires a computation load. Increases, and the processing speed decreases greatly. Further, since the omnidirectional camera 1 continues to shoot indoors or outdoors all day, the whole image becomes dark uniformly when it becomes dark in the evening or at night. For this reason, it is necessary to correct the brightness of the screen. However, since this correction also requires a large amount of data on the entire screen, the calculation load increases if normal screen correction is performed.
[0019]
The present invention has been made to solve such problems, and an embodiment thereof will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of an embodiment of the present invention.
[0020]
From the base image input unit 21, the developed image 11b is input. The sampling block selection unit 22 selects, by sampling, a block (for example, 16 × 16 pixels) for performing feature measurement for change detection on the base image input from the base image input unit 11. The position and density of the block to be sampled are variable according to the time given from the outside and the frame information. For example, as shown in FIG. 2 from A → B → C → D → A →..., Several frames from a certain time to a certain time select the sampling block A, and from the next time to the next certain time. A sampling block B is selected for several frames. Similarly, sampling blocks C and D are selected every several frames, and this selection cycle is repeated. In this way, it is possible to periodically switch the positions and density patterns of the sampling blocks determined in advance with inertia of several frame times.
[0021]
The reference image creation / update unit 23 creates a reference image for change detection from the base image input from the base image input unit 21. Various algorithms can be used to create the reference image. For example, it is possible to obtain the most frequent signal value in time for each pixel value for the number of image frames and use this as a reference image.
[0022]
Next, the full screen offset measurement / correction unit 24 stores the reference block supplied from the reference image creation / update unit 23 or the frame buffer 25 with respect to the sampling block selected by the sampling block selection unit 22. The signals at the same position between the previous images are compared. The signal comparison calculation is not specified, but for example, only the luminance signal, or the sum of absolute differences (SAD) of the luminance signal and the color difference signal, or the mean square error (MSE) can be used. Hereinafter, the calculation value of the signal comparison is referred to as a feature amount. The full screen offset measurement / correction unit 24 performs full screen offset correction based on the signal comparison result. For example, when there are a certain number or more of sampling blocks having the same value as the feature value in the entire frame, it is possible to perform offset correction that gives the same effect to the entire image frame according to the feature value. . When the entire screen becomes dark uniformly in the evening or at night, the brightness of the entire screen can be increased by this offset correction so that the brightness of the screen is always kept constant.
[0023]
After the offset correction, the significant / non-significant block classification / significance rate measurement unit 26 performs the classification of the significant / non-significant blocks. For example, when the offset correction is not performed, the feature amount of each sampling block is used as it is and compared with a predetermined threshold value. If the feature amount is larger than the threshold value, the case is significant, and the other case is not significant. On the other hand, when offset correction is performed, the offset value is subtracted from the original feature value to obtain a new feature value, which can be made significant if it is large and otherwise non-significant by comparison with a threshold value. It is. Here, the significance rate measurement unit 26 calculates the significance rate, and when the significance rate is larger than a predetermined threshold, issues a command to the reference image creation / update unit 23 to update the reference image. . Thereby, the sampling block determined to be significant can be optimized.
[0024]
Further, regarding the block determined to be a significant block, the significant block periphery re-measurement unit 27 re-investigates the periphery thereof at a higher density as shown in FIG. There are various methods of re-investigation. For example, significant and non-significant are examined for the block A2 around 4 to 8 around the significant block [A], and further around the significant block [A2]. Significant or non-significant is examined for block A3 of ~ 8. The significant block periphery re-measurement unit 27 performs recursive loop measurement or stepwise measurement in this way.
[0025]
When a significant block is obtained in this way, the significant area cutout unit 28 extracts a significant area. Various methods are conceivable for extracting the significant region. For example, the significant region can be categorized based on the adjacency of the significant block, and each category can be separated. The significant partial image generation unit 29 generates an image of a significant part based on the separated significant region.
[0026]
As is clear from the above description, according to the present embodiment, (1) it becomes possible to always perform correction of the luminance change of the video and detection of the moving object with a low calculation load, and (2) the change portion of the video. It will be possible to specify with high accuracy, (3) It will be able to correct the overall change such as background brightness change and detect the change in the screen accurately, (4) Automatically appropriate An effective reference image can be created, and accurate changes can be detected with low load.
[0027]
Although one embodiment of the present invention has been described above, the processing units 22 to 24 and 26 to 28 in FIG. 1 can take various methods other than the present embodiment without departing from the spirit of the present invention. Of course.
[0028]
【The invention's effect】
According to the present invention, significant and non-significant classification is performed on a sampled block in the screen, and a significant area is detected based on the significant block. At a high speed.
[0029]
In addition, since the feature amount is obtained for the sampled block in the screen, and the offset correction that gives the same effect to the entire image frame is performed according to the feature amount, the offset correction is performed with a low calculation load. And can be performed at high speed.
[0030]
In addition, an appropriate reference image is automatically created, and accurate change detection can be performed with a low load.
[0031]
It should be noted that the present invention has a great effect when applied to an image having a large amount of data of one frame, such as a screen in which a 360 ° omnidirectional image is panorama developed.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of an embodiment of the present invention.
FIG. 2 is an explanatory diagram of an operation of a sampling block selection unit in FIG. 1;
FIG. 3 is an explanatory diagram of an operation of a significant block periphery re-measurement unit in FIG. 1;
FIG. 4 is an explanatory diagram of a configuration of an omnidirectional camera.
FIG. 5 is a diagram illustrating a display example when an image captured by an omnidirectional camera is displayed on a display.
[Explanation of symbols]
21 ... Base image input unit, 22 ... Sampling block selection unit, 23 ... Reference image creation / update unit, 24 ... Full screen offset measurement / correction unit, 26 ... Significant / non-significant block Classification, significance rate measurement unit, 27... Significant block periphery remeasurement unit, 28... Significant region segmentation unit, 29.

Claims (5)

In an image monitoring device that detects changes in the screen,
A sampling block selector for sampling some blocks in the screen;
For each pixel value for the past several frames related to the screen, a reference value creating unit that obtains a signal value with a high temporal frequency and uses the signal value as a reference image for each pixel value;
A screen offset correction unit that performs brightness correction of the screen in comparison with the reference image;
A significant / non-significant block classification unit that performs significant / non-significant classification on the sampled blocks;
The screen offset correction unit is characterized in that, for the sampling block selected by the sampling block selection unit and the reference image at the same position, a difference absolute value sum, a mean square error, etc. regarding at least one of a luminance signal and a color difference signal If there are more than a certain number of sampling blocks with the same feature amount in the entire frame, offset correction that gives the same effect to the entire image frame is performed,
The significant / non-significant block classification unit compares the feature amount of the sampling block with a predetermined threshold, and classifies the significant block or the non-significant block according to the size,
An image monitoring apparatus, wherein a significant region including a significant block after the screen offset correction is cut out. The image monitoring apparatus according to claim 1,
The image monitoring apparatus, wherein the sampling block selection unit periodically switches the position of the sampling block every plural frames. The image monitoring apparatus according to claim 1 or 2,
The significant / non-significant block classification unit calculates a significance rate, and if the significance rate is greater than a predetermined threshold value, instructs the reference image creation unit to update a reference image. Monitoring device. The image monitoring device according to any one of claims 1 to 3,
The image monitoring apparatus according to claim 1, wherein the significant area is obtained by searching for a significant block in a block surrounding the significant block. In the image monitoring device according to any one of claims 1 to 4,
The image monitoring apparatus according to claim 1, wherein the screen is a panoramic image of a 360 ° omnidirectional image.

Images