JP2016134636A - Angle-of-view deviation correction device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an angle-of-view deviation correction device capable of detecting an angle-of-view deviation due to an annual change after instillation of a camera, and achieving an estimation of an angle-of-view deviation.SOLUTION: An angle-of-view deviation correction device 1 corrects an angle-of-view deviation of a camera by using a reference image obtained by photographing a predetermined place with the camera and an object image obtained by photographing the place similar to the reference image. The angle-of-view deviation correction device includes: an edge extraction part 20 that extracts an edge of the reference image and the object image; a difference image generation part 50 that generates a difference image constructed at a difference absolute value of each pixel by using the edge extraction images of the reference image and the object image; deviation amount evaluation part 70 that calculates a deviation amount, on the basis of the deviation image, of which an evaluation value becomes the smallest value as that the sum of absolute values of the deviation value included in each pixel; and an image correction part 80 that corrects the angle-of-view deviation of object image on the basis of the calculated deviation amount.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a video display system that displays video taken over a long period of time, and more particularly, to a correction technique that corrects a display shift when a field angle of a camera to be shot shifts.

  In recent years, many techniques for correcting a display shift of a moving image have been seen. For example, Patent Document 1 discloses a technique for performing image pattern matching using a plurality of template patterns, obtaining a movement vector, and correcting a displayed image shift. Further, Patent Document 2 discloses an image processing apparatus capable of calculating parameters for correcting a large image shake with high accuracy even when the accuracy of a sensor that measures the movement of the photographing apparatus is low. ing.

  As in Patent Document 1, in order to calculate an image shift (movement vector) using pattern matching, it is assumed that pixel values indicating the same object in the image do not change. When the pixel value of the same object changes, in the pattern matching, the pixel value difference becomes too large, and it is difficult to estimate that the same object is the same object. However, when the purpose is to correct shaking such as camera shake, there is almost no shift in the time taken for the pattern matching image (within 1 second), and the pixel value of the same object changes greatly. The image shift can be detected.

Japanese Patent No. 4040651 Japanese Patent No. 5204785

In the method described in the background art described above, there is no problem in detecting the angle of view by comparing images in a short time, such as shake correction. However, this is not always effective for changes in the angle of view of an image, such as the camera turning downward due to secular change after installation of the camera.
For example, in a phenomenon in which the camera turns down one pixel over a month (30 days), when comparing images for about 1 second, the angle of view shift in 1 second is 1/2, 592. 000 pixels, and the angle of view cannot be detected.

  In addition, in the above case, it takes several days or more to detect the deviation of the angle of view, and the brightness situation changes due to the difference in the weather conditions of the comparison image in that case, and the same object is changed. Since the pixel value changes, it is difficult to estimate the same object by pattern matching using the pixel value. Therefore, it becomes difficult to detect a shift amount by pattern matching using pixel values of the same object, and it is difficult to detect a view angle shift.

  The present invention has been made in view of such conventional circumstances, and is capable of detecting a field angle deviation due to a secular change after camera installation and capable of estimating an amount of field angle deviation. An object of the present invention is to provide a deviation correction device.

  In order to achieve the above object, an angle-of-view correction apparatus according to the present invention includes a reference image obtained by photographing a predetermined location with a camera and a target image obtained by photographing the same location as the reference image with the camera after taking the reference image. An angle-of-view deviation correction apparatus that corrects an angle-of-view deviation of the camera by using an edge extraction unit that extracts edges of the reference image and the target image, and edge extraction of the reference image extracted by the edge extraction unit Using the image and the edge extraction image of the target image, based on the difference image generation unit that generates a difference image composed of the absolute difference value of each pixel, and the difference image generated by the difference image generation unit, An absolute value sum of difference values for each pixel is used as an evaluation value, and a deviation amount evaluation unit that calculates a deviation amount that minimizes the evaluation value; and based on the deviation amount calculated by the deviation amount evaluation unit, the target image Without angle of view And having and an image correcting unit that corrects.

  In order to achieve the above object, the angle-of-view correction apparatus according to the present invention includes a reference image obtained by photographing a predetermined place with a camera, and a target image obtained by photographing the same place as the reference image with the camera after taking the reference image. The angle-of-view deviation correction apparatus for correcting the angle-of-view deviation of the camera using the edge extracting unit for extracting the edge of the reference image and the target image, and the reference image extracted by the edge extracting unit Using the binarization processing unit that binarizes each of the edge extraction image and the edge extraction image of the target image, and the two edge extraction images binarized by the binarization processing unit, Measuring a distance between a pixel to be an edge and an edge portion of the reference image, and generating a mapping image in which a measurement result is mapped for each pixel; and the edge comparison unit Based on the tapped image, the absolute value sum of the difference values for each pixel is used as an evaluation value, and a deviation amount evaluation unit that calculates a deviation amount that minimizes the evaluation value, and the deviation amount calculated by the deviation amount evaluation unit And an image correction unit that corrects the angle-of-view shift of the target image.

  Further, in the above-described field angle deviation correction device, the edge comparison unit obtains an edge density of each of the two edge-extracted images subjected to the binarization process, and a portion having a high edge density is excluded from the comparison of the edge portions. And

  According to the present invention, it is possible to provide a field angle deviation correction apparatus that can detect a field angle deviation due to a secular change after the camera is installed and can estimate the field angle deviation amount.

It is a block diagram which shows an example of a structure of the view angle deviation correction apparatus which concerns on one Embodiment of this invention. It is a figure which shows the image image | photographed on the basis of a certain weather condition. It is a figure which shows the image image | photographed on the basis of the weather conditions different from FIG. It is a figure which shows the edge extraction image which extracted the edge based on the image of FIG. 2 and FIG.

Hereinafter, an angle of view correction apparatus according to an embodiment of the present invention will be described.
An angle-of-view correction apparatus according to an embodiment of the present invention includes a storage area in which data is not lost even when the power is turned off, and stores an initial image taken at the time of camera installation and a normal image taken at the time of camera operation in the storage area. On the other hand, an edge extraction image obtained by extracting the edge portion of each image is stored, and the edge extraction image of the initial image is compared with the edge extraction image of the normal image taken during camera operation, and the amount of deviation of the camera angle of view is calculated. The angle of view of the normal image is corrected based on the calculated amount of deviation of the angle of view.

[Configuration of field angle correction device]
Next, the configuration of the field angle deviation correction apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing an example of the configuration of a field angle deviation correction apparatus according to an embodiment of the present invention.
As shown in FIG. 1, the field angle deviation correction apparatus 1 according to an embodiment of the present invention includes an image input unit 10, an edge extraction unit 20, a binarization processing unit 30, an image recording unit 40, and a difference. The image generation unit 50, the edge comparison unit 60, the shift amount evaluation unit 70, and the image correction unit 80 are configured.

The image input unit 10 has a function of capturing an image (p1) from a camera (not shown), a function of passing the captured image to the edge extraction unit 20, and a function of passing the captured image to the image correction unit 80.
The edge extraction unit 20 has a function of performing edge (boundary) extraction on the image passed from the image input unit 10 by performing a difference calculation for a change in pixel value. Further, the edge extraction unit 20 has a function of passing the edge-extracted edge-extracted image to the binarization processing unit 30 and a function of reading and writing the edge-extracted image with the image recording unit 40.

The binarization processing unit 30 has a function of binarizing the edge extraction image passed from the edge extraction unit 20 and a function of reading / writing the binarized image from / to the image recording unit 40. The binarization processing unit 30 has a function of passing the binarized processing image to the difference image generation unit 50 and the edge comparison unit 60. At that time, the binarization processing unit 30 passes both the normal image (target image) currently being taken and the initial image (reference image) recorded in the image recording unit 40 passed from the edge extraction unit 20. .
The image recording unit 40 has a function of recording the edge extraction image passed from the edge extraction unit 20 and the binarization processing image passed from the binarization processing unit 30. In addition, the image recording unit 40 receives the edge extracted image or binarization recorded in the edge extracting unit 20 or the binarizing processing unit 30 in response to a request from the edge extracting unit 20 and the binarizing processing unit 30. It has a function of passing a processed image. Further, the image stored in the image recording unit 40 is treated as a reference image in the processing in the difference image generation unit 50 and the edge comparison unit 60.

The difference image generation unit 50 uses the target image and the reference image passed from the binarization processing unit 30 to generate a difference image configured with the absolute difference value of each pixel. The generated difference image is transferred to the deviation amount evaluation unit 70.
The edge comparison unit 60 uses the target image and the reference image passed from the binarization processing unit 30 to measure the distance between the pixel that is the edge of the target image and the edge portion of the reference image, and the measurement result is the pixel A mapped image is generated every time. The generated mapping image is transferred to the deviation amount evaluation unit 70.

The deviation amount evaluation unit 70 uses the output result of the difference image generation unit 50 when using the edge difference image, and uses the output result of the edge comparison unit 60 when using the edge distance. The deviation amount evaluation unit 70 calculates an evaluation value based on the sum of absolute values of the values for each pixel based on the passed result, and the difference image generation unit 50 and the edge comparison unit 60 provide difference images having different deviation amounts. In addition, the generation of a mapping image is instructed. There are various methods for determining the amount of deviation to be instructed. For example, a method of evaluating all the ranges of a certain amount of deviation (both horizontal and vertical) can be considered. A plurality of deviation amounts are evaluated, and the deviation amount having the smallest evaluation value is adopted. The adopted deviation amount is transferred to the image correction unit 80.
The image correction unit 80 corrects the shift of the image delivered from the image input unit 10 according to the shift amount input from the shift amount evaluation unit 70, and the corrected image (p2) is displayed on a display unit or the like (not shown). Output.

[Specific angle-of-view correction processing in the angle-of-view correction device]
Next, a specific view angle shift correction process in the view angle shift correction apparatus according to the embodiment of the present invention will be described with reference to FIGS. 1, 2, 3, and 4. FIG. 2 is a diagram showing an image taken under certain weather conditions. FIG. 3 is a diagram showing an image taken under a weather condition different from that in FIG. FIG. 4 is a diagram illustrating an edge extracted image obtained by performing edge extraction based on the images of FIGS. 2 and 3.
The images in FIG. 2 and FIG. 3 are the same images obtained by photographing a kite along the road with a camera, and the dates and weather conditions and the like are different. An image 501 in FIG. 2 includes a road portion 111 and a ridge portion 101, and the image 502 in FIG. 3 is different from the image 501 in FIG. And part 102.

  In the images of FIGS. 2 and 3, there is a difference in pixel values between the ridge portion and the road portion, respectively, and the boundary is clearly observed. In addition, there is a vertical column in the ridge part, and there is a difference in pixel value in that part, and the boundary is clearly observed. However, comparing the images of FIGS. 2 and 3, since the sunlight is inserted in different directions, the pixel values of the road portions 111 and 112 and the eaves portions 101 and 102 are different. In this case, in pattern matching using pixel values, it is difficult to evaluate that both the road portions 111 and 112 and the fence portions 101 and 102 are the same object.

  Therefore, the edge extraction unit 20 extracts an edge (boundary) by performing a difference operation on the change of the pixel value for the image input from the image input unit 10. For edge extraction, an image can be obtained by applying an image filter to the target image. Some image filters for edge extraction use a Canny method, a Sobel filter, a Roberts filter, a Laplacian filter, and the like. When applying a filter to two images to be compared, the same filter is used.

As a result of performing edge extraction in both the images of FIGS. 2 and 3 by the edge extraction unit 20, an edge extraction image 121 as shown in FIG. 4 can be obtained. The edge extraction image 121 is an image having gradation according to the edge state of the image. As it is, the result of comparing the edge states by the difference by the processing in the difference image generation unit 50 may be used. In that case, the deviation amount that minimizes the evaluation value is the deviation amount of the images to be compared, with the absolute value sum of the difference values as the evaluation value.
Focusing on the position of the edge, the edge extraction image extracted by the edge extraction unit 20 is binarized with a threshold value by the binarization processing unit 30, and evaluation is performed using the binarized edge extraction image. May be. Also in that case, the sum of absolute values of the differences between the values of the two binarized edge extracted images is used as the evaluation value, and the amount of deviation that minimizes the evaluation value is the amount of deviation of the images to be compared.

  Note that the edge extracted image is not used as it is, but can also be used after being processed. In the edge extracted image, when the density of edges is increased, there is a possibility that an overlap occurs in a portion other than the same edge portion. For this reason, the evaluation may be performed using only a portion having a low edge density while excluding a portion having a high edge density.

The edge density is calculated as follows.
In the binarized edge extraction image, for example, a non-edge portion can be set to 0, and an edge portion can be set to 1. At this time, in the range of ± 16 pixels around the target pixel, the average value of the edge extracted image values is calculated. If the average value is less than 0.1, it is determined that the edge density is low, and the edge portions are compared. I do. Further, when the average value is 0.1 or more, it is determined that the edge density is high, and is excluded from the comparison of the edge portions.

The determination of the edge density is not necessarily performed by the above method, and other methods may be used.
In the image state, the pixel change is not always steep, and depending on the brightness state, there is a possibility that an edge is not generated at the same position. In this case, the evaluation is performed using the edge distance. The edge distance is as follows.
At the time of image comparison, two images to be compared are set as a reference image and a target image. For example, in the case of field angle deviation correction, an initial image at the time of camera installation is set as a reference image, and an image captured at the time of field angle deviation check at the time of camera operation is set as a target image.
Using the reference image as a reference, the pixel position determined to be the edge of the edge extracted image as the target image and the pixel position having the closest positional relationship among the pixel positions determined as the edge on the reference image used as a reference are used. The straight line distance is the edge distance. An average value of edge distances of pixel positions determined to be edges is used as an evaluation value, and a shift amount that minimizes the evaluation value is a shift amount of images to be compared.

  As described above, according to the view angle deviation correction apparatus according to the embodiment of the present invention, it is possible to detect an angle of view deviation due to secular change after installation of the camera, and realize estimation of the amount of view angle deviation. Thus, it is possible to provide a field angle deviation correction apparatus that can

  It should be noted that the configuration and operation of the above-described embodiment are examples, and it goes without saying that the embodiment can be appropriately modified and executed without departing from the spirit of the present invention.

1: angle of view correction apparatus, 10: image input unit, 20: edge extraction unit, 30: binarization processing unit, 40: image recording unit, 50: difference image generation unit, 60: edge comparison unit, 70: deviation Quantity evaluation unit, 80: image correction unit, 101, 102: hail portion, 111, 112: road portion, 121: edge extracted image, 501: image, 502: image.

Claims (3)

An angle-of-view deviation correction device that corrects an angle of view deviation of the camera using a reference image obtained by photographing a predetermined place with a camera and a target image obtained by photographing the same place as the reference image after the reference image is taken. There,
An edge extraction unit that extracts edges of the reference image and the target image;
A difference image generation unit that generates a difference image composed of absolute difference values of each pixel using the edge extraction image of the reference image extracted by the edge extraction unit and the edge extraction image of the target image;
Based on the difference image generated by the difference image generation unit, an absolute value sum of difference values for each pixel is used as an evaluation value, and a deviation amount evaluation unit that calculates a deviation amount that minimizes the evaluation value;
An image correction unit that corrects a field angle shift of the target image based on the shift amount calculated by the shift amount evaluation unit;
An angle-of-view correction apparatus characterized by comprising: An angle-of-view deviation correction device that corrects an angle of view deviation of the camera using a reference image obtained by photographing a predetermined place with a camera and a target image obtained by photographing the same place as the reference image after the reference image is taken. There,
An edge extraction unit that extracts edges of the reference image and the target image;
A binarization processing unit that binarizes each of the edge extraction image of the reference image extracted by the edge extraction unit and the edge extraction image of the target image;
Using the two edge extraction images binarized by the binarization processing unit, the distance between the pixel as the edge of the target image and the edge portion of the reference image is measured, and the measurement result is obtained for each pixel. An edge comparison unit for generating a mapped mapping image;
Based on the mapping image generated by the edge comparison unit, an absolute value sum of difference values for each pixel is used as an evaluation value, and a deviation amount evaluation unit that calculates a deviation amount that minimizes the evaluation value;
An image correction unit that corrects a field angle shift of the target image based on the shift amount calculated by the shift amount evaluation unit;
An angle-of-view correction apparatus characterized by comprising: 3. The field angle deviation correction apparatus according to claim 2, wherein the edge comparison unit obtains an edge density of each of the two edge extracted images subjected to the binarization process, and excludes a portion having a high edge density from the comparison of the edge portions. An angle of view correction apparatus characterized by the above.