JP2009038558A - Correction apparatus for monitor camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To appropriately correctthe positional deviation of image caused by a change in a camera mounting position. <P>SOLUTION: A comparison pattern calculating section 6 extracts a first specific portion and a second specific portion on a captured image outputted from a camera 2 and calculates a comparison pattern by modeling a positional relation between these extracted first and second specific portions. An image correcting section 7 performs deviation correction upon the captured image in such a way that a position of the comparison pattern on the image matches a position of a reference pattern, on the image, read from a reference pattern storage section 5. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a correction device for a monitoring camera mounted on a vehicle, and more particularly, to correction of an image shift caused by a change in a camera mounting position.

  Patent Document 1 discloses an environment recognition device that detects a change in the mounting position of an in-vehicle camera. In order to detect this change in position, a mark (one point) that gives a positional reference is placed in the imaging area of the camera. As the mark, for example, a mark fixed on the windshield of the host vehicle, a vehicle inspection seal, a mascot attached to the hood of the host vehicle, or a part of the ceiling is used. Then, the position of the mark is sequentially extracted from the captured image output from the camera, and the amount of deviation from the original position is calculated. By performing image correction according to the amount of deviation, it is possible to correct changes in the camera mounting position caused by vibration during travel, contact with objects, changes over time, and the like.

JP-A-8-16999

  However, in Patent Document 1 described above, since a change in the camera mounting position is detected focusing on only a single part (one point), there is a problem in detection accuracy.

  Therefore, the present invention is to appropriately correct the image misalignment caused by the change in the camera mounting position.

  In order to solve such a problem, the present invention provides a correction device for a monitoring camera that includes a camera, a reference pattern storage unit, a comparison pattern calculation unit, and an image correction unit and is mounted on a vehicle. The camera outputs a captured image by capturing an image of the monitoring area. The reference pattern storage unit stores a reference pattern. The reference pattern is used to provide a positional reference when correcting the deviation of the captured image. The first specific part existing at a known position in the monitoring area and the first specific part in the monitoring area. This is a model of an initial positional relationship with a second specific part existing at a known position different from the part. The comparison pattern calculation unit extracts the first specific part and the second specific part on the captured image output from the camera, and calculates the extracted first specific part and second specific part. A comparison pattern modeling the positional relationship is calculated. The image correction unit corrects the deviation of the captured image so that the position of the comparison pattern on the image matches the position of the reference pattern on the image. The first specific part is a linear part in the vehicle, and the comparison pattern is calculated by extracting the linear component of the first specific part.

  Here, in the present invention, the comparison pattern calculation unit includes a linear component extraction unit that extracts the linear components of the first specific part and the second specific part, a linear component of the first specific part, It is preferable to use an intersection calculation unit that calculates an intersection with the linear component of the specific part, and use the position of the intersection on the image as the comparison pattern.

  In the present invention, the comparison pattern calculation unit includes a shape component extraction unit that extracts the shape component of the second specific part as a feature point, and uses a positional relationship between the linear component and the feature point as the comparison pattern. It is preferable.

  Further, in the present invention, the camera images the rear side of the host vehicle as the monitoring area, and the first specific part is at least a side pillar or a side door weather strip of the host vehicle located in the imaging range of the captured image. It is preferable.

  According to the present invention, the temporal change in the camera mounting position is detected based on the positional relationship between a plurality of specific parts existing at known positions in the monitoring area. Therefore, as compared with the case where detection is performed with only one point, it is possible to accurately detect a change in the camera mounting position, and it is possible to appropriately correct a positional deviation of an image due to this. In particular, when a linear part is used as the first specific part, it is possible to perform extraction of a linear component on a captured image and calculation of a comparison pattern using the straight part at high speed with a simple calculation.

  Prior to description of a specific system configuration, an image shift correction method according to the present embodiment will be described using the captured images shown in FIGS. 1 to 4 as examples. These captured images are captured by a surveillance camera attached in the vicinity of the door mirror of the vehicle, and based on this, an object (another vehicle, etc.) existing in the surveillance area (the rear side of the host vehicle). Is monitored. In such an in-vehicle camera, if the mounting position (including the mounting angle) deviates from the original mounting position due to vibration during vehicle travel, contact with an object, changes over time, etc., the monitoring area Cannot be captured properly. In order to solve this problem, a certain “reference part” that gives a positional reference in the monitoring area is set, the deviation amount with respect to this “reference part” is successively monitored, and the calibration according to this deviation amount ( Correction) may be performed. In the following, the conditions related to the “reference part” will be examined.

  1A and 1B are diagrams illustrating captured images for explaining the influence of a location, where FIG. 1A illustrates a captured image when the vehicle is traveling, and FIG. 1B illustrates a captured image when the vehicle is stopped. When the camera is installed in the host vehicle, the side surface of the host vehicle is always visible no matter how the location changes, and the side surface exists at a known position in the monitoring area. Therefore, it is preferable to use a reference part that is not easily affected by the place and exists at a known position, specifically, a part of the host vehicle that is displayed in the monitoring area.

  FIG. 2 is a diagram illustrating a captured image for explaining the influence of luminance. As shown in FIG. 5A, the other vehicle on the rear side is reflected in the side window A, and a white line is reflected in the side window B and the curved side body C. Such reflection changes variously according to the environment. Therefore, it is not preferable to use these specific parts A, B, and C that are easily affected by luminance as reference parts. On the other hand, as shown in FIG. 5B, such reflection is not generated in the roof weather strip D and the side door weather strip E, and it can be said that the portion is hardly affected by luminance. Therefore, these specific parts D and E are easy to extract edge components and shape components even if the environment changes, and are suitable as reference parts.

  FIG. 3 is a diagram illustrating a captured image for explaining a setting location of the reference part. As is clear from the above-mentioned examination, when calibration is performed, correction is performed using a rubber part that is not easily affected by luminance and has a clear edge (luminance difference) as a reference part (the rubber part itself). There is almost no reflection of the scenery). As such a rubber part, a side door weather strip can be used as a head, a roof weather strip, a C pillar, or the like can be used. Etc. may be used.

  FIG. 4 is an explanatory diagram of matching mainly using the side door weather strip E. FIG. The side door weatherstrip E that extends straight in the front-rear direction of the side of the vehicle has the highest level of certainty in detecting the linear component and is the least susceptible to luminance, so it has excellent eligibility as a reference part. Yes. In the following, three methods will be exemplified in which the side door weather strip E is mainly used as the reference portion, and another specific portion is combined with the reference portion.

(Method 1) Combination with Roof Weather Strip (Linear Component) As shown in FIG. 6A, the linear component of the side door weather strip E is extracted and the linear component of the roof weather strip D is also extracted. And the pattern which modeled the positional relationship of these linear components is calculated. As this pattern, for example, an intersection point p1 between the straight line L1 including the linear component of the side door weather strip E and the straight line L2 including the linear component of the roof weather strip D is calculated, and the position of the intersection p1 (on the image plane) Coordinate) can be used. Then, the relative shift amount of the comparison pattern calculated on the basis of the actual captured image is calculated based on the reference pattern indicating the initial positional relationship. The shift of the comparison pattern occurs due to a change in the camera mounting position due to vibration during traveling, contact with an object, change with time, and the like. Therefore, if the image correction is performed so that the comparison pattern matches the reference pattern, in other words, the positions of the two match on the image plane, the change in the camera mounting position can be compensated.

(Method 2) Combination with C Pillar (Linear Component) As shown in FIG. 5B, the linear components of the side door weather strip E and C pillar F are extracted. And the pattern which modeled the positional relationship of these linear components is calculated. As this pattern, the intersection point p2 of the straight line L1 including the linear component of the side door weather strip E and the straight line L3 including the linear component of the C pillar F is calculated, and the position (coordinate on the image plane) of the intersection p2 is calculated. Can be used. Then, image correction is performed so that the comparison pattern matches the reference pattern.

(Method 3) Combination with Roof Weather Strip (Shape Component) As shown in FIG. 5C, the linear component of the side door weather strip E is extracted and the shape component of the roof weather strip D is extracted. And the pattern which modeled the positional relationship of these components is calculated. Then, image correction is performed so that the comparison pattern matches the reference pattern.

(First embodiment)
In the present embodiment, a surveillance camera system that includes a correction device that implements the correction methods according to the above-described methods 1 and 2 and is mounted on a vehicle will be described. FIG. 5 is a block configuration diagram of the surveillance camera system 1 according to the present embodiment. The surveillance camera system 1 is based on a surveillance system constituted by a camera 2, an image memory 3, and a surveillance unit 4, and includes a reference pattern storage unit 5, a comparison pattern calculation unit 6, and an image correction unit. 7 is added to the correction system. Hereinafter, although the case where the method 1 is used as an example will be described, the same applies to the case of the method 2.

  The camera 2 is attached near the door mirror of the vehicle. The camera 2 repeatedly captures the situation behind the host vehicle, which is a monitoring area, at predetermined intervals, and sequentially outputs captured images. The captured image output from the camera 2 is sequentially stored in the image memory 3. The monitoring unit 4 sequentially reads the captured image for one frame stored in the image memory 3, and uses a known image recognition technique to detect an object (for example, another vehicle or pedestrian) in the monitoring area from the captured image. Recognize and monitor at predetermined intervals. Then, the monitoring unit 4 prompts the driver for attention and warning as necessary.

  On the other hand, the reference pattern storage unit 5 stores a reference pattern that gives a positional reference when correcting a deviation of a captured image. This reference pattern models an initial positional relationship regarding a plurality of specific parts (positionally different from each other) existing at known positions in the monitoring area. Specifically, when the side door weather strip E and the roof weather strip D are used as the specific part (reference part), as shown in FIG. 4A, the image plane relating to the intersection point p1 of the two straight lines L1, L2. The upper “original” position (ie, the coordinates as the initial value) is used as the reference model. The “original” as used herein means an initial state in which the camera mounting position does not change due to vibration during travel, contact with an object, change with time, and the like.

  The comparison pattern calculation unit 6 sequentially reads the captured image for one frame from the image memory 3, and calculates a comparison pattern based on this for each frame. This comparison pattern models the positional relationship on the captured image plane with respect to the plurality of specific parts described above. Specifically, first, the grayscale conversion unit 6a converts the captured image for one frame into a grayscale image (monochrome image). Next, the binarization unit 6b binarizes the multi-tone luminance values constituting the gray scale image with a predetermined threshold value, and generates a binarized image. Then, the edge enhancement unit 6c performs a process of enhancing the edge of the binarized image generated by the binarization unit 6b as preprocessing for extracting the side door weather strip E. As such edge enhancement processing, for example, a Sobel filter can be used. As is well known, a Sobel filter is a filter that performs averaging with a weight added to the center during smoothing in a method of extracting edges while suppressing noise. On the other hand, the pattern recognition unit 6d performs, as preprocessing for extracting the roof weather strip D, an image area having a change pattern (white → black → white) from the binarized image generated by the binarization unit 6b. Extract. In the image area near the roof weather strip D in FIG. 4A, the roof weather strip D itself is black, the sky projected above it is white, and the side window projected below it is white due to light reflection. (White → black → white).

  The straight line component extracting unit 6e extracts the straight line component of the side door weather strip E based on the edge enhanced image output from the edge enhancing unit 6c, and based on the pattern recognized screen output from the pattern recognizing unit 6d. Then, the linear component of the roof weather strip D is extracted. Then, the intersection calculation unit 6f calculates a straight line L′ 1 including the linear component of the side door weather strip E and a straight line L′ 2 including the linear component of the roof weather strip D as shown in FIG. Is calculated. Then, the intersection calculation unit 6f calculates the coordinates of the intersection p′1 of these straight lines L′ 1 and L′ 2. The coordinates of the intersection point p′1 calculated on the captured image basis are output to the image correction unit 7 as a comparison pattern.

  The image correction unit 7 corrects the deviation of the captured image so that the position of the comparison pattern on the image matches the position of the reference pattern on the image. Specifically, the coordinates of the intersection point p1 as a reference pattern are read from the reference pattern storage unit 5, and are compared with the coordinates of the intersection point p′1 as a comparison pattern. Here, if the intersection point p′1 shown in FIG. 4A coincides with the intersection point p1 shown in FIG. 4, it means that the camera 2 remains in its original mounting position (no correction is required). On the other hand, when the intersection point p′1 is deviated from the intersection point p1, it means that the mounting position of the camera 2 has changed (correction required). In this case, the shift of the captured image is corrected so as to cancel out the vector between the two intersections p1 and p′1. Specific examples of the correction method include a method of mechanically correcting the posture of the camera 2 and a method of performing address conversion of the image memory 3.

  As described above, according to the present embodiment, the temporal change in the camera mounting position is detected based on the positional relationship between a plurality of specific parts existing at known positions in the monitoring area. Therefore, as compared with the case where this is performed with only one point, it is possible to accurately detect the temporal change in the camera mounting position, and it is possible to appropriately correct the image shift caused by this.

  Further, according to the present embodiment, when a linear part is used as a specific part (reference part), it is easy to extract a linear component on a captured image by Hough transformation or the like and to calculate a comparison pattern using the linear part. It can be performed at high speed by calculation. In particular, when imaging and monitoring the rear side of the host vehicle, if a side door weather strip or the like existing on the side of the host vehicle is used as the reference site, the reference site can be accurately extracted without installing a special mark on the vehicle. be able to.

(Second Embodiment)
In the present embodiment, a monitoring system including a correction apparatus that implements the correction method according to the method 3 described above will be described. FIG. 6 is a block configuration diagram of the monitoring system 1 according to the present embodiment. Since there is a difference from the configuration of FIG. 5 in a part of the comparison pattern calculation unit 6, the same blocks are denoted by the same reference numerals and description thereof is omitted here.

  As for the side door weather strip E, the linear component of the side door weather strip E is extracted through the processing by the edge enhancement unit 6c and the linear component extraction unit 6e as in the first embodiment. On the other hand, for the roof weather strip D, the feature points are calculated by the shape component extraction unit 6g by extracting and grouping the shape components. Then, the positional relationship between the linear component of the side door weather strip E and the feature point of the roof weather strip D is output as a comparison pattern.

  The image correction unit 7 corrects the deviation of the captured image based on the comparison pattern and the reference pattern read from the reference pattern storage unit 5. The reference pattern in the present embodiment is a model of the original position of the linear component of the side door weather strip E and the original position of the feature point of the roof weather strip D.

  According to the present embodiment, the same effect as that of the first embodiment described above can be obtained by the combined use of extraction of linear components and extraction of shape components regarding a plurality of reference parts.

  In each of the above-described embodiments, the monitoring camera system that monitors the rear side of the host vehicle has been described as an example. However, the present invention is not limited to this, and other various monitoring camera systems (for in-vehicle use) Whether or not).

The figure which shows the picked-up image for explaining the influence by a place The figure which shows the captured image for demonstrating the influence by a brightness | luminance The figure which shows the captured image for demonstrating the setting place of a reference | standard site | part Illustration of matching mainly on side door weatherstrip The block block diagram of the surveillance camera system which concerns on 1st Embodiment Block diagram of a surveillance camera system according to the second embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Surveillance camera system 2 Camera 3 Image memory 4 Monitoring part 5 Reference pattern memory | storage part 6 Comparison pattern calculation part 6a Gray scale conversion part 6b Binarization part 6c Edge emphasis part 6d Pattern recognition part 6e Linear component extraction part 6f Intersection calculation part 6g Shape component extraction unit 7 Image correction unit

Claims (4)

In a correction device for a surveillance camera mounted on a vehicle,
A camera that outputs a captured image by imaging a monitoring area;
In order to provide a positional reference for correcting the deviation of the captured image, the first specific part existing at a known position in the monitoring area is different from the first specific part in the monitoring area. A reference pattern storage unit that stores a reference pattern that models an initial positional relationship with the second specific part existing at a known position;
On the captured image output from the camera, the first specific part and the second specific part are extracted, and the positional relationship between the extracted first specific part and the second specific part A comparison pattern calculation unit that calculates a comparison pattern that models
An image correction unit that corrects a deviation of the captured image so that a position on the image of the comparison pattern matches a position on the image of the reference pattern;
The monitoring camera correction apparatus according to claim 1, wherein the first specific part is a linear part in the vehicle, and the comparison pattern is calculated by extracting a linear component of the first specific part. The comparison pattern calculation unit
A linear component extraction unit that extracts linear components of each of the first specific part and the second specific part;
An intersection calculation unit for calculating an intersection of the linear component of the first specific part and the linear component of the second specific part;
The monitoring camera correction apparatus according to claim 1, wherein a position of the intersection point on the image is used as the comparison pattern. The comparison pattern calculation unit includes a shape component extraction unit that extracts a shape component of the second specific part as a feature point;
The monitoring camera correction apparatus according to claim 1, wherein a positional relationship between the linear component and the feature point is used as the comparison pattern. The camera images the rear side of the host vehicle as the monitoring area,
4. The surveillance camera according to claim 1, wherein the first specific part is at least a side pillar or a side door weather strip of the host vehicle located in an imaging range of the captured image. 5. Correction device.

Images