JP2005333442A - Periphery viewing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the picture quality of a part large in magnification factor at the time of transforming coordinates into plane view coordinates by geometric correction. <P>SOLUTION: In a display device 15 for displaying an image by a half of resolution concerned with the image pickup of a camera 11, coordinate transformation for a part on which an image is transformed by a prescribed magnification factor or less by coordinate transformation is performed by referring to either one of an odd field and an even field, and coordinate transformation for a part on which an image is transformed by a magnification factor more than the prescribed magnification factor by coordinate transformation is performed by referring to both of odd and even fields. Consequently images of clear picture quality can be displayed on the display device independently of the magnification factor. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a peripheral visual recognition apparatus that captures an image of a periphery of a vehicle with a predetermined imaging unit and displays the image on a display unit in the vehicle, and a technology related thereto.

  In the field of automobiles, for example, in order to make it easier for the driver to check the back of the vehicle, the back is imaged with a back camera installed at the rear of the vehicle and displayed on a display device such as a liquid crystal panel installed in the vehicle. Things have been done.

  As an image displayed on the display device in this way, an original image of a normal back camera can be used. FIG. 4 shows an original image captured by the back camera. In this original image, as shown in FIG. 5, since the installation position of the camera 1 is set with a certain height H from the ground, the original image entering the visual field range 3 of the camera 1 is imaged from a perspective looking down obliquely. It has been made.

  However, the original image of the viewpoint looking down at an angle shown in FIG. 4 is not necessarily easy to grasp the surrounding image sensuously because the installation position of the camera 1 is generally set differently from the viewing height of the viewer. Often not a thing.

  Therefore, by using a viewpoint conversion technique such as applying geometric correction to the image, the image of the viewpoint looking down obliquely is converted into the image of the plane view coordinates, so that a sense of distance can be obtained as shown in FIG. It is possible to make it easy to understand.

  4 and 6, reference numeral 5 denotes a vehicle bumper, and reference numeral 7 in FIG. 6 denotes a dark display portion that is displayed corresponding to a pixel portion that was not in the original image by performing coordinate conversion. ing.

  However, in the image after viewpoint conversion by geometric correction as shown in FIG. 6, the portion indicated by A2 in the vicinity of the vehicle is reduced (pixel thinning) corresponding to the portion A1 of the original image. However, the portion indicated by B2 far from the vehicle is enlarged (pixel interpolation) corresponding to the portion B1 of the original video.

  Here, FIG. 7 is a principle diagram in the case where the image enlargement ratio is small and almost no pixel interpolation is required, and FIG. 8 is a principle diagram in the case where the image interpolation ratio is large and pixel interpolation is performed. 7 and 8, y0 to y3 represent pixels in the original image, and Y0 to Y3 represent pixels in the image after viewpoint conversion by geometric correction.

  In the case where the enlarged portion B2 exists in the image after the viewpoint conversion by the geometric correction, when the enlargement ratio is small as shown in FIG. 7, for example, the pixel Y0 is generated from the pixels y0 and y1 in the original video, Since the resolution of the original video is substantially maintained such that the pixel Y1 is generated from the pixels y1 and y2 in the original video, the image quality is not deteriorated.

  On the other hand, when the enlargement ratio is large as shown in FIG. 8, for example, the pixel Y0 is generated from the pixels y0 and y1 in the original video, whereas the pixel Y1 is also the pixels y0 and y1 in the original video, for example. Since the output pixels Y0 to Y3 are continuously interpolated with reference to the same two pixels y0 to y3 in the original video, the video is blurred in the vertical direction. .

  Since the enlargement ratio of the image increases as the distance from the vehicle increases, the image quality in the image portion far away from the vehicle (for example, B2) in the image after viewpoint conversion by geometric correction as shown in FIG. Deterioration is likely to occur.

  Therefore, an object of the present invention is to provide a peripheral visual recognition device that can improve the visibility of a portion where the enlargement ratio is high when performing coordinate conversion to a planar coordinate by geometric correction.

  In order to solve the above-mentioned problem, the invention described in claim 1 is a peripheral visual device that captures a peripheral image by a predetermined imaging unit and displays the peripheral image on a predetermined display unit. The image pickup means for outputting a video divided into an odd field consisting of only an even field and an even field consisting only of even lines, a display means for displaying the video at half the resolution related to the image pickup of the image pickup means, and the original video Image processing means for performing a coordinate transformation and outputting a video to the display means, the image processing means for the odd field and the even number for a portion where the video is transformed at a predetermined magnification or less by the coordinate transformation. While the coordinate transformation is performed with reference to any one of the fields, the portion where the video is transformed by the coordinate transformation exceeding the predetermined enlargement ratio is And performs the coordinate transformation with reference to both the odd field and the even field.

  A second aspect of the present invention is the peripheral visual recognition apparatus according to the first aspect, wherein the predetermined enlargement ratio is 1.0.

  A third aspect of the present invention is the peripheral visual recognition apparatus according to the first or second aspect, wherein the image processing means converts the video at a magnification of 2.0 times by the coordinate conversion. The coordinate conversion is performed by alternately arranging the odd lines of the odd field and the even lines of the even field.

  Invention of Claim 4 is the periphery visual recognition apparatus in any one of Claim 1 to 3, Comprising: The said image processing means converts with the expansion ratio higher than at least 1.0 by the said coordinate conversion. A part of the video is upsampled to double the resolution, and pixel interpolation is performed using a predetermined sampling function.

  The peripheral visual recognition device according to the first aspect of the present invention converts the video at a predetermined magnification or less by coordinate conversion when the display device displays the video at half the resolution related to the imaging of the imaging unit. For the portion, coordinate conversion is performed with reference to one of the odd field and the even field, and on the other hand, both the odd field and the even field are converted for the portion in which the video is converted beyond the predetermined enlargement ratio by the coordinate conversion. Since the coordinate conversion is performed with reference, a clear image can be displayed on the display device regardless of the magnification. Therefore, it is possible to display a video with good visibility.

  In this case, as in claim 2, if the coordinate conversion is performed with reference to both the odd field and the even field when the enlargement ratio exceeds 1.0 times, the enlargement ratio is at least twice or less in all the portions. Thus, it is possible to obtain almost the same image quality as that of only one of the odd field and the even field.

  According to the third aspect of the present invention, the peripheral visual recognition device according to the third aspect of the present invention is such that coordinate conversion is performed by alternately arranging odd-numbered lines and even-numbered lines in a portion where an image is converted at a magnification of 2.0 times by coordinate conversion. The image quality can be easily clarified simply by performing a simple process.

  According to the fourth aspect of the present invention, since the peripheral vision device of the present invention up-samples a part of an image to double the resolution and performs pixel interpolation using a predetermined sampling function, the resolution is reduced when performing pixel interpolation. It is possible to prevent image quality degradation.

  Generally, a camera as an imaging unit that captures an image outputs (image transmission) a signal related to the captured image to an image processing ECU (image processing unit) as an NTSC signal. In this image transmission in the NTSC system, for example, out of a total of 480 lines in the vertical direction, an odd field composed of 240 odd lines and an even field composed of 240 even lines are alternately divided. An image is being transmitted.

  By the way, in order for a passenger such as a driver (user) in a car to check the safety of the surroundings, a video around the car is captured by a camera and displayed on a liquid crystal display (LCD) in the vehicle interior. Since the liquid crystal display device needs to be arranged in a limited space in the vehicle interior, it is difficult to install a liquid crystal display device having a large screen, and 240 lines are suitable for the number of video lines displayed on the liquid crystal display device. Yes.

  Therefore, conventionally, only one of the odd field composed of 240 odd lines and the even field composed of 240 even lines is displayed, for example, only the odd field. In other words, even fields are captured by the camera and transmitted to the image processing ECU as NTSC signals, but are not discarded by the image processing ECU and displayed on the liquid crystal display device.

  However, when the image enlargement ratio is large as shown in FIG. 8, if the pixel interpolation is performed on the image of only the odd field, the image becomes blurred in the vertical direction, and particularly the geometrical shape as shown in FIG. In the video after the viewpoint conversion by the academic correction, there is a problem that image quality deterioration is likely to occur in a video part (for example, B2) far from the vehicle.

  Therefore, in the peripheral visual recognition apparatus according to the embodiment of the present invention, the resolution is increased by using both the odd field and the even field for an image of a part where the image quality deterioration is likely to occur at a far distance from the vehicle. Then, processing for reducing image quality degradation is performed.

  FIG. 1 is a block diagram showing a peripheral visual recognition apparatus according to an embodiment of the present invention.

  This peripheral visual recognition device is an element as shown in FIG. 4 from the viewpoint of looking down obliquely as shown in FIG. 5 with respect to the periphery of the automobile so that an occupant such as a driver (user) in the automobile can check the surrounding safety. An image is picked up, a viewpoint conversion is performed on the original image, and the viewpoint is converted into a planar coordinate by geometric correction.

  Specifically, as shown in FIG. 1, this peripheral visual recognition device is received by an image processing ECU (image processing means) 13 through, for example, an in-vehicle LAN 12 for a camera 11 that captures the periphery of a vehicle, and this image. The image is processed by the processing ECU 13 and displayed on the display device (display) 15.

  Here, the camera 11 is a general camera provided with a predetermined image pickup device such as a CCD, for example, and picks up an original image from a viewpoint looking down obliquely as shown in FIG. The original image captured by the camera 11 has 480 lines of pixels in the vertical direction. Of these, the odd field is composed of 240 odd lines and the even lines are 240 lines. Divided into even fields and alternately transmitted to the image processing ECU 13 through the in-vehicle LAN 12.

  As shown in FIG. 1, the image processing ECU 13 includes a communication I / F unit 21 that is a communication interface for performing communication via the in-vehicle LAN 12, a monitor I / F unit 23 connected to a display device (display) 15, and image processing. The image processing ASIC 25 for managing the image, a conversion table memory 27 for storing a conversion table for performing coordinate conversion in the image processing ASIC 25, and a frame memory 29 such as an SDRAM for temporarily storing video during the image processing by the image processing ASIC 25. And a CPU 31 for controlling the image processing ASIC 25 and a rewritable nonvolatile memory 33 for providing various parameters and the like in the image processing in the image processing ASIC 25.

  Here, the frame memory 29 stores both odd and even fields at least for each frame of the original video. Preferably, the frame memory 29 can store data having a larger capacity than the data per frame of the original video.

  The image processing ASIC 25 has a first function for performing viewpoint conversion on the original image and performing viewpoint conversion to the plane view coordinates by geometric correction, and it is necessary to enlarge the original image in the vertical direction during the viewpoint conversion. A second function that refers only to pixels in odd-numbered fields in the frame memory 29 for a non-existing portion (that is, a portion having an enlargement ratio of 1.0 times or less), and the original image needs to be enlarged in the vertical direction at the time of viewpoint conversion There is a third function of performing pixel interpolation with reference to both odd-numbered field and even-numbered field pixels in the frame memory 29 for a certain part (that is, a part having an enlargement ratio larger than 1.0 times).

  Here, FIG. 2 is a principle diagram in the case where pixel interpolation is performed for a portion where the enlargement ratio is 2.0 times, and the symbols yo0 to yo3 in FIG. 2 are pixels arranged in the vertical direction in the odd field, the symbol ye0. ... Ye2 represents pixels arranged in the vertical direction in the even field, and symbols Y0 to Y6 represent pixels arranged in the vertical direction in the video after pixel interpolation (see FIG. 8).

  For example, in the portion where the enlargement ratio is 1.0, 240 lines are set in the vertical direction as the conforming resolution of the display device 15, so that each pixel yo0 to yo3 in the odd field of the original video is sufficiently clear. Can be obtained.

  On the other hand, for the portion where the enlargement ratio is 2.0 times, it is necessary to interpolate one pixel at a time between the pixels yo0 to yo3 adjacent in the vertical direction in the odd field. Therefore, as shown in FIG. 2, the image processing ASIC 25 interpolates the pixels ye0 to ye2 in the even field, which have been conventionally discarded, between the pixels yo0 to yo3 adjacent in the vertical direction in the odd field as they are. An image as clear as the portion where the enlargement ratio is 1.0 times can be obtained.

  In FIG. 2, the pixels yo0 to yo3 in the odd field are directly adopted as Y0, Y2, Y4, and Y6 of the display image, and the pixels ye0 to ye2 in the even field are directly used as the pixels Y1, Y3, and Y3 of the display image. An example adopted as Y5 is shown.

  If the enlargement ratio is larger than 1.0 and smaller than 2.0, the image processing ASIC 25 uses the pixels Y0 to Y6 of the image after interpolation shown in FIG. The image is reduced by performing a thinning process accompanied with the.

  Further, when the enlargement ratio is larger than 2.0 times, the image processing ASIC 25 uses the pixels Y0 to Y6 of the image after interpolation shown in FIG. 2, for example, a predetermined pixel interpolation processing as shown in FIG. To enlarge the image.

  In FIG. 3, the pixels yo0 to yo3 in the odd field are directly adopted as the pixels Y0, Y2, Y4, and Y6 of the display image. For example, the pixel Y3 of the display image is subjected to pixel interpolation with reference to one pixel ye1 in the even field and four pixels yo0 to yo3 in the odd field. As a pixel interpolation method applied at this time, a general method of weighting and adding peripheral pixels is used. For example, an existing pixel interpolation method such as a bilinear method or a bicubic method is adopted.

  In this way, an image with an enlargement ratio of 2.0 times or less can be obtained with at least the same resolution as the original image.

  Note that in normal pixel interpolation, image quality degradation is not noticeable until the enlargement ratio is about 1.5 times. In this embodiment, the image is as clear as the original image up to an enlargement ratio of 2.0. Therefore, the image quality degradation is not noticeable in the portion where the pixel interpolation with the enlargement ratio of 3.0 times (= 2.0 times × 1.5 times) or less is performed, and the enlargement ratio is relatively far away. Deterioration of the image quality is felt only in a portion exceeding 3.0 times.

  Therefore, when performing coordinate conversion to planar coordinates by geometric correction, the enlargement ratio can be increased only by changing the image processing inside the image processing ECU 13 without changing the conventional camera 11 or display device 15 at all. The visibility of the part which becomes can be improved easily.

  In the above-described embodiment, in the original image having 240 lines in the vertical direction, coordinate conversion is performed without changing the number of samples, and each pixel yo0 to the odd field is applied to a portion having a low resolution far from the vehicle at the time of the coordinate conversion. Interpolate the pixels ye0 to ye2 in the even field with respect to yo3 to obtain a clear image with a magnification of 2.0 times in the vertical direction, and perform pixel interpolation and thinning processing using this image to perform coordinate conversion However, the method is not limited to the above method as long as each pixel ye0 to ye2 in the even field, which has been conventionally discarded, is used in addition to the pixels yo0 to yo3 in the odd field.

  For example, at the time of coordinate conversion of an original image having 240 lines in the vertical direction, a resolution of 2 is obtained by up-sampling a portion with a low resolution far away from the vehicle (for example, a portion with a vertical enlargement ratio of 3.0 times or more). It is also possible to increase the resolution twice and perform interpolation by a sampling (sink) function to increase the resolution and reduce image quality deterioration. In the upsampling in this case, if each of the pixels ye0 to ye2 in the even field is used, a video portion with a clear image quality can be displayed on the display device 15 even in a portion with a high enlargement ratio as in the above embodiment. . In this case as well, it is needless to say that the sharpness of the overall image quality can be improved by performing pixel interpolation using not only the pixels yo0 to yo3 in the odd field but also the pixels ye0 to ye2 in the even field. .

  In the above embodiment, the example in which the camera 11 is installed in the automobile and the periphery of the vehicle is visually recognized by the display device 15 in the vehicle interior has been described. However, for example, the camera 11 is installed on the wall of a parking lot or the like. You may apply to what visually recognizes the periphery of the camera installation position of a parking lot with this display apparatus 15.

It is a block diagram which shows the periphery visual recognition apparatus which concerns on one Embodiment of this invention. It is a figure which shows the principle which performs the pixel interpolation whose expansion ratio is 2.0 time in the periphery visual recognition apparatus which concerns on one Embodiment of this invention. It is a figure which shows the principle which performs the pixel interpolation in the part in which the expansion ratio exceeds 2.0 time in the periphery visual recognition apparatus which concerns on one Embodiment of this invention. It is a figure which shows the original image imaged with the camera. It is a figure which shows the example of the installation position of the camera in a vehicle. It is a figure which shows the example of an image | video which performed coordinate transformation to planar view coordinate by geometric correction in the conventional periphery visual recognition apparatus. It is a principle figure in case the expansion rate of a part of image | video is small. It is a principle figure in case the expansion rate of a part of image | video is large.

Explanation of symbols

11 Camera 12 In-vehicle LAN
13 Image processing ECU
15 Display device

Claims (4)

A peripheral visual recognition device that captures a peripheral image by a predetermined imaging unit and displays the peripheral image on a predetermined display unit,
The imaging means for capturing an original image and outputting the image divided into an odd field composed of only odd lines and an even field composed of only even lines;
Display means for displaying video at half the resolution associated with imaging by the imaging means;
Image processing means for performing coordinate transformation on the original video and outputting the video to the display means,
The image processing means performs the coordinate conversion with reference to one of the odd field and the even field for a portion where a video is converted at a predetermined magnification or less by the coordinate conversion, while the coordinate The peripheral visual recognition apparatus, wherein the coordinate conversion is performed with reference to both the odd field and the even field for a portion where the video is converted exceeding the predetermined enlargement ratio by the conversion. The peripheral visual recognition device according to claim 1,
A peripheral visual recognition device in which the predetermined enlargement ratio is 1.0. It is a periphery visual recognition device according to claim 1 or 2,
The image processing means alternately arranges the odd lines of the odd field and the even lines of the even field for the portion in which the image is converted at a magnification of 2.0 times by the coordinate conversion. Periphery visual recognition device characterized by performing conversion. The peripheral visual recognition device according to any one of claims 1 to 3,
The image processing means up-samples a part of an image converted at an enlargement ratio higher than 1.0 by the coordinate conversion, doubles the resolution, and performs pixel interpolation by a predetermined sampling function. Periphery visual recognition device characterized.