JP4706896B2 - Wide-angle image correction method and vehicle periphery monitoring system - Google Patents

Description

  The present invention relates to a method for correcting a wide-angle image for displaying a wide-angle image having a wide viewing angle in the horizontal direction in which the outside of the vehicle is imaged, and a vehicle periphery monitoring system using the image. .

  In recent years, in conjunction with the spread of navigation systems, the number of vehicles equipped with a monitor system that displays a situation around the vehicle on a display device in the passenger compartment is increasing. Examples of the situation around the vehicle include the position of another vehicle, the situation of an obstacle, and road markings such as a center line and a stop line. These surrounding conditions are displayed on a display device such as a back monitor when the vehicle is backed or a blind spot monitor under a front bumper or a corner. Such a monitor system is intended to reduce the burden on the driver, and in view of the large amount of information that the driver needs to acquire during driving, it is excellent in visibility and driving at a glance without a sense of incongruity. It is desirable to display an image from which a person can obtain necessary information. Of course, it is desirable that the field of view of the image is wide. In general, a human still body field of view is about 200 degrees with one eye gazing at one point. Of these, it is said that it can be confirmed up to about 70 degrees including colors such as red, blue and yellow. Furthermore, the moving object visual field when the object is viewed while moving is narrowed according to the operation speed, and is lowered to about 100 degrees, which is half of the static object visual field, at about 40 km / h. In order to compensate for this, the driver usually tries to keep a wide viewing angle without gazing at a single point. However, the monitoring system as described above preferably complements such human physiological limits and efforts. It is hoped that

  However, in general, the viewing angle of an image taken by a camera or the like is as narrow as about 50 to 65 degrees, and a sufficient viewing angle cannot be obtained simply by installing the camera in a vehicle. Various methods have been proposed to compensate for this. For example, as shown in FIGS. 7 to 8, there is a method in which a place that can be a blind spot from the driver's seat is photographed with priority and this is displayed in a separate display frame of the display device. However, when divided and displayed in this way, the driver cannot grasp the environment sufficiently without being aware of the mutual relation positions of the respective captured images, and there is a problem that it is difficult to obtain information on the surroundings of the vehicle at a glance.

  On the other hand, Patent Document 1 proposes a technique for displaying an image overlooking a wide range including the host vehicle. This is composed of one or a plurality of cameras, means for converting an image input from the camera into other coordinates by perspective transformation, and combining the converted image into a single image in relation to the image of the own vehicle. Means and a display (display device) for displaying this image to an occupant (driver). The perspective transformation implemented in this document refers to road surface (plane) coordinates from the camera screen image, where the vehicle center is the origin, the left and right sides of the vehicle traveling direction are the X axis, and the vehicle traveling direction is the Y axis. To convert. Then, the synthesized image shows the own vehicle shown in the illustration, the other vehicle, the road sign, the object, etc., which are perspective-transformed, shown in plane coordinates, and this is displayed on the display device.

  Further, Patent Document 2 proposes a method of displaying a wide-angle image viewed from the viewpoint of the driver and emphasizing a particularly required portion without performing perspective conversion to plane coordinates. According to this, the emphasis region and the non-enhancement region are defined in the wide-angle image, and the image in the emphasis region is displayed by being emphasized by processing such as change of brightness, mask processing, or enlargement. It makes it easy to obtain the information that is particularly needed from the information.

Japanese Patent Laid-Open No. 3-99952 (page 2, FIGS. 1 to 3) Japanese Patent Laid-Open No. 2002-230698 (pages 3 to 5, FIGS. 3 and 4)

  Perspective transformation is also called viewpoint transformation, but taking the technique of Patent Document 1 as an example, screen coordinates viewed from a direction parallel to the XY plane composed of the X axis and the Y axis are expressed with respect to the XY plane. This is converted into planar coordinates viewed from the orthogonal direction. Therefore, an object standing perpendicular to the XY plane, such as a pole or a wall, is clearly visible in the screen coordinates, but in the plane coordinates, it is aggregated into points, lines, etc. It becomes impossible to display the image or it is displayed as an extremely uncomfortable image. Even if it is not as extreme as this example, if an image taken from a certain viewpoint in a three-dimensional space is converted into a planar image, images that are not on the plane to be converted will not be converted correctly, resulting in an uncomfortable image. .

  Further, in the technique described in Patent Document 2, although the highlighted portion is conspicuous, the displayed image is different from an image that looks natural to humans and is not suitable for grasping the direction and distance of the object at first glance. .

  SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an image correction method for displaying a wide-angle image having a wide viewing angle in the horizontal direction in which the outside of the vehicle is photographed in the passenger compartment without any sense of incongruity, and the correction. The object is to provide a vehicle periphery monitoring system using images.

Feature of the correction method of the wide-angle image for displaying the wide-angle image having a horizontally wide viewing angle were taken outside the vehicle according to the present invention for achieving the above object on the display device of the vehicle cabin, the vehicle The wide-angle image to be displayed on the indoor display device is an image in which the traveling direction of the vehicle is located in the horizontal central portion, and is equidistant from the center point set in the horizontal central portion at the lower end of the wide-angle image. Is set on the wide-angle image, and after setting the equidistant line, the wide-angle image is arranged so that the equidistant lines are evenly spaced on the wide-angle image. The point is to correct by enlarging or reducing .

At the top of the vertical direction of the wide-angle image that has a wide viewing angle in the horizontal direction obtained by photographing the outside of the vehicle, for example, a road in the traveling direction and a distant building such as a building ahead of it are reflected, and the vertical direction Often the road surface is visible at the bottom. In many cases, the corners of the vehicle, the scenery of the side, etc. are shown in the horizontal direction. Here, as described above, if the wide-angle image is reduced toward the upper part in the vertical direction and the wide-angle image is enlarged toward the lower part, the upper part in the vertical direction is relatively small. Therefore, a visual effect that is far away can be obtained, and on the contrary, the lower vertical portion is relatively large, so that a visual effect that is close can be obtained. Therefore, it is possible to obtain an appropriate perspective, which is preferable.
In addition, the human still body field of view has about 180 to 200 degrees, but the range that can actually be confirmed including the color is about 70 degrees. Therefore, the side near the viewpoint, for example, the side of the vehicle, tends to be a blind spot. However, according to the feature of the present invention, since the side image is enlarged, the side view that is difficult to be recognized due to the problem of the physiological limit of human vision is emphasized and shown effectively. Can be perceived.

Also, the closer to the viewpoint, the larger the image area is taken, so the distance between the equidistant lines near the center point that is the viewpoint is wider, and the distance between the equidistant lines is smaller as the distance is further away. Therefore, for example, if the wide-angle image is enlarged or reduced so that the distance between the equidistant lines is substantially equal, the enlargement ratio is relatively increased as the equidistant line is moved away from the center point. Then, the adjacent horizontal line below the wide-angle image becomes an arc-shaped continuous curve centered on the upper side in the vertical direction and having a larger curvature than the far horizontal line, and goes upward as it goes to the side (left and right direction). As it goes, the images on the left and right ends are enlarged. The center part represents the distance in the front, and the side part represents the vicinity in the left-right direction such as the corner part of the bumper, for example, so that an image that gives a sense of perspective that matches the sense of distance can be obtained. .
Therefore, by performing correction using this equidistant line, it is possible to obtain a wide-angle image with little sense of discomfort to the human eye such as perspective. In addition, since the side image of the neighborhood that is particularly prone to blind spots is enlarged from the lower corner of the wide-angle image to the upper side, if any event occurs at this blind spot, it compensates for human physiological limitations. It can be suitably sensed.

The wide-angle image correction method according to the present invention is applied to a vehicle periphery monitoring system that displays a wide-angle image having a wide viewing angle in the horizontal direction taken from the outside of a vehicle on a display device in a vehicle interior, thereby obtaining the wide-angle image. It is preferable to include an image input means for image processing, an image processing means for correcting the wide-angle image by the wide-angle image correction method according to the present invention, and a display means for displaying the corrected wide-angle image. .
By applying the present invention to a vehicle periphery monitoring system, it is possible to display an image that does not feel uncomfortable for the human eye in the passenger compartment, and to obtain a system that suitably complements human physiological limits and efforts. .

  Hereinafter, an embodiment in the case of applying a wide-angle image correction method according to the present invention to a vehicle periphery monitoring system will be described with reference to the drawings.

  As shown in FIG. 1, the vehicle periphery monitoring system according to the present embodiment includes a camera 1 at a front portion of a vehicle 10 and displays a captured image on a monitor (display device) 3 in a vehicle interior. . The camera 1 is arranged with a depression angle of about 15 to 30 degrees from a horizontal direction to a slightly lower side. The monitor in the vehicle compartment may also be used as a navigation system or the like. Here, the captured image is a wide-angle image W having a wide horizontal viewing angle of about 180 to 200 degrees. The image processing unit 20 in which software and hardware of an ECU (Electronic Control Unit) 2 composed of a microcomputer or the like cooperates performs image processing so that the image can be seen naturally without any sense of discomfort to the human eye, and the display image M And displayed on the monitor 3. Note that the captured image as the wide-angle image W may be a composite of images captured by the plurality of cameras 1, or may be an image captured using a wide-angle lens such as a fisheye lens.

  As shown in FIG. 2, the image processing unit 20 performs perspective correction that performs relative enlargement / reduction so that the wide-angle image W is reduced toward the upper part in the vertical direction and the wide-angle image W is enlarged toward the lower part. The perspective correction means 21 and the lateral distance correction means 22 that performs the lateral distance correction that enlarges the wide-angle image W from the vertical reference line passing through the horizontal center of the wide-angle image W toward both ends in the horizontal direction. And have. Each of these means performs respective correction processes in cooperation with the hardware and software of the ECU 2.

In general, there is a sense of perspective that human beings see large things close and things small in the distance. Perspective correction is a process for stimulating this sensation so that the wide-angle image W can be seen by human eyes without a sense of incongruity. In addition, the side is usually a part that is not particularly conscious even if it is in the field of view. That is, since the human eye moves its viewpoint (line of sight) to a conscious place, the side is usually not conscious even if it is in the field of view. However, when an event of interest occurs in the field of view, it is handled by moving the viewpoint in the direction in which the event occurred to confirm it. However, it is not easy to change the viewpoint based on the consciousness like a human being in the shooting by the camera 1 fixedly provided on the vehicle. Therefore, it is important to represent the image so that the events on the side can be transmitted without shortage, particularly on the side of the wide-angle image W. And it is necessary to show that it is a lateral distance with an appropriate sense of distance without destroying the balance of the entire wide-angle image W, that is, without giving a sense of incongruity to human eyes. The process of correcting the wide-angle image W so as to obtain such an image is lateral distance sense correction.
Hereinafter, a specific method for performing the perspective correction and the lateral distance correction will be described.

[First embodiment]
In the present embodiment, a method of performing perspective correction and lateral distance correction by projecting the wide-angle image W onto a trapezoid will be described. FIG. 3 is an explanatory diagram showing an example of perspective correction according to the present invention. As shown in FIG. 3, the wide-angle image W is projected onto a trapezoid having a short side above the vertical direction of the wide-angle image W that is the original image before correction and a long side below, ie, a trapezoid. Perspective correction is performed by performing deformation. In this way, the upper image of the wide-angle image W is relatively small and the lower image is large.

  The wide-angle image W obtained by photographing the outside of the vehicle has an upper part in the vertical direction, for example, a distant scene such as a road in the traveling direction or a building ahead, and a road surface in the lower part in the vertical direction. The scene near the bumper, such as the ground. Also, in the horizontal direction, the corners of the bumper of the vehicle, the side view, etc. are shown. Here, as described above, when the wide-angle image W is projected onto a trapezoid having a short side at the top and a long side at the bottom, the wide-angle image W is reduced toward the upper part in the vertical direction, and as it goes toward the lower part. The relative enlargement / reduction is performed so that the wide-angle image W is enlarged. As a result, the vertical upper part is relatively small with respect to the lower part, so that a visual effect is obtained at a distance, and conversely the vertical lower part is relatively large with respect to the upper part so that it is close. Visual effects can be obtained. Therefore, an appropriate perspective can be obtained.

  FIG. 4 is an explanatory diagram showing an example of the lateral distance correction according to the present invention. As shown in FIG. 4, the wide-angle image W or the perspective correction is made into a trapezoid having a short side on the vertical reference line V passing through the central portion in the horizontal direction of the wide-angle image W and having long sides on both ends in the horizontal direction. Lateral distance sense correction is performed by projecting a later image, that is, by performing a trapezoidal deformation. In this way, the image at the center of the wide-angle image W is relatively small and the images at both sides are large. The central portion is the center of the vehicle in the traveling direction, and is a correction that is preferable from the viewpoint of perspective because it is a distant image.

  Further, as already described, the human still body field of view is about 200 degrees, and the range that can be actually confirmed including the color is about 70 degrees. Accordingly, a side near the viewpoint, for example, a corner portion of a vehicle bumper is likely to be a blind spot. However, according to the present embodiment, since the side image is enlarged, the images on both sides are enlarged, so that the side view that is difficult to recognize because it is near the limit of the visual field is effectively perceived by humans. Can be made. That is, the side view that is difficult to be recognized due to the problem of the physiological limit of human vision is emphasized and shown, so that humans can effectively perceive. FIG. 5 is an example of an image after performing the perspective correction and the lateral distance correction by the trapezoidal deformation.

  Such trapezoidal deformation can be realized by linear calculation, and does not impose a large calculation load on a device that performs image processing such as a microcomputer constituting the ECU 2. Therefore, it does not require an expensive microcomputer with excellent calculation capability, and does not require large power consumption due to high-speed calculation. In the block configuration shown in FIG. 2, the perspective correction means 21 and the lateral distance correction means 22 are shown to be processed separately, but of course the present invention is not limited to this. Absent. For example, a result of performing both corrections as shown in FIG. 5 may be prepared in advance as a coordinate table, and both corrections may be performed collectively based on this table. Although a storage medium such as a flash memory is required as a storage device for storing the coordinate table, conversion time and calculation load can be reduced. The coordinate table prepared in advance may be obtained by calculation when the camera is attached to the vehicle or during maintenance such as inspection.

[Second Embodiment]
A second embodiment of the present invention will be described based on FIG. 6 which is an explanatory diagram showing another example of perspective correction and lateral distance correction according to the present invention. As shown in FIG. 6A, annual ring-shaped equidistant lines R that are on the vertical reference line V and that are equidistant from the center point O that is the viewpoint of the wide-angle image W are shown on the wide-angle image W. Set above. Since the closer to the viewpoint, the larger the image area is taken, the distance between the equidistant lines R in the vicinity of the center point O is wider, and the distance between the equidistant lines R is narrower as the distance is further away. Here, as shown in FIG. 6B, the wide-angle image W is enlarged / reduced so that the intervals between the equidistant lines R are substantially equal. When such enlargement / reduction is performed, the distant horizontal line (distant horizontal line H1) above the wide-angle image W becomes a single continuous curve in the shape of an arc centered on the lower side in the vertical direction. The more you go (left and right), the lower you go. A neighboring horizontal line (neighboring horizontal line H2) below the wide-angle image W is a continuous curve in an arc shape with the upper side in the vertical direction being the center side and the far horizontal line H1 having a large curvature, and to the side (left and right direction). As it goes, it goes upwards, and the images at the left and right ends are enlarged.

  By performing the correction using the equidistant line R, it is possible to obtain a wide-angle image W with less sense of incongruity to human eyes such as perspective. In addition, since a side image in the vicinity that is likely to become a blind spot is enlarged above the wide-angle image W, it can be suitably detected when any event occurs in this blind spot.

  In the example of the equidistant line R shown in FIG. 6, the equidistant line R having the same width in both the vertical direction and the horizontal direction is used for easy explanation. However, it is not necessary to express the side (horizontal direction) and the front (vertical direction) with the same width. For example, the width of the equidistant line R that is more magnified to the side that is likely to become a blind spot, and the width of the equidistant line R that is about the current level is maintained for the front that is a normal human viewpoint. For example, it is more preferable that the width of the equidistant line R is different between the horizontal direction and the vertical direction of the enlarged image.

Further, such correction using equidistant lines may be considered to be executed simultaneously by the perspective correction means 21 and the lateral distance correction means 22 in the image processing unit 20 shown in FIG. That is, in the first embodiment, a coordinate table based on equidistant lines may be stored in place of the trapezoidal deformation coordinate table prepared in advance. Of course, both the coordinate table based on trapezoidal deformation and the coordinate table based on equidistant lines are stored, and either coordinate table is selected according to the driving situation (high speed driving, low speed driving, parking, etc.) Then, correction may be performed.

  As described above, according to the present invention, a wide-angle image having a wide viewing angle in the horizontal direction obtained by photographing the outside of the vehicle can be displayed in the vehicle interior without a sense of incongruity.

The figure which shows the outline | summary of the periphery monitoring system of the vehicle which concerns on embodiment of this invention. 1 is a block diagram of a vehicle periphery monitoring system according to an embodiment of the present invention. Explanatory drawing showing an example of perspective correction according to the present invention Explanatory drawing which shows an example of the lateral distance feeling correction which concerns on this invention The figure which shows an example of the image after performing correction processing in FIG.3 and FIG.4 Explanatory drawing which shows the other examples of perspective correction and lateral distance correction which concern on this invention The figure which shows an example of the imaging | photography range of the periphery monitoring apparatus of the conventional vehicle The figure which shows an example of the picked-up image display form of the periphery monitoring apparatus of the conventional vehicle

Explanation of symbols

W Wide-angle image 21 Perspective correction means 22 Side distance correction means

Claims (2)

A wide-angle image correction method for displaying on a display device in a vehicle interior a wide-angle image having a wide viewing angle in a horizontal direction obtained by photographing the outside of a vehicle,
The wide-angle image for displaying on the display device in the vehicle interior is an image in which the traveling direction of the vehicle is located in the center in the horizontal direction,
An annual ring-shaped equidistant line indicating a position equidistant from the center point set at the center in the horizontal direction at the lower end of the wide-angle image is set on the wide-angle image.
A method of correcting a wide-angle image in which, after setting the equidistant lines, the wide-angle image is enlarged and reduced so that the equidistant lines on the wide-angle image are evenly spaced . Image input means for obtaining the wide-angle image;
Image processing means for correcting the wide-angle image by the wide-angle image correction method according to claim 1 ;
A vehicle periphery monitoring system comprising display means for displaying the corrected wide-angle image.

Images