JP2007028443A - Image display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display system capable of providing an image enabling a user to visually recognize the rear and rear side of a vehicle. <P>SOLUTION: The image display system 10 is provided with a camera 12 attached on a vehicle and having a wide angle view field in the rear of the vehicle; a first image processor 32 for executing distortion correction for an image to be output from the camera; a second image processing means 36 for dividing the distortion-corrected image into an image of a first region R4 extending in a longitudinal direction with a predetermined width through the center of the image, an image of a second region R5 of one side for the first region in a lateral direction, and an image of a third region R6 of the other side, projecting the image of the second region onto a projection surface IP2 crossing an edge of one side of a projection surface IP4 corresponding to the image of the first region and slanted to an optical center O side, and projecting the image of the third region onto a projection surface IP6 crossing an edge of the other side of the projection surface corresponding to the image of the first region and slanted to the optical center side; and a display device 16 for displaying a display target image based on an image to be output from the second image processing means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an in-vehicle image display system, which relates to an image display system for displaying rear and rear side images of a vehicle.

There are various image display systems for displaying an image behind the vehicle. For example, JP 2003-212041 A discloses an image display system having a back camera.
JP 2003-212041 A

  The above-described image display system is intended to assist a driver who parks backwards, and can be used to obtain information on the rear and rear sides of the vehicle while traveling. However, normally, in the image taken by the back camera, the rear and rear side images are projected on the same projection plane. Therefore, it is difficult for the image to see information on the rear and rear sides of the vehicle. The rear side means a side region with respect to the rear of the vehicle.

  Therefore, an object of the present invention is to provide an image display system capable of providing an image that can be easily viewed from the rear and rear sides of a vehicle.

  The image display system of the present invention includes a camera, a first image processing unit, a second image processing unit, and a display device. The camera is attached to the vehicle. This camera has a wide-angle field of view behind the vehicle. The first image processing means performs distortion correction on the image output from the camera. The second image processing means converts the image subjected to the distortion correction by the first image processing means, the first area image extending in the vertical direction with a predetermined width in the center of the image, and the first image in the horizontal direction. Is divided into an image of the second region on one side and an image of the third region on the other side with respect to the region of, crossing the edge on one side of the projection plane corresponding to the image of the first region, and An image of the second area is projected onto a projection plane that tilts toward the optical center, and the second plane is projected onto the projection plane that intersects the other edge of the projection plane corresponding to the image of the first area and tilts toward the optical center. The image of area 3 is projected. The display device displays a display target image based on the image output from the second image processing means.

  According to this image display system, distortion of an image captured by a camera having a wide-angle field of view is corrected, and the image whose distortion is corrected is projected onto a projection surface with a more uniform distance from the optical center. At this time, the image of the first area corresponding to the rear of the vehicle in the image whose distortion has been corrected differs from the image of the second area and the third area corresponding to the rear side of the vehicle. Projected onto a surface. As a result, it is easier to visually recognize the rear and rear side of the vehicle, and a natural image is displayed on the display device.

  The image display system of the present invention determines whether the vehicle speed detected by the vehicle speed detection means is equal to or higher than a predetermined speed, and determines whether the vehicle speed is higher than a predetermined speed by the determination means. In this case, it is preferable to further include display selection means for displaying the display target image on the display device.

  According to this configuration, the display target image is displayed on the display device only when the speed of the vehicle is equal to or higher than a predetermined speed, that is, particularly when information on the rear side is required.

  ADVANTAGE OF THE INVENTION According to this invention, the image display system which can provide the image which is easy to visually recognize the back and back side of a vehicle is provided.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals. FIG. 1 is a diagram showing a configuration of an image display system according to an embodiment of the present invention. An image display system 10 illustrated in FIG. 1 includes a camera 12, an image processing ECU (Electric Control Unit) 14, and a display (display device) 16.

  The camera 12 is attached to a vehicle such as an automobile. FIG. 2 is a perspective view showing the vehicle to which the camera 12 is attached. As shown in FIG. 2, the camera 12 is attached to the rear portion of the vehicle 100.

The camera 12 has a wide-angle visual field of, for example, 130 degrees (reference symbol θ in the figure) or more. The camera 12 is to be inclined downwardly with respect to the axis Z of the optical axis Z _A extends horizontally and rearward is attached to the vehicle 100.

FIG. 3 is a diagram illustrating an example of an image captured by the wide-angle camera 12. The camera 12 has a wide-angle lens system such as a fisheye lens. Therefore, as shown in FIG. 3, the image (original image) output from the camera 12 includes distortion such as so-called barrel distortion. Further, since the optical axis Z _A is inclined downwardly, in the original image output from the camera 12, it is mainly manifested vehicle 100 behind the road surface.

  Returning to FIG. 1, the image output from the camera 12 is processed by the image processing ECU 14 and then displayed on the display 16 as a display target image. A vehicle speed sensor 104, a blinker 106, and a navigation unit 108 are connected to the image processing ECU 14 via a CAN (Control Area Network) 102.

  The vehicle speed sensor 104 detects the speed of the vehicle 100 and transmits a vehicle speed signal for specifying the speed to the CAN 102. An example of the vehicle speed sensor 104 is a wheel speed sensor attached to a wheel. In addition, a direction indication signal output when a direction indication is given by the blinker 106 is transmitted on the CAN 102.

  The navigation unit 108 acquires current position information for specifying the current position of the vehicle 100, and displays a map image with the current position on the map on the display 16 via the image processing ECU 14. A position specifying signal output from the navigation unit 108 is transmitted on the CAN 102 when the current position of the vehicle 100 is on a road traveling at a predetermined speed or higher (for example, on an expressway or an automobile-only road). Has been.

  Hereinafter, the image processing ECU 14 will be described in detail. The image processing ECU 14 includes an image processing ASIC (Application Specific Integrated Circuit) 20, a camera interface (I / F) 22, a display interface (I / F) 24, a CPU 26, a nonvolatile memory 28, and a frame memory 30 such as an SDRAM. Have.

  The ASIC 20 is connected to the camera 12 via the camera I / F 22. The ASIC 20 is connected to the display 16 via the display I / F 24. An image input from the camera 12 via the camera I / F 22 is stored in the frame memory 30 via the ASIC 20. In the frame memory 30, display target images processed by the ASIC 20 are stored, and the display target images are displayed on the display 16 via the display I / F 24.

  The ASIC 20 includes a first image processing unit (first image processing unit) 32, a second image processing unit 34, and a third image processing unit (second image processing unit) 36. Specifically, the first image processing unit 32, the second image processing unit 34, and the third image processing unit 36 are each configured in the ASIC 20 as dedicated logic. The first image processing unit 32, the second image processing unit 34, and the third image processing unit 36 may be configured as a function of the ASIC 20 that operates by a dedicated program.

  The first image processing unit 32 generates a first image by executing distortion correction on the image captured by the camera 12. This distortion correction is executed by referring to a table stored in the nonvolatile memory 28, for example.

  In the table stored in the nonvolatile memory 28, the correspondence between the pixel position before distortion correction and the pixel position after distortion correction is registered. By referring to this table, the first image processing unit 32 interpolates the pixel value of each pixel in the first image from the pixel value of the pixel in the vicinity of the corresponding position in the image (original image) before correction. Ask by. FIG. 4 is a diagram illustrating an image (first image) after distortion correction of the image illustrated in FIG. 3. As shown in FIG. 4, according to the first image processing unit 32, distortion of an image photographed by the camera 12 is corrected.

The second image processing unit 34 generates a second image by performing optical axis conversion on the first image. FIG. 5 is a diagram for explaining the concept of optical axis conversion. In FIG. 5, O is the optical center. Axis Y _A and axis Z _A is an axis of the camera coordinate system before the transformation optical axis, respectively, the axis Z _A is the optical axis of the camera 12, the axis Y _A vertical direction of the image plane of the projection surface IP1 That camera 12 Axis. An axis Y and an axis Z are axes of the camera coordinate system after optical axis conversion, the axis Z is the above-described axis extending horizontally rearward of the vehicle, and the axis Y is a projection plane IP2 (image plane orthogonal to the Z axis) IP2) is the vertical axis. The angle beta, and the angle between the axis Z and the optical axis Z _A, the angle [psi, is the angle which the point P and the optical center O is formed. The point P is a point to be imaged, and the Y coordinate is Y _P and the Z coordinate is Z _P. Point P1 is an image of point P before optical axis conversion, and its Y coordinate is y1. Point P2 is an image of point P after optical axis conversion, and its Y coordinate is y2.

As shown in FIG. 5, the optical axis converter according to the second image processing section 34 is a process for converting the optical axis Z _A to the optical axis Z. That is, according to the optical axis conversion, original image formed by the projection on an image plane IP1 perpendicular to the optical axis Z _A is projected on the image plane IP2 perpendicular to the axis Z. Specifically, the optical axis conversion by the first image processing unit 32 is realized by a calculation based on the following equation (1).

  The second image processing unit 34 obtains the coordinates of the Y coordinate y1 before conversion corresponding to the Y coordinate y2 of each pixel in the second image after the optical axis conversion, thereby obtaining the first of the pixels of the second image. The corresponding position in one image is obtained. Then, the second image processing unit 34 determines the pixel value of the pixel in the second image by interpolating the pixel value from the pixel value of the first image near the corresponding position. In the present embodiment, the second image generated in this way is displayed on the display 16 as a display target image.

  FIG. 6 is a diagram illustrating a second image after the optical axis conversion of the image illustrated in FIG. 4. As shown in FIG. 6, according to this optical axis conversion, the ratio of the landscape above the road surface to the second image is large. In addition, as shown in FIG. 6, a subject extending in the vertical direction (for example, a frame on a fence appearing on the upper side of the image) extends in the vertical direction also in the second image. That is, the second image is a natural image that is easily visible from the rear and rear sides of the vehicle 100.

  The third image processing unit (second image processing means) 36 divides the second image into a region corresponding to the rear of the vehicle and a region corresponding to the rear side of the vehicle, and sets them on different projection planes. Projecting to generate a third image. FIG. 7 is a diagram for explaining the concept of projection plane conversion. 7A shows the relationship between the projection plane corresponding to the second image (projection plane IP2 shown in FIG. 5) and the projection plane used for projection plane conversion. FIG. 7B shows an image area corresponding to each projection plane after the projection conversion process.

  In FIG. 7A, the axes X and Z are axes of the camera coordinate system before the projection plane conversion, the axis Z is an axis extending horizontally behind the vehicle 100, and the axis X is the width of the vehicle 102. The axis of direction. Projection planes used for this projection plane conversion include projection planes IP4, IP5, and IP6. The projection plane IP4 is a projection plane extending in the vertical direction with a width 2L in the center of the projection plane IP2. The image projected on the projection plane IP4 in the projection plane IP2 does not change even after the projection conversion. The projection plane IP5 is a projection plane that intersects one edge of the projection plane IP4 in the horizontal direction and is tilted toward the optical center O side. The projection plane IP6 is a projection plane that intersects with the other edge of the projection plane IP4 in the horizontal direction and is tilted toward the optical center O side. Both projection planes IP5 and IP6 intersect with projection plane IP4 at an angle α.

  Point P1 is a point on the second image, and its X coordinate is x1. A line connecting the point P1 and the optical center O intersects the Z axis at an angle γ. The point P2 is a corresponding point on the projection plane IP6 of the point P1, and its X coordinate is x2.

  The third image processing unit 36 has a first region R1 extending in the longitudinal direction with a length L on one side and a length L on the other side around a point C on the optical axis, and a first region R1 The second image is divided into a second region R2 on one side and a third region R3 on the other side of the first region. In addition, 2L which is the width | variety (predetermined width | variety) of 1st area | region R1 is made to respond | correspond to the width | variety of a vehicle, for example. As a result, the image of the first region R1 more reliably corresponds to the image behind the vehicle.

この演算によって、第３の画像の各画素の第２の画像における対応位置が求められる。第３の画像処理部３６は、当該対応位置近傍の第２の画像の画素値を用いた補間によって、第３の画像の画素の画素値を算出する。なお、この補間には、例えば、線形補間、バイキュービック補間といった補間を用いることができる。 Then, the third image processing unit 36 obtains the X coordinate x1 in the second image corresponding to the X coordinate x2 of each pixel in the third image by the calculation of the following expressions (2) and (3).

By this calculation, the corresponding position in the second image of each pixel of the third image is obtained. The third image processing unit 36 calculates the pixel value of the pixel of the third image by interpolation using the pixel value of the second image near the corresponding position. For this interpolation, for example, interpolation such as linear interpolation or bicubic interpolation can be used.

  In the third image generated by such projection plane conversion, the image of the first region R1 is copied as it is into the region R4 extending in the vertical direction at the center of the image. The image of the second region R2 is reduced in the region R5 on one side with respect to the region R4. Further, the image of the third region R3 is reduced in the region R6 on the other side with respect to the region R1.

  FIG. 8 is a third image after the projection plane conversion of the second image shown in FIG. According to the projection plane conversion by the third image processing unit 36, the second image is projected onto a projection plane having a more uniform distance from the optical center O. Further, according to the projection plane conversion by the third image processing unit 36, the first region R1 corresponding to the rear of the vehicle, the second region R2 and the third region R3 corresponding to the rear side of the vehicle, Are projected on different projection planes, so that it is easy to distinguish the images of the respective areas. Therefore, as shown in FIG. 8, the third image (display target image) is an image that is more natural and easy to see for the driver who observes the rear and rear sides of the vehicle 100.

  Returning to FIG. 1 again, the CPU 26 operates as the determination unit 38 and the selection unit 40. The determination part 38 and the selection part 40 are comprised by CPU26 which operate | moves by the command of a dedicated program module.

  The determination unit 38 acquires a vehicle speed signal from the vehicle speed sensor 104 and determines whether or not the forward speed of the vehicle 100 is equal to or higher than a predetermined speed from the vehicle speed signal. As this predetermined speed, for example, a speed of 40 km / h can be employed.

  When the determination unit 38 determines that the forward speed of the vehicle 100 is equal to or higher than a predetermined speed, the selection unit 40 displays the display target image generated by the ASIC 20. For example, the display target image generated by the ASIC 20 is displayed on the display 16 that displays another image such as a map image from the navigation unit 108. The determination unit 38 includes (1) that the speed of the vehicle 100 is equal to or higher than a predetermined speed, (2) that a direction instruction signal is output from the winker 106 of the vehicle 100, and (3) the navigation unit 108. The determination may be made using a combination of at least one of the fact that the position specifying signal is output from.

  Hereinafter, the flow of processing by the ASIC 20 will be described. FIG. 7 is a flowchart showing the flow of processing in the ASIC 20. In the ASIC 20, first, the first image processing unit 32 applies distortion correction to an image (original image) taken by the camera 12 (step S01). As a result, a first image is generated.

  Next, in the ASIC 20, the second image processing unit 34 applies the above-described optical axis conversion to the first image (step S02). As a result, a second image is generated. This second image is displayed on the display 16 as a display target image. Subsequently, projection plane conversion is applied to the second image by the third image processing unit 36 (step S03). As a result, a third image is generated. In the present embodiment, the third image generated and output by the third image processing unit 36 is displayed on the display 16 as a display target image.

  As described above, the display target image can be easily distinguished from the rear and rear side images. For example, in order to change the lane during high-speed forward, the driver assists in confirming the own lane or the succeeding vehicle in the adjacent lane. It is effective for.

  In addition, as shown in FIG. 10, it is also suitable to display on the display 16 the 3rd image which attached | subjected the mask M which simulated the inside of a vehicle. By performing the mask processing in this way, it becomes easier to intuitively recognize the distinction between the rear side and the rear side in the image displayed on the display 16. The information of the mask M is stored in the nonvolatile memory 28, for example. Further, the mask processing may be performed on the image whose optical axis has been converted by the second image processing unit 34, or the second image before the projection plane conversion by the third image processing unit 36. May be executed.

  It is possible to generate a desired image according to the purpose by changing the angle α formed by the projection plane IP5 and the projection plane IP6 with the projection plane IP4 and the width 2L of the projection plane IP4. Further, the images of the regions R5 and R6 may be enlarged so as to fit the display size of the display 12.

  Furthermore, the camera 12 may be attached to the vehicle so that the optical axis thereof is horizontal, and the installation position is not limited to the rear part of the vehicle, and the field of view only needs to be directed backward. And when the camera 12 is attached to a vehicle so that an optical axis may become horizontal previously, the 2nd image process part 34 does not necessarily need to be provided. In this case, the third image processing unit (second image processing unit) 36 may perform the enlargement / reduction process on the first image whose distortion has been corrected by the first image processing unit 32.

FIG. 1 is a diagram showing a configuration of an image display system according to an embodiment of the present invention. FIG. 2 is a perspective view showing a vehicle to which a camera is attached. FIG. 3 is a diagram illustrating an example of an image captured by a wide-angle camera. FIG. 4 is a diagram illustrating a first image after distortion correction of the image illustrated in FIG. 3. FIG. 5 is a diagram for explaining the concept of optical axis conversion. FIG. 6 is a diagram illustrating a second image after the optical axis conversion of the image illustrated in FIG. 4. FIG. 7 is a diagram for explaining the concept of projection plane conversion. FIG. 8 is a fourth image after the projection plane conversion of the image shown in FIG. FIG. 9 is a flowchart showing the flow of processing in the ASIC shown in FIG. FIG. 10 is a third image after the mask has been applied.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Image display system, 12 ... Camera, 14 ... Image processing ECU, 16 ... Display (display apparatus), 20 ... Image processing ASIC, 26 ... CPU, 28 ... Nonvolatile memory, 30 ... Frame memory, 32 ... 1st Image processing unit, 34 ... second image processing unit, 36 ... third image processing unit (second image processing means), 38 ... determination unit, 40 ... selection unit.

Claims (2)

A camera attached to the vehicle and having a wide-angle field of view behind the vehicle;
First image processing means for performing distortion correction on an image output from the camera;
The image subjected to the distortion correction by the first image processing means is divided into an image of a first area extending in the vertical direction with a predetermined width at the center of the image, and the first area in the horizontal direction. The image is divided into an image of the second region on one side and an image of the third region on the other side, intersects the edge on the one side of the projection surface corresponding to the image of the first region and is on the optical center side Projecting the image of the second region onto the projection surface tilting toward the projection surface, intersecting the other edge of the projection surface corresponding to the image of the first region and tilting toward the optical center side Second image processing means for projecting an image of the third region;
A display device for displaying a display target image based on the image output from the second image processing means;
An image display system comprising: Determining means for determining whether the speed of the vehicle detected by the vehicle speed detecting means is equal to or higher than a predetermined speed;
Selection means for displaying the display target image on the display device when the determination means determines that the speed of the vehicle is equal to or higher than the predetermined speed;
The image display system according to claim 1, further comprising:

Images