JP5226621B2 - Image display device for vehicle - Google Patents

Description

  The present invention relates to a vehicular image display device that mounts a plurality of cameras on a vehicle and synthesizes and displays images captured by the plurality of cameras.

  2. Description of the Related Art Conventionally, there is a vehicle image display device in which a plurality of cameras are mounted on a vehicle, an image around the vehicle is captured, and the captured image is displayed on a display or the like provided on the vehicle. In such a vehicle image display device, by confirming the image displayed on the display device, the distance to the obstacle around the vehicle can be grasped, and the vehicle collides with or collides with the obstacle. Can be prevented.

  In addition, recently, when images captured by a plurality of cameras are combined and displayed, a top view image (overhead image) of the host vehicle is generated and the top view image is displayed on the combined image. . Displaying the top view image of the host vehicle makes it easier to grasp the surrounding situation, and can assist the driver in driving. The top view image is an image viewed from a virtual viewpoint set above the host vehicle, and the appearance of the composite image can be changed by changing the position of the virtual viewpoint.

  Patent Document 1 discloses an image processing apparatus that inputs images taken by a plurality of cameras, generates a composite image viewed from a virtual viewpoint, and displays an illustration image showing the host vehicle.

  By the way, in the conventional vehicular image display device equipped with a plurality of cameras, a blind spot may occur when shooting a surrounding video. In order to eliminate the blind spot, it is necessary to increase the number of cameras, which is not preferable because the cost and the burden of image processing increase.

  Patent Document 2 discloses a vehicle driving support device in which four cameras are installed, that is, cameras are installed at the right end and the left end of the front bumper of the vehicle and the right end and the left end of the rear bumper, respectively. In this example, the outside of the vehicle including the blind spot area of the driver is photographed, and the parking pattern when the vehicle was parked last time is stored in the storage means.

  Patent Document 3 discloses an in-vehicle driving support information display device equipped with a camera for photographing the left rear, the right rear, the front, and the rear. In this example, the display screen is divided into three, the left screen displays a camera image that is diagonally left rear, the right screen displays a camera image that is diagonally right rear, and the center screen displays the front or blind spot of the host vehicle. The left and right front or rear camera images are displayed.

  However, in the display devices described in the related art and Patent Documents 1 to 3, when the vehicle travels, for example, when making a right turn at an intersection, turning right or left at a T-junction, or entering a road while backing, A surrounding image may be hidden by a part of the host vehicle, and a blind spot may occur in a captured image. For this reason, further improvement is required.

Japanese Patent No. 3300334 JP 2007-118876 A Japanese Patent Laid-Open No. 2001-114048

  In a conventional vehicle image display device, when a vehicle turns right at an intersection, turns right / left on a T-junction, or enters a road while backing up, the surrounding image is hidden by a part of the own vehicle. In some cases, blind spots occurred in the captured images. For this reason, if the number of cameras is increased, the cost is increased and the burden of image processing is increased. Therefore, further improvement has been desired.

  In view of such circumstances, an object of the present invention is to provide a vehicular image display apparatus that reduces the number of blind spots with the minimum number of cameras necessary to generate a bird's-eye view image.

The vehicle image display apparatus according to claim 1, wherein the way before the vehicle, constituted by the rear and the left, right directions of the four cameras having viewing angle of 180 degrees or more for each shooting the image of the predetermined range, adjacent to each other The camera can be photographed in all directions around the vehicle by connecting the camera images, and is arranged at a corner on the opposite lane side of the front portion of the vehicle and tilted by a preset angle from the front toward the opposite lane side A first camera attached to the vehicle, and second and third cameras distributed from the first camera and arranged on the left and right corners of the vehicle and attached to the left and right sides of the vehicle, From an image pick-up section comprising a fourth camera disposed behind the vehicle and mounted rearward from the vehicle, an image generation section for generating an overhead view image of the vehicle, and images taken by the four cameras According to the progress of the vehicle A main image is set, a camera image including the main image is processed to generate a background image, an image composition unit that combines the background image and the overhead image, and an image synthesized by the image composition unit is displayed. And a possible display portion.

  According to the image display device for a vehicle of the present invention, it is possible to photograph all directions around the vehicle with four cameras attached to the front and rear and left and right corners of the host vehicle, and the blind spots can be reduced. Further, by selecting and displaying the main image according to the progress of the vehicle, obstacles such as oncoming vehicles can be displayed accurately, and safe driving can be supported.

1 is a block diagram showing an overall configuration of a vehicle image display apparatus according to an embodiment of the present invention. Explanatory drawing which shows roughly an example of the background image of the vehicle produced | generated using a some camera image. The top view which shows the example of arrangement | positioning of the several camera to a vehicle. The top view which shows the imaging | photography direction and viewing angle of a some camera. Explanatory drawing which shows roughly arrangement | positioning and the description range of several cameras for left-hand traffic and right-hand traffic. Explanatory drawing which shows schematically the other example of arrangement | positioning and description range of the some camera for left-hand traffic and right-hand traffic. Explanatory drawing which shows the general vehicle progress situation at the time of turning right at an intersection. Explanatory drawing which shows the advancing condition of the vehicle in this embodiment at the time of turning right at an intersection. Explanatory drawing which shows the advancing condition of the vehicle in this embodiment at the time of turning left at a T-junction. Explanatory drawing which shows the advancing condition of the vehicle in this embodiment at the time of turning right at a T-junction. Explanatory drawing which shows the advancing condition of the vehicle in this embodiment when approaching a roadway in back. Explanatory drawing which shows the other advancing condition of the vehicle in this embodiment when approaching a roadway in back. 6 is a flowchart showing a display image switching operation in the present embodiment.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing the overall configuration of a vehicle image display apparatus according to an embodiment of the present invention. The vehicle image display apparatus of FIG. 1 includes an imaging unit 10, a vehicle information generation unit 20, a display pattern storage unit 30, a display pattern determination unit 40, an image composition unit 50, and a display unit 60.

  The imaging unit 10 includes four cameras 11, 12, 13, and 14 that capture images around the host vehicle. The four cameras 11, 12, 13, and 14 shoot the front, rear, left side, and right side of the vehicle, respectively, and are preferably attached at the same height position of the vehicle. Each camera 11-14 uses, for example, a fisheye camera in order to photograph the entire field of view (360 degrees). The camera position and shooting range will be described later.

  The vehicle information generation unit 20 includes a navigation device 21. The navigation device 21 includes a GPS receiving unit that detects the current position of the host vehicle, a recording medium that stores map data, a search unit that searches for a route to the destination, and the like. It also has a vehicle speed sensor and a gyro sensor that detect the vehicle speed and inclination of the host vehicle, and these GPS receivers, recording media, search units, vehicle speed sensors, gyro sensors, etc. have an in-vehicle LAN such as a CAN (Controller Area Network). Connected through. Therefore, information such as the current position of the host vehicle and the traveling direction (forward, back, right / left turn) can be obtained from the vehicle information generation unit 20.

  The display pattern storage unit 30 includes an image storage unit 31 and a mapping table 32. The image storage unit 31 stores a bird's-eye view image such as a top view image viewed from directly above the host vehicle or an image viewed obliquely from behind. That is, the cameras 11 to 14 each photograph the surroundings of the own vehicle, and generate a background image of the own vehicle based on the photographed image, but only the actual scene is displayed on the background image. Therefore, in order to synthesize an overhead image at a predetermined position of the background image, an image obtained by photographing the host vehicle from directly above or an image viewed obliquely from the rear is stored in the image storage unit 31 as an overhead image. Or the illustration image which shows the own vehicle is produced | generated and this illustration image is memorize | stored as a bird's-eye view image. The number of captured images and illustration images is not limited to one, but a plurality of types are stored so that arbitrary images can be selected.

  The mapping table 32 is a table that describes the correspondence between the pixel data of the images captured by the cameras 11 to 14 and the pixel data of the background image obtained by viewing the periphery of the host vehicle from the upper virtual viewpoint. Conversion information indicating which pixel of the background image corresponds to the pixel of the.

  The display pattern determination unit 40 determines an overhead image necessary for generating a composite image from the image data stored in the image storage unit 31. The image storage unit 31 and the display pattern determination unit 40 constitute an image generation unit. The image synthesizing unit 50 synthesizes the front image, the rear image, the left side image, and the right side image of the vehicle photographed by the cameras 11 to 14 with reference to the mapping table 32 to generate a background image. Further, the overhead image determined by the display pattern determination unit 40 is synthesized with this background image. By referring to the mapping table 32, the background image displayed in accordance with the viewpoint changes when the position of the virtual viewpoint changes. Details of a background image generation method using a mapping table are described in Patent Document 1, for example.

  In addition, the image composition unit 50 obtains the vehicle position, traveling direction, and angle information from the vehicle information generation unit 20, determines the moving direction (forward, back) of the vehicle and the turning direction (right turn / left turn), and The captured images of the cameras 11 to 14 are selected according to the above and the composition process is performed. The display unit 60 displays a composite image of the background image and the bird's-eye view image of the host vehicle, and includes a liquid crystal display or the like.

  FIG. 2A schematically shows a composite image obtained when the entire periphery of the vehicle is photographed by the cameras 11 to 14. FIG. 2A conceptually shows a panoramic display of the composite image P obtained by photographing the front part a and the rear part b and the left side part c and the right side part d of the vehicle 1.

  FIG. 2B shows another example of a composite image of the background image displayed on the display unit 60 and the overhead image (top view image). In FIG. 2 (b), the front image of the vehicle, the left side image, and the right side image are mainly combined to panorama display the background image P0 in the periphery in the traveling direction, and the top view image P1 of the vehicle is displayed on the background image P0. An example of the combined display is shown. In FIG. 2B, the position of the virtual viewpoint is set directly above and behind the vehicle. However, when the position of the virtual viewpoint is changed, the display of the background image P0 also changes.

  Next, a specific example of the arrangement of the cameras 11 to 14 will be described. That is, when a plurality of cameras are arranged on a vehicle, images taken by four cameras are required to create an overhead image. In addition, in order to show the end of each camera image neatly, it is necessary to attach each camera to the center of the front, rear, left and right sides of the vehicle body. However, it is not sufficient to display a clean image, and it is necessary to eliminate a blind spot in order to provide information that can be driven more safely. In consideration of such points, in this embodiment, the cameras 11 to 14 are arranged as shown in FIG.

  FIG. 3 is a view of the vehicle 1 as viewed from directly above, and four cameras 11 to 14 are attached to the vehicle 1. For example, in the case of left-hand traffic, the camera 11 is arranged at the front right end (that is, the opposite lane side) of the vehicle 1 and the camera 12 is arranged diagonally behind the camera 11 at the left end. In addition, a camera 13 is disposed on the left front portion of the vehicle 1, and a camera 14 is disposed diagonally with the camera 13 on the right rear portion. Thus, the cameras 11 to 14 are arranged at the corners of the vehicle 1. In the following description, the vehicle 1 including the cameras 11 to 14 may be referred to as the own vehicle 1.

  FIG. 4 is a plan view showing the photographing direction of the cameras 11 to 14 and the basic viewing angle (drawing range). Arrows a1, b1, c1, and d1 indicate the directions of the cameras 11, 12, 13, and 14, respectively. The camera 11 captures the front of the vehicle 1 but is attached so as to face a direction a1 slightly inclined from the front toward the opposite lane (right side). The camera 12 captures the rear of the vehicle 1, the camera 13 captures the left side of the vehicle 1, and the camera 14 captures the right side of the vehicle 1. , D1).

  In addition, the depiction ranges of the cameras 11 to 14 are indicated by w1, w2, w3, and w4, and the photographing ranges of the cameras 11 to 14 are wider than 180 degrees so that no blind spots are generated around the entire vehicle 1. doing. As described above, since the cameras 11 to 14 are installed slightly tilted to the right with respect to the front, rear, left, and right directions of the vehicle 1, each camera 11 to 14 has a wider field of view on the right side. In the following description, the camera 11 may be referred to as a front camera, the camera 12 as a rear camera, the camera 13 as a left camera, and the camera 14 as a right camera.

  FIG. 5 is a diagram schematically showing the arrangement and depiction range of the cameras 11 to 14 in FIG. 4. FIG. 5 (a) shows left-hand traffic, and FIG. 5 (b) shows right-hand traffic. . The right-hand traffic is a left-right target arrangement with respect to the left-hand traffic camera arrangement (a).

  FIG. 6 shows a modified example of the arrangement of the cameras 11 to 14, and is an example in which the rear camera 12 is installed at the center of the rear end of the vehicle 1 so as to face directly behind the vehicle 1. FIG. 6A shows left-hand traffic, and FIG. 6B shows right-hand traffic. In the following, an example for left-hand traffic will be described in accordance with Japanese laws and regulations.

  The present invention is characterized in that when the captured images of the cameras 11 to 14 are combined in the image combining unit 50, a main image is selected from a plurality of camera images according to the progress of the vehicle, and the combining process is performed. is there. This will be specifically described below.

  FIG. 7 shows the progress of the host vehicle 1 when turning right at the intersection. FIG. 7 is a diagram showing a general photographing method in which the camera 11 is attached to the front center portion of the host vehicle 1. When the camera 11 is attached to the front center of the host vehicle 1 and the right turn vehicle 2 is in the opposite lane when turning right at the intersection, it is difficult to see another oncoming vehicle 3 behind the right turn vehicle 2. For this reason, in the image photographed by the camera 11, the oncoming vehicle 3 is hidden behind the right turn vehicle 2, and the oncoming vehicle 3 cannot be captured sufficiently.

  Therefore, as shown in FIG. 8, the position of the camera 11 is installed at the front right end. In FIG. 8, even when there is an opposite right turn vehicle 2, the image of the oncoming vehicle 3 can be captured by the front camera 11. Therefore, when the host vehicle 1 makes a right turn, an image captured by the front camera 11 is used as a main image and is combined with images captured by the left camera 13 and the right camera 14 so that an obstacle during a right turn can be accurately grasped. .

  FIG. 9 shows the progress of the host vehicle 1 when turning left on the T-junction. When the front camera 11 is attached to the front center portion of the vehicle 1, when the direction of the vehicle is tilted to the left on the T-junction, the vehicle 4 coming from the right side cannot be photographed because the own vehicle 1 becomes a blind spot. However, since the front camera 11 is installed at the front right end, it is possible to shoot a video on the right side with the front camera 11 and capture the vehicle 4 coming from the right side. In addition, the left camera 13 can take a video of the left side of the vehicle 1. Therefore, when making a left turn, it is possible to accurately grasp an obstacle at the time of a left turn by combining the captured images of the front camera 11 and the left camera 13 as a main image.

  Also, as shown in FIG. 10, when turning right on the T-junction, the vehicle 1 may be temporarily stopped by tilting the vehicle 1 in the right direction. At this time, the right camera can shoot the right image with the front camera 11, but the left side of the front camera 11 is hidden behind the host vehicle 1. However, since the left camera 13 is installed at the front end, the left camera 13 can capture the left image, and the vehicle 5 coming from the left side can be captured. Therefore, when making a right turn at the T-junction, it is possible to accurately grasp the obstacle at the time of the right turn by combining the captured images of the front camera 11 and the left camera 13 as the main image.

  FIG. 11 shows the progress of the host vehicle 1 when entering the front lane of a wide road while backing. In this case, turn back and turn left to enter the front lane, and then head forward. At this time, since the rear camera 12 is installed at the rear left end, the rear camera 12 can shoot the left image, and the rear camera 12 can capture the oncoming vehicle 7 and the like.

  Further, if the vehicle turns leftward at the back, the rear camera 12 becomes blind spot on the right side due to the own vehicle and cannot catch a walking car or the like. However, since the right camera 14 is installed at the right rear end, the right camera 14 can shoot a video on the right side, and can catch an oncoming vehicle 6 or a pedestrian.

  Also, as shown in FIG. 12, when the host vehicle 1 is advanced to the far lane of a wide road and then moves forward to the right or left side, the right camera 14 is similarly used to shoot the right side image and the rear camera. 12, it is possible to check the presence or absence of oncoming vehicles 6 and 7 by taking a video on the left side.

  Therefore, when entering from a narrow parking lot or the like while back on the road, the obstacles can be accurately grasped by combining the captured images of the rear camera 12 and the right camera 14 as the main image.

  Whether the road is a crossroad or a T-shaped road can be determined based on information from the navigation device 20, whether the vehicle 1 is going to turn right, left, or back. Can be determined from vehicle information such as a gyro sensor and a steering angle. The image composition unit 50 can select which camera image is the main image based on these determination results.

  When the road is on the right side, the camera mounting positions are symmetrical as shown in FIG. 5B, so the main image (especially the left camera and right camera images) can also be selected with a right turn. In accordance with the left turn, a symmetric image is selected for left-hand traffic.

  In the embodiment described above, as shown in FIG. 4 or 5, the positions of the front camera 11, the rear camera 12, the left camera 13, and the right camera 14 are shifted to positions close to the corners of the vehicle. The entire periphery of the host vehicle 1 can be photographed without omission. Further, by selecting the main image according to the progress of the vehicle 1, the obstacle can be accurately grasped, and the driver can be safely driven. In addition, since the ground around the host vehicle 1 can be taken, an overhead image from above the vehicle 1 can also be displayed.

  In addition, when driving the host vehicle 1, driving is performed while viewing the camera image (back image) of the rear camera 12, but the rear camera 12 is displaced from the center portion, so that it is difficult to understand the sense of distance in the horizontal direction, and it is uncomfortable. May occur. Therefore, when backing, assuming that the image of the rear camera 12 is viewed, only the rear camera 12 may be installed at the rear center as shown in FIG.

  FIG. 13 is a flowchart illustrating a display image switching operation according to the embodiment of the present invention. It is assumed that the cameras 11 to 14 are arranged at the positions shown in FIG. In FIG. 13, step S1 selects an overhead image read from the image storage unit 31, and creates an illustration image, for example. In step S2, a joint is set when the images captured by the cameras 11 to 14 are connected to generate a display image. For example, the cameras 11 to 14 are connected so that the right side can be seen as much as possible.

  Steps S <b> 3 to S <b> 7 are steps for switching the display in accordance with the progress of the host vehicle 1. The progress of the host vehicle 1 is determined, the main image is selected, and the display is switched. That is, in step S4, it is determined whether or not the host vehicle 1 turns right at the intersection. When making a right turn, the host vehicle 1 is moved in step S5 and enters the intersection. As a result, the oncoming vehicle can be photographed by the front camera 11. In step S6, the captured image of the front camera 11 is preferentially displayed as the main image. Therefore, when there is an oncoming vehicle turning right, it is possible to grasp whether another oncoming vehicle or a motorcycle is coming from behind.

  Thereafter, the process returns from step S7 to S3, and the display is switched in accordance with the next situation of the vehicle 1. If it is determined in step S4 that the vehicle does not turn right, it is determined in step S8 whether or not to enter a T-junction. When entering the T-junction, the host vehicle 1 is slightly inclined in step S9 and enters the T-junction. In step S10, the captured images of the left camera 13 and the front camera 11 are preferentially displayed as main images. Therefore, even if there are wrinkles on the left and right, it is possible to grasp whether another vehicle or the like has come from the left and right.

  When making a right turn at a T-junction, it is checked whether another vehicle is coming from the left based on the image captured by the left camera 13, and another vehicle is viewed from the right based on the image captured by the front camera 11. It can be confirmed whether or not. Similarly, when making a left turn, it is possible to check whether another vehicle is coming from the right side or the left side based on the images taken by the left camera 13 and the front camera 11.

  If it is determined in step S8 that the vehicle does not enter the T-junction, it is determined in step S11 whether or not the vehicle enters the road in the back. When entering the road in the back, in step S12, the vehicle enters at a right angle or an inclination with respect to the intersecting road. In step S13, the captured images of the rear camera 12 and the right camera 14 are preferentially displayed as the main image. Therefore, the rear camera 12 can grasp the situation on the left side and the right camera 14 can grasp the situation on the right side, so that it is possible to grasp whether another vehicle or the like is not coming from the left and right even if there are wrinkles on the left and right. .

  Note that the determination of NO in any of steps S4, S8, and S11 is when the vehicle 1 goes straight or turns left at the intersection, and the vehicle 1 moves forward relatively safely, so the main image is fixed in advance. For example, an image captured by the front camera 11, the left camera 13, and the right camera 14 may be set as a main image, and an image as shown in FIG. 2B may be displayed.

  As described above, according to the embodiment of the present invention, blind spots can be reduced by capturing images of all surrounding directions with the four cameras attached to the front and rear and the left and right corners around the host vehicle. In addition, since the minimum number of cameras necessary for generating the overhead view image such as the top view image is sufficient, the cost can be reduced. Furthermore, since the main image is selected and displayed according to the progress of the vehicle, obstacles such as oncoming vehicles can be displayed accurately, and safe driving can be supported.

  Various modifications can be considered for the embodiment described above. For example, since the lateral direction with respect to the vehicle 1 appears larger, the left camera 13 and the right camera 14 may be provided with a special lens so that pixels in the lateral direction are fine. Actually, since it is directed in a direction slightly inclined with respect to the lateral direction, the lateral direction is shifted by that amount, but if the lens is specially processed, it is possible to make the image in the lateral direction easier to see.

  Further, more than four cameras may be provided. In this case, the orientation and position of each camera may be shifted so that the direction in which the image is projected is not affected by the position and orientation of the attached camera. Basically, it seems natural to increase the number of cameras to four or more, so it is better to increase the number of cameras in the direction that you want to see more naturally. In addition, since the camera image overlaps in the increased portion, it is possible to cope with a wider range of shooting by shifting the position and orientation of other cameras.

  In addition, since the background in the front and rear of the vehicle 1 can be captured by the two cameras, the obstacle detection may be performed by performing stereo image processing. Obstacles can be easily detected by viewing in stereo. In addition, the cameras 11 to 14 are mounted using a specific mounting tool, such as a side mirror, a fender mirror or a side under mirror (a small mirror for viewing the ground just in front of the fender mirror), a rear under mirror, etc. It may be attached to the protrusion.

  The embodiment of the present invention is not limited to the configuration described above, and various modifications can be made without departing from the scope of the claims.

DESCRIPTION OF SYMBOLS 10 ... Imaging part 11, 12, 13, 14 ... Camera 20 ... Vehicle information generation part 21 ... Navigation apparatus 30 ... Display pattern storage part 31 ... Image storage part 32 ... Mapping table 40 ... Display pattern determination part 50 ... Image composition part 60 ... Display section

Claims (2)

How preceding vehicle, constituted by the rear and the left, four cameras having 180 degrees or more of viewing angle for the image respectively captured within a predetermined range in the right direction, the entire periphery of the vehicle by joining the camera image that are adjacent to each other A first camera that is capable of photographing an azimuth and is disposed at a corner portion on the opposite lane side of the front portion of the vehicle and attached in a direction inclined by a preset angle from the front toward the opposite lane side; 2nd and 3rd cameras distributed from the left and right corners of the vehicle and mounted on the left and right sides of the vehicle, and rearward of the vehicle and rearward of the vehicle An imaging unit consisting of a fourth camera attached toward
An image generator for generating an overhead image of the vehicle;
A main image is set according to the progress of the vehicle from images taken by the four cameras, the camera image including the main image is processed to generate a background image, and the background image and the overhead image are combined. An image composition unit to
An image display device for a vehicle, comprising: a display unit capable of displaying an image synthesized by the image synthesis unit. Said fourth camera characterized by being arranged diagonally position relative to the first camera according to claim 1 for a vehicle image display apparatus according.

Images