JP6657937B2 - Display device for vehicles - Google Patents

Description

  The present invention relates to a display device for a vehicle.

  2. Description of the Related Art Conventionally, a composition in which a corresponding part (a part that cannot be seen by the own vehicle) of a past image imaged before the own vehicle reaches the current position is superimposed on a part of the surrounding image captured by the on-vehicle camera that reflects the own vehicle. A technique for displaying an image together with a contour line of a vehicle body is known (for example, Patent Document 1).

  With this technique, the driver can perform the driving operation while grasping the situation of the blind spot area that is not visible behind the vehicle.

JP 2003-244688 A

  However, since the aerial portion of the blind spot (outside the imaging range) of the camera itself is not displayed, it may be difficult for the driver or the like to recognize the blind spot.

  In view of the above problems, it is an object of the present invention to provide a display device that enables a driver or the like to easily recognize a blind spot by displaying based on a captured image of a camera that captures an area around a host vehicle.

In order to achieve the above object, in one embodiment, the vehicle display device includes:
An imaging unit for imaging the periphery of the vehicle,
An image three-dimensionally showing an aerial region among regions outside the range imaged by the imaging unit, and an image showing the periphery of the own vehicle imaged by the imaging unit or an image imaged by the imaging unit An image generation unit that generates a display image to be displayed as one image by combining a virtual viewpoint image generated by performing viewpoint conversion on
A display unit for displaying the display image ,
The display unit,
A corner portion of a region outside the range imaged by the imaging section is displayed, and the portion other than the corner portion is omitted and displayed .

  According to the present embodiment, it is possible to provide a display device that allows a driver or the like to easily recognize a blind spot by displaying based on a captured image of a camera that captures an area around the own vehicle.

FIG. 2 is a block diagram illustrating an example of a configuration of a display device according to the embodiment. FIG. 3 is a diagram illustrating an imaging range of a camera. FIG. 4 is a diagram illustrating an example of a base image generated by a base image generation process. FIG. 6 is a diagram illustrating an example of an area outside a range where an image is captured. It is a figure showing the 1st example of a display picture. It is a figure showing the 2nd example of a display picture. It is a figure showing the 3rd example of a display picture.

  Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating an example of a configuration of a display device 1 according to the present embodiment.

  The display device 1 includes a camera 10, an ECU (Electric Control Unit) 20, and a display 30, and displays an image of the surroundings of the own vehicle (the vehicle on which the display device 1 is mounted) captured by the camera 10 on the display 30. Thereby, the driver of the own vehicle can perform the driving operation while checking the situation around the own vehicle displayed on the display 30.

  The camera 10 is an imaging unit that includes an imaging element such as a CMOS (Complementary Metal Oxide-Semiconductor) and captures the surroundings of the vehicle. The cameras 10 include a camera 10a, a camera 10b, a camera 10c, and a camera 10d that capture images of the front, rear, left, and right sides of the host vehicle, respectively.

  The camera 10a is provided, for example, at the center of the front grill of the host vehicle and captures an image of the front of the host vehicle.

  The camera 10b is provided, for example, at the center of the upper end of the rear window of the host vehicle, and captures an image of the rear of the host vehicle.

  The camera 10c is provided, for example, below the door mirror on the left side of the host vehicle and captures an image of the left side of the host vehicle.

  The camera 10d is provided, for example, below the door mirror on the right side of the host vehicle and captures an image of the right side of the host vehicle.

  Each of the cameras 10a to 10d is communicably connected to the ECU 20 via a one-to-one communication line, an in-vehicle LAN, or the like, and a captured image is transmitted to the ECU 20.

  Here, the imaging range of the camera 10 will be described with reference to FIG.

  FIG. 2 is a diagram illustrating an example of an imaging range of the camera 10.

  In addition, in the figure, the cameras 10b and 10d are in a state where they cannot be visually recognized behind the own vehicle.

  As illustrated in FIG. 2, the cameras 10a to 10d have imaging ranges that can image road surface portions A1a to A1d adjacent to the front, rear, left, and right sides of the vehicle, respectively. Therefore, the cameras 10a to 10d are adjacent to a front fender on the passenger seat side viewed from a driver in a vehicle having a relatively high vehicle height, which cannot be visually recognized from the interior of the vehicle (hereinafter, referred to as a vehicle interior). Road surface, etc.).

  Further, a road surface portion A1 in which the road surface portions A1a to A1d are overlapped surrounds the host vehicle. That is, the camera 10 has an imaging range in which the road surface portion A1 surrounding the own vehicle by 360 ° can be imaged.

  However, the cameras 10a to 10d cannot capture an image of the road surface portion A2 inside (on the side of the host vehicle) the road surface portion A1. That is, the camera 10 may have a blind spot (area outside the imaging range) adjacent to the own vehicle in the vicinity of the own vehicle. For example, the front end of the host vehicle has an appearance shape connected to the left and right side portions by a curve having a certain curvature in a plan view, and thus tends to be farther away from the imaging range of the camera 10a as approaching the left and right ends. Therefore, the blind spot of the camera 10a is wider on the road surface portion adjacent to the left and right end portions than on the road surface portion adjacent to the front portion of the central portion of the front end of the vehicle, and obstacles existing on the road surface portion are detected by the camera 10a. It may not be possible to take an image.

  Returning to FIG. 1, the ECU 20 is an electronic control unit that executes main control processing in the display device 1. The ECU 20 is configured by, for example, a microcomputer or the like, and implements various control processes by executing various control programs stored in the ROM on the CPU. Specifically, a display image to be displayed on the display 30 (display area 30a) is generated based on each captured image received from the cameras 10a to 10d. The display image displayed on the display 30 may be an image representing an omnidirectional situation viewed from the host vehicle, or may be an image representing a specific azimuth angle range viewed from the host vehicle. That is, the ECU 20 generates a display image using a part or all of the images captured by the cameras 10a to 10d. The ECU 20 is communicably connected to the display 30 through a direct line, an in-vehicle LAN, or the like, and a generated display image (an image signal corresponding to the display image) is transmitted to the display 30.

  Note that when a state of a specific azimuth range can be displayed on the display 30 as a display image, an operation unit that can set an azimuth range to be displayed on the display 30 by a driver or the like may be provided.

  The display 30 is a display unit that displays a state around the own vehicle captured by the camera 10, and includes, for example, a liquid crystal panel or the like. The display 30 displays the display image in its own display area 30a (for example, a rectangular area having an aspect ratio of 16: 9) based on the image signal transmitted from the ECU 20.

  The display 30 may be installed at an arbitrary position in the vehicle cabin where the driver or the like is easily visible, and is provided, for example, above the center stack of the instrument panel.

  Next, a process of generating a display image by the ECU 20 will be described.

  First, the ECU 20 performs a known viewpoint conversion process on a captured image received from the camera 10 (10a to 10d) to generate an image (virtual viewpoint image) viewed from a predetermined virtual viewpoint in the vehicle interior. As the virtual viewpoint, for example, an eye point (eye position) corresponding to a statistically standard physique of a driver may be set.

  Note that the viewpoint conversion process in the present embodiment means a process of converting (pixels) of each coordinate of a captured image of the camera 10 (10a to 10d) into coordinates of a coordinate system viewed from a virtual viewpoint, and generates a base image. (Base image generation processing). However, the base image generation processing may be realized as a part of the viewpoint conversion processing in terms of mounting.

  Subsequently, the ECU 20 combines the generated virtual viewpoint image with an image (blind spot area image) representing the blind spot area of the camera 10 (10a to 10d) to generate a base image serving as a base of the display image (base). Image generation processing).

  Here, FIG. 3 is a diagram illustrating an example of the base image BP generated by the base image generation processing.

  In this example, a display image showing an azimuth angle range corresponding to the front left of the own vehicle (an azimuth angle range including a road surface portion next to the left front fender from the driver's seat of a right-hand drive vehicle to a blind spot) is displayed. 30 shows a base image BP to be displayed on 30.

  Referring to FIG. 3, the base image BP is obtained by combining the blind spot area images BSP1 and BSP2 with a virtual viewpoint image VP showing a parking lane marking of a parking lot where the vehicle is stopped, a pedestrian in the parking lot, and the like. Generated by The blind spot area image BSP1 corresponds to a blind spot area (an area outside the imaging range) of the cameras 10a and 10c on the side approaching the own vehicle. Further, in the base image BP, the blind spot area image BSP2 corresponds to a blind spot area (an area outside the imaging range) of the camera 10c on the side away from the host vehicle.

  As described above, when the blind spot area images BSP1 and BSP2 representing the blind spot area of the camera 10 (10a, 10c) are combined with the virtual viewpoint image VP, the side approaching the own vehicle around the own vehicle captured by the camera 10 It is possible to make the driver or the like recognize that the blind spot area exists on the far side.

  FIG. 4 is a diagram illustrating an example of an area that is out of an imaged range. As shown in the drawing, an area outside the range to be imaged, that is, a blind spot area often exists in the air. Specifically, in the example shown in the drawing, an area including an aerial portion among the areas outside the imaging range is the area DP or the like. When this area DP is displayed in a plane on the display image, it is difficult for a driver or the like to recognize that there is a blind spot in the air. Thus, the display image displays, for example, an image described below.

  FIG. 5 is a diagram illustrating a first example of a display image. FIG. 5A is an example of a display image generated by synthesizing an image showing an omnidirectional situation viewed from the own vehicle generated based on the captured image and an image showing the area DP. This example is an example in which a display image is generated by synthesizing an image showing the periphery of the host vehicle and an image showing the area DP three-dimensionally with a barricade-like mark or the like. When such a display screen is displayed, the driver can recognize that there is also a blind spot in the air.

  Further, the display image may be generated by synthesizing the virtual viewpoint image and the image indicating the area DP. Specifically, the display image may be an image as shown in FIG.

  FIG. 6 is a diagram illustrating a second example of the display image. The display image may be an image as shown in FIG. Specifically, a display image generated by combining an image indicating the periphery of the host vehicle with an image indicating the area DP with a barricade-like mark may be an image as illustrated in FIG. Compared to FIG. 5 (A), the display image shown in FIG. 6 (A) is different in that the central portion DPC of the region DP is omitted and displayed. As described above, the area DP may be partially omitted and displayed. In this manner, when only the corners of the area DP are displayed, the driver or the like can be less annoyed.

  As shown in FIG. 6B, in the display image generated by combining the virtual viewpoint image and the image indicating the area DP, the area DP partially includes It may be omitted and displayed.

  FIG. 7 is a diagram illustrating a third example of the display image. The display image may be an image as shown in FIG. Specifically, a display image generated by combining an image indicating the periphery of the vehicle and an image indicating the area DP with a barricade-like mark may be an image as illustrated in FIG. Compared to FIG. 6A, the display image shown in FIG. 7A is different in that the area DP is displayed after being subjected to the transmission processing. As described above, the area DP may be displayed after being subjected to the transparency processing. As described above, when the area DP is displayed after being subjected to the transmission processing, the driver or the like can be less annoyed.

  Also, as shown in FIG. 7B, in the display image generated by combining the virtual viewpoint image and the image indicating the area DP, the area DP is subjected to the transmission processing as in FIG. 7A. May be displayed.

  Each image is generated from, for example, a captured image and is stored in advance in an internal memory or the like of the ECU 20. For this reason, the ECU 20 can generate a display image by reading from the internal memory or the like an image determined according to detection information such as the azimuth angle range and the clearance sonar viewed from the own vehicle displayed as the virtual viewpoint image VP. it can.

  When the display image is displayed in a format as shown in FIGS. 5 to 7, the driver or the like can three-dimensionally recognize the blind spot area.

  As described above, the embodiments for carrying out the present invention have been described in detail. However, the present invention is not limited to the specific embodiments, and various modifications may be made within the scope of the present invention described in the appended claims. Can be modified and changed.

  For example, in the above-described embodiment, the display images shown in FIGS. 5 to 7 are generated by combining the virtual viewpoint image VP with an image representing the blind spot area (area DP) of the camera 10. Not done. That is, a display image may be generated by combining an image representing the blind spot area of the camera 10 with the image itself captured by the camera 10. Accordingly, the same operation and effect as those of the above-described embodiment (a driver or the like can easily recognize the blind spot of the camera) can be obtained.

1 display device 10, 10a to 10d camera (imaging unit)
20 ECU (image generation unit)
30 Display (display unit)
30a display area

Claims (2)

An imaging unit for imaging the periphery of the vehicle,
An image three-dimensionally showing an aerial region among regions outside the range imaged by the imaging unit, and an image showing the periphery of the own vehicle imaged by the imaging unit or an image imaged by the imaging unit An image generation unit that generates a display image to be displayed as one image by combining a virtual viewpoint image generated by performing viewpoint conversion on
A display unit for displaying the display image ,
The display unit,
A display device for a vehicle, which displays a corner portion of a region outside a range to be imaged by the imaging unit and omits and displays a portion other than the corner portion . The display unit,
The display device for a vehicle according to claim 1, wherein an area outside a range imaged by the imaging unit is subjected to transmission processing and displayed.