JP7076976B2 - Display device for vehicles - Google Patents

Description

The present invention relates to a vehicle display device, and more particularly to a vehicle display device that displays an image around the own vehicle.

Conventionally, a display device that displays a rear image when looking at the rear of the vehicle from one virtual viewpoint by synthesizing an image taken by the camera at the rear of the vehicle and an image taken by the cameras on the left and right of the vehicle is known. Has been done.

For example, Patent Document 1 describes an image pickup camera that captures the rear of the own vehicle from the central portion in the vehicle width direction of the own vehicle, an image pickup camera that is installed on the side mirror of the own vehicle and captures the rear of the own vehicle, and the like. An in-vehicle image display device having an output control unit that synthesizes image data captured by an imaging camera to generate a rear image, a display that displays the generated rear image, and a sensor that detects a vehicle behind the vehicle. Is disclosed. This in-vehicle image display device changes the boundary on which the image data is synthesized according to the detected position of the rear vehicle.

Japanese Unexamined Patent Publication No. 2010-287163

However, in the conventional display device as described above, since the images obtained from a plurality of cameras installed at different positions are combined to generate a rear image, a double image is generated near the boundary between the combined images. Image and distortion occur. Therefore, when the rear vehicle is displayed at a position straddling the boundary between the images, there is a problem that it is difficult to grasp the positional relationship between the rear vehicle and the own vehicle and the distance to the rear vehicle.

The present invention has been made to solve the above-mentioned problems of the prior art, and when a single image is generated and displayed by synthesizing images obtained from a plurality of cameras, a rear vehicle or a rear obstacle It is an object of the present invention to provide a vehicle display device capable of displaying an image around the own vehicle in which the driver can more intuitively recognize the positional relationship and distance with an object.

In order to achieve the above object, the vehicle display device of the present invention is a vehicle display device that displays an image of the surroundings of the vehicle, and includes a display means for displaying an image and a rear camera for photographing the rear of the vehicle. , A side camera that shoots the sides and rear of the vehicle, a rear image taken by the rear camera, and a rear side image taken by the side camera are combined to generate a single image, and a single image is generated. A display control means for generating a display image by superimposing a mask image on a position overlapping the boundary between the rear image and the rear side image synthesized in the above and displaying the display image on the display means is provided , and the display means is a vehicle. The display is provided at the center of the upper end of the front windshield in the vehicle width direction and is directed toward the rear of the vehicle. The position where the opaque part that imitates the rearmost pillar of the vehicle that can be seen in the mirror is included, and the part that imitates the rearmost pillar overlaps the boundary between the rear image and the rear side image synthesized in a single image. Is placed in .
In the present invention configured as described above, the rear image taken by the rear camera and the rear side image taken by the side camera are combined to generate a single image, and the single image is combined. Since the mask image is superimposed on the position that overlaps the boundary between the rear image and the rear side image to generate the display image, the rear vehicle and the rear obstacle are located at the position that straddles the boundary between the rear image and the rear side image. Even in this case, the double image and distortion of the rear vehicle and the rear obstacle in the vicinity of the boundary can be hidden by the mask image. This prevents the driver from erroneously grasping the positional relationship and distance between the vehicle behind and the vehicle behind due to double images and distortion, and the area around the vehicle that the driver can more intuitively recognize. Image can be displayed.
In addition, a display is provided at the position where the conventional mirror was provided, and a mask image is superimposed on the display by synthesizing the rear image taken by the rear camera and the rear side image taken by the side camera. Since the displayed image is displayed, the image around the own vehicle that the driver can more intuitively recognize a wide range from the rear of the vehicle to the side is displayed with the same feeling as checking the rear of the vehicle with a conventional mirror. be able to.
In addition, the mask image includes an opaque portion simulating the rearmost pillar of the vehicle, and the position where the portion simulating the rearmost pillar overlaps the boundary between the rear image and the rear side image synthesized in the single image. Even if the rear vehicle or rear obstacle is located at a position that straddles the boundary between the rear image and the rear side image, the double image or distortion of the rear vehicle or rear obstacle near the boundary can be displayed. It can be hidden by the part of the mask image that simulates the pillar at the end. This not only prevents the driver from erroneously grasping the positional relationship and distance between the rear vehicle or the rear obstacle and the own vehicle due to the double image or distortion, but also the rear vehicle or the rear obstacle and the own vehicle. The positional relationship of the vehicle can be expressed by the relationship with the mask image simulating the rearmost pillar, and the image of the area around the own vehicle that allows the driver to more intuitively recognize the positional relationship and distance with the vehicle behind and obstacles behind. Can be displayed.

Further, in the present invention, preferably, the mask image is opaque, simulating the ceiling of a vehicle that is reflected in the mirror from the driver's point of view when it is assumed that the mirror having the mirror is provided at the position of the display. And an opaque part that simulates the structure of the rear end of the vehicle.
In the present invention configured as described above, the mask image includes an opaque portion simulating the ceiling of the vehicle and an opaque portion simulating the structure of the rear end portion of the vehicle (for example, a rear gate or a trunk). When the driver checks the rear, such as the upper part, the unnecessary area can be hidden by the mask image without discomfort. As a result, the amount of information that the driver should recognize can be reduced, and an image around the own vehicle that the driver can more intuitively recognize the positional relationship and distance with the vehicle behind and the obstacle behind can be displayed. ..

Further, in the present invention, preferably, the side camera is attached to the door mirror of the vehicle, and the mask image is reflected on the mirror from the driver's point of view, assuming that the mirror is provided at the position of the display. Includes a translucent part that simulates the structure of the side of the vehicle that can be seen.
In the present invention configured in this way, the mask image includes a translucent portion simulating the structure of the side surface of the vehicle (for example, the rear quarter panel), so that while displaying other vehicles and obstacles on the side of the vehicle, they are displayed. The positional relationship between the other vehicle or obstacle and the own vehicle can be expressed by the relationship with the mask image simulating the structure of the side surface of the vehicle, and the driver can more express the positional relationship and distance with the rear vehicle or rear obstacle. It is possible to display an intuitively recognizable image of the surroundings of the own vehicle.

Further, in the present invention, preferably, the rear camera and the side camera have a fisheye lens, and the display control means develops the rear image taken by the rear camera on a virtual plane perpendicular to the front-rear direction of the vehicle and laterally. The rear side image taken by the camera is developed on a virtual cylindrical surface whose central axis extends in the vertical direction of the vehicle, and the rear image developed on a plane and the rear side image developed on the cylindrical surface are combined to form a single image. To generate.
In the present invention configured as described above, the rear image taken by the rear camera having a fish-eye lens is developed into a virtual plane perpendicular to the front-rear direction of the vehicle, and the rear side image taken by the side camera having the fish-eye lens is developed. Is developed on a virtual cylindrical surface whose central axis extends in the vertical direction of the vehicle, so that the straight line in the actual space is displayed as a straight line in the image for the rear image, and the sense of distance is felt over the entire angle of view for the rear side image. It can be made uniform. Then, by synthesizing these rear images and rear side images to generate a single image, it is possible to display a single image that has both a natural appearance and an appropriate sense of distance. It is possible to display an image around the own vehicle in which the driver can more intuitively recognize the positional relationship and distance with the obstacle behind.

According to the vehicle display device according to the present invention, when a single image is generated and displayed by synthesizing images obtained from a plurality of cameras, the driver can further determine the positional relationship and distance with the vehicle behind and the obstacle behind. It is possible to display an intuitively recognizable image of the surroundings of the own vehicle.

It is a top view which shows the arrangement of the plurality of cameras provided in the display device for a vehicle by embodiment of this invention. It is a front view which shows the periphery of the instrument panel of the vehicle to which the display device for a vehicle by embodiment of this invention is applied. It is a block diagram which shows the electric structure of the display device for a vehicle by embodiment of this invention. It is a flowchart of the display process executed by the display device for a vehicle according to the embodiment of this invention. It is a figure which exemplifies the image in the process of generating a single image by synthesizing the images obtained from a plurality of cameras. It is a figure which illustrates the single image which combined the image obtained from a plurality of cameras, and the display image which superposed the mask image on the single image.

Hereinafter, a vehicle display device according to an embodiment of the present invention will be described with reference to the accompanying drawings.
First, the configuration of the vehicle display device according to the embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a plan view showing the arrangement of a plurality of cameras included in the vehicle display device according to the embodiment of the present invention, and FIG. 2 is an instrument of a vehicle to which the vehicle display device according to the embodiment of the present invention is applied. It is a front view which shows the periphery of a panel, and FIG. 3 is a block diagram which shows the electric structure of the display device for a vehicle by embodiment of this invention.

In FIG. 1, reference numeral 1 indicates a vehicle display device according to an embodiment of the present invention, wherein the vehicle display device 1 is a right-hand camera 4 attached to the lower part of the right side mirror of the vehicle 2 and a left side mirror of the vehicle 2. It includes a left-side camera 6 attached to the lower part and a rear camera 8 attached to the rear end of the vehicle 2. These cameras 4, 6 and 8 use a fisheye lens, the right side camera 4 and the left side camera 6 are directed to the side of the vehicle 2, and the rear camera 8 is directed to the rear of the vehicle 2. As a result, the right side camera 4 and the left side camera 6 photograph the front, side, and rear of the vehicle 2, and the rear camera 8 photographs the rear of the vehicle 2. With these cameras 4, 6 and 8, it is possible to capture a wide range from the rear of the vehicle 2 to the side of the side mirror, and capture the range not included in the driver's field of view facing the front of the vehicle 2 without omission. Can be done.

Next, as shown in FIG. 2, a front windshield 10 and an instrument panel 12 extending in the vehicle width direction below the front windshield 10 are provided in the front portion of the vehicle interior of the vehicle 2.
The vehicle display device 1 is provided at the center of the upper end of the front windshield 10 in the vehicle width direction, an electronic rearview mirror 14 having a display 14a facing the rear of the vehicle 2, and the instrument panel 12 at the center of the instrument panel 12 in the vehicle width direction. It is provided with a center display 16 having a display 16a directed to the rear of the vehicle 2.

Further, as shown in FIG. 3, the vehicle display device 1 generates a single image from the images taken by the cameras 4, 6 and 8, and displays it on the electronic rear-view mirror 14 and the center display 16. The part 18 is provided.
The display control unit 18 includes a CPU, various programs interpreted and executed on the CPU (including basic control programs such as an OS, and application programs that are started on the OS and realize specific functions), programs, and various data. It consists of a computer equipped with an internal memory such as a ROM or RAM for storing the data.

Next, the display process executed by the vehicle display device 1 will be described with reference to FIGS. 4 to 6.
FIG. 4 is a flowchart of a display process executed by the vehicle display device 1 according to the embodiment of the present invention. FIG. 5 is a diagram illustrating an image in a process of synthesizing images obtained from a plurality of cameras 4, 6 and 8 to generate a single image, and FIG. 5A is a diagram illustrating each camera 4 and 6 using a fisheye lens. , 8 is a diagram showing an image taken by 8, (b) is a diagram showing an developed image of each image shown in (a), and (c) is a diagram showing a trimmed image of the image shown in (b). (D) is a figure which shows the single image which combined the image shown in (c). FIG. 6 is a diagram illustrating a single image obtained by synthesizing images obtained from a plurality of cameras 4, 6 and 8 and a display image in which a mask image is superimposed on the single image, and FIG. 6A is a diagram. 5 (d) is a diagram showing a single image, and (b) is a diagram showing an image displayed on an electronic room mirror 14 by superimposing a mask image on the single image.

The display process shown in FIG. 4 is activated when the ignition of the vehicle 2 is turned on, and is repeatedly executed at a predetermined cycle (for example, every 200 ms).

As shown in FIG. 4, when the display process is started, in step S1, the display control unit 18 displays the side and rear images (rear side image) of the vehicle 2 taken by the right side camera 4 and the left side camera 6. The SV) and the image behind the vehicle 2 (rear image RV) taken by the rear camera 8 are acquired. As described above, since each of the cameras 4, 6 and 8 uses a fisheye lens, the acquired image is distorted as shown in FIG. 5 (a). Further, as shown in FIG. 5A, the display control unit 18 acquires an image in which the scenery actually seen from the installation positions of the cameras 4, 6 and 8 is reversed left and right.

Next, in step S2, the display control unit 18 corrects each acquired image. Specifically, the display control unit 18 displays the rear side image SV taken by the right side camera 4 and the left side camera 6 as a virtual cylindrical surface C whose central axis extends in the vertical direction of the vehicle 2 (indicated by an arc in FIG. 1). The process of expanding the rear image RV taken by the rear camera 8 into a virtual plane P (indicated by a straight line in FIG. 1) perpendicular to the front-rear direction of the vehicle 2 (planar). Unfold). As a result, as shown in FIG. 5B, the distortion of each image is corrected.

Next, in step S3, the display control unit 18 trims the image corrected in step S2. Since the image pickup range of the rear camera 8 and the image pickup range of the right side camera 4 and the left side camera 6 partially overlap, it is necessary to remove the overlapped part. Further, each image includes a wide range from the upper side to the lower side of the vehicle 2, and also includes an unnecessary area when the driver confirms the rear of the vehicle 2. Therefore, the display control unit 18 trims each image according to a predetermined boundary using, for example, coordinate information. In the example of FIG. 5B, the display control unit 18 cuts out a region surrounded by a rectangular boundary (indicated by a alternate long and short dash line in FIG. 5B) from each image. The image trimmed in step S3 is shown in FIG. 5 (c).

Next, in step S4, the display control unit 18 synthesizes the image trimmed in step S3 into a single image. As a result, as shown in FIG. 5D, a single image IT showing a wide range from the rear to the side of the vehicle 2 can be obtained.

Next, in step S5, the display control unit 18 superimposes the mask image M created in advance and stored in the memory or the like on the single image IT synthesized in step S4.
As shown in FIG. 6B, the mask image M is a vehicle that appears to be reflected in the rearview mirror from the driver's point of view when it is assumed that the rearview mirror having the mirror is provided at the position of the electronic rearview mirror 14. It includes a portion M1 simulating the ceiling of 2, a portion M2 simulating a pillar, a portion M3 simulating a rear gate at the rear end of the vehicle 2, and a portion M4 simulating a rear quarter panel. Of these, the portion M1 simulating the ceiling of the vehicle 2, the portion M2 simulating the pillar at the rear end of the vehicle 2, and the portion M3 simulating the rear gate are opaque, and the other parts are translucent. Further, the portion M1 simulating the ceiling of the vehicle 2 and the portion M2 simulating the pillar at the rearmost portion of the vehicle 2 are arranged at positions overlapping the boundary B of each image synthesized in step S4.

Next, in step S6, the display control unit 18 causes the display image D on which the mask image M is superimposed in step S5 to be displayed on the display 14a of the electronic room mirror 14. As a result, the driver can intuitively check a wide range from the rear of the vehicle 2 to the side with the same feeling as checking the rear of the vehicle 2 with a conventional rear-view mirror.
Further, since the portion M1 simulating the ceiling of the vehicle 2 and the portion M2 simulating the pillar at the rear end of the vehicle 2 in the mask image M cover the boundary B between the images synthesized in step S4, each image. Even when the rear vehicle is located at a position straddling the boundary B between them, the double image and distortion in the vicinity of the boundary B are hidden by the mask image M. Therefore, it is possible to prevent the driver from erroneously grasping the positional relationship between the rear vehicle and the own vehicle 2 and the distance to the rear vehicle due to the occurrence of double image or distortion of the rear vehicle in the vicinity of the boundary B.
After step S6, the display control unit 18 ends the process.

Next, a further modification of the embodiment of the present invention will be described.
In the above-described embodiment, the display control unit 18 has described that the image is displayed on the display 14a of the electronic rear-view mirror 14, but the image may be displayed on the center display 16.

Further, in the above-described embodiment, the mask image M includes a portion M1 simulating the ceiling of the vehicle 2, a portion M2 simulating a pillar of the vehicle 2, a portion M3 simulating a rear gate, and a portion M4 simulating a rear quarter panel. However, the shape of the mask image M can be appropriately changed according to the structure of the vehicle 2. For example, in the case of the sedan type vehicle 2, a portion simulating a trunk at the rear end of the vehicle 2 may be provided instead of the portion M3 simulating the rear gate. Further, a portion simulating the front and rear door panels may be added. Further, the mask image M does not necessarily have to simulate the ceiling, pillars, rear gate, rear quarter panel, etc. of the vehicle 2.

Further, in the above-described embodiment, it has been described that the right side camera 4 is attached to the lower part of the right side mirror of the vehicle 2 and the left side camera 6 is attached to the lower part of the left side mirror. It may be attached to a portion, or may be attached to another place on the side surface of the vehicle 2.

Further, in the above-described embodiment, it has been described that each of the cameras 4, 6 and 8 uses a fisheye lens, but a wide-angle lens other than the fisheye lens may be used.

Next, the effect of the vehicle display device 1 according to the above-described embodiment of the present invention and the modification of the embodiment of the present invention will be described.

First, the display control unit 18 combines the rear image RV taken by the rear camera 8 with the rear side image SV taken by the right side camera 4 and the left side camera 6 to generate a single image IT, and a single image IT is generated. Since the mask image M is superimposed on the position overlapping the boundary B between the rear image RV and the rear side image SV synthesized in the image IT to generate the display image D, the boundary between the rear image RV and the rear side image SV is generated. Even when a rear vehicle or a rear obstacle is located at a position straddling B, the double image or distortion of the rear vehicle or the rear obstacle in the vicinity of the boundary B can be hidden by the mask image M. As a result, it is possible to prevent the driver from erroneously grasping the positional relationship and distance between the own vehicle 2 and the rear vehicle or the rear obstacle due to the double image or distortion, and the driver can more intuitively recognize the own vehicle. 2 Peripheral images can be displayed.

Further, an electronic room mirror 14 is provided at a position where a room mirror having a conventional mirror is provided, and the rear image RV taken by the rear camera 8 and the right side camera 4 and the left side camera 6 take pictures on the display 14a. Since the display image D on which the mask image M is superimposed is displayed by synthesizing the rear side image SV, the rear side of the vehicle 2 is viewed from the rear side with the same feeling as when checking the rear side of the vehicle 2 with a conventional rearview mirror. It is possible to display an image around the own vehicle 2 in which the driver can more intuitively recognize a wide range.

Further, the mask image M includes an opaque portion M2 simulating the rearmost pillar of the vehicle 2, and the portion M2 simulating the rearmost pillar is the rear image RV and the rear side image synthesized in the single image IT. Since it is arranged at a position overlapping the boundary B with the SV, even if a rear vehicle or a rear obstacle is located at a position straddling the boundary B between the rear image RV and the rear side image SV, the rear vehicle near the boundary B The double image and distortion of the rear obstacle and the rear obstacle can be hidden by the part of the mask image M simulating the rearmost pillar. This not only prevents the driver from erroneously grasping the positional relationship and distance between the rear vehicle or the rear obstacle and the own vehicle 2 due to the double image or distortion, but also the rear vehicle or the rear obstacle and the self. The positional relationship with the vehicle 2 can be expressed by the relationship with the mask image M simulating the rearmost pillar, and the driver can more intuitively recognize the positional relationship and the distance with the vehicle behind and the obstacle behind. An image around the vehicle 2 can be displayed.

Further, since the mask image M includes an opaque portion M1 simulating the ceiling of the vehicle 2 and an opaque portion M3 simulating the structure of the rear end portion of the vehicle 2 (rear gate in the embodiment), the upper part of the vehicle 2 and the like. When the driver checks the rear, the unnecessary area can be hidden by the mask image M without discomfort. As a result, the amount of information that the driver should recognize can be reduced, and an image around the own vehicle 2 that allows the driver to more intuitively recognize the positional relationship and distance with the vehicle behind and obstacles behind can be displayed. can.

Further, since the mask image M includes a semi-transparent portion M4 simulating the structure of the side surface of the vehicle 2 (rear quarter panel in the embodiment), other vehicles and obstacles on the side of the vehicle 2 are displayed while those other vehicles are displayed. The positional relationship between the vehicle and the own vehicle 2 can be expressed by the relationship with the mask image M simulating the structure of the side surface of the vehicle 2, and the driver can further determine the positional relationship and distance with the rear vehicle and the rear obstacle. It is possible to display an image around the own vehicle 2 that can be intuitively recognized.

Further, the rear image RV taken by the rear camera 8 having a fisheye lens is developed on a virtual plane P perpendicular to the front-rear direction of the vehicle 2, and the rear side image SV taken by the right side camera 4 and the left side camera 6 having the fisheye lens is developed. Is developed on the virtual cylindrical surface C whose central axis extends in the vertical direction of the vehicle 2, so that the straight line in the actual space is displayed as a straight line in the image for the rear image RV, and the entire angle of view for the rear side image SV. The sense of distance can be made uniform over the entire range. Then, by synthesizing these rear image RVs and rear side image SVs to generate a single image IT, it is possible to display a single image IT that has both a natural appearance and an appropriate sense of distance. , It is possible to display an image around the own vehicle 2 in which the driver can more intuitively recognize the positional relationship and the distance with the rear vehicle and the rear obstacle.

1 Vehicle display device 2 Vehicle 4 Right camera 6 Left camera 8 Rear camera 10 Front windshield 12 Instrument panel 14 Electronic room mirror 14a, 16a Display 16 Center display 18 Display control unit SV Rear side image RV Rear image C Virtual cylinder Surface P Virtual plane IT Single image D Display image M Mask image M1 Ceiling simulated part M2 Pillar simulated part M3 Rear gate simulated part M4 Rear quarter panel simulated part B Boundary

Claims (4)

A vehicle display device that displays images around the vehicle.
Display means for displaying images and
With a rear camera that shoots the rear of the vehicle,
A side camera that captures the sides and rear of the vehicle,
The rear image taken by the rear camera and the rear side image taken by the side camera are combined to generate a single image, and the rear image combined in the single image and the rear side image are combined. A display control means that superimposes a mask image on a position overlapping the boundary with the image to generate a display image and causes the display means to display the display image.
Equipped with
The display means is a display provided at the center of the upper end of the front windshield of the vehicle in the vehicle width direction and directed to the rear of the vehicle.
The mask image includes an opaque portion simulating the rearmost pillar of the vehicle that is visible in the mirror from the driver's point of view, assuming the mirror is provided at the position of the display. A portion simulating a pillar of a portion is arranged at a position overlapping the boundary between the rear image and the rear side image synthesized in the single image.
Display device for vehicles. The mask image is a structure of an opaque portion simulating the ceiling of the vehicle and the rear end portion of the vehicle as seen from the driver's point of view when the mirror is provided at the position of the display. The vehicle display device according to claim 1 , further comprising an opaque portion simulating. The side camera is attached to the door mirror of the vehicle and
The mask image comprises a translucent portion simulating the structure of the side surface of the vehicle as seen from the driver's point of view, assuming the mirror is provided at the position of the display. 2. The vehicle display device according to 2. The rear camera and the side camera have a fisheye lens.
The display control means develops the rear image taken by the rear camera on a virtual plane perpendicular to the front-rear direction of the vehicle, and displays the rear side image taken by the side camera on the vehicle whose central axis is up and down. Any of claims 1 to 3 , wherein the rear image developed on a virtual cylindrical surface extending in a direction is combined with the rear side image developed on the plane to generate the single image. The vehicle display device according to item 1.

Images