JP6744064B2 - Automotive image display system - Google Patents

Description

The present invention relates to an image display system for an automobile, which is used particularly for supporting driving of a vehicle such as an automobile.

BACKGROUND ART Conventionally, various vehicle image display systems have been developed to support safe driving of vehicles.

For example, there is known a vehicle information transmission device for transmitting a danger and navigation around a vehicle to a driver in an intuitive manner (see Patent Document 1, for example). This device detects dangerous objects such as pedestrians, bicycles, and other vehicles in the vicinity of the front by using a camera, radar sensor, etc. mounted on the vehicle, and displays them graphically on the display device of the instrument panel. Then, the driver is made aware of its existence.

However, in the device disclosed in Patent Document 1, the detected dangerous object is displayed on the display device of the instrument panel, and therefore the driver looks at it while looking forward through the front windshield while driving. Must be transferred to the display device, and the recognition is delayed in the situation where the danger must be judged instantaneously.

Similarly, there is known a vehicle periphery monitoring device that displays the existence and type of a detection target object that is highly likely to contact the vehicle on an image display device including a HUD (head-up display) to notify the driver. (For example, see Patent Document 2). At this time, the type of the detection target object is determined based on its shape and size, such as a pedestrian, a bicycle, an animal, etc., and a mark having a different shape depending on the type and the position of the detection target object The surrounding rectangular frame is displayed on the HUD.

International Publication No. 2013/088535 JP, 2010-108264, A

According to the conventional technique of Patent Document 2, it is not necessary for the driver to change the line of sight to display information on the front windshield, which is more advantageous than the case of Patent Document 1.

However, even if the information is displayed on the front windshield, the driver's line of sight is ahead of the front windshield, so the eyes should be focused to see the information displayed on the front windshield surface. It has to be changed, and sometimes it is impossible to recognize or overlook it.

The present invention has been made to solve the above problems, and an object of the present invention is to provide an image display system for an automobile capable of displaying information toward a front windshield in an easy-to-understand manner for a driver. It is in.

The present invention is
1) A vehicle image display system for displaying information provided to passengers of a vehicle as an image,
A stereo camera that outputs a right-eye image and a left-eye image by photographing a surrounding situation in a blind spot formed by the automobile structure,
A steering angle sensor that detects the rotating operation of the steering wheel,
Display means for projecting the right-eye image and the left-eye image forming a stereoscopic image for stereoscopically displaying the image captured by the stereo camera toward the front windshield of the automobile,
Using the pattern data file the contour which is the image feature of the captured image of the detection object by the stereo camera is registered in advance, specifying the sense target contour information included in the three-dimensional image pattern matching process to Means and
Means for calculating the distance to the detection target from the parallax of the left and right images of the stereo camera, and determining whether the distance is within a predetermined range,
With
When the steering angle sensor detects a rotation operation of the steering wheel in the blind spot direction, the detection target is specified and the distance to the detection target is within a predetermined range, the right eye image and the It is possible to provide an image display system for a vehicle, which displays a left-eye image as a stereoscopic image at a focal position in the back direction in front of the front windshield of the vehicle.

Also,
2) A stop control unit that brings the vehicle into a stopped state when the distance is within a predetermined range,
The vehicle image display system according to claim 1, further comprising:

According to the present invention, since the information provided to the occupant is projected by the right-eye image and the left-eye image, the occupant can visually recognize the provided information in the stereoscopic image. Therefore, the occupant can visually recognize the provided information while the eyes are focused on the front side through the front windshield, and it is not necessary to change the focus of the eyes to check the provided information, and the oversight can be prevented.

1 is a block diagram showing the overall configuration of an automobile image display system according to the present invention. 1 is a plan view showing an example of a vehicle in which a camera and a sensor are arranged in an image display system for a vehicle according to the present invention. FIG. 1 is a conceptual view of a front windshield and a dashboard of a vehicle including an image display system for a vehicle according to the present invention as seen from a driver's seat side. The schematic explanatory drawing of the image display by a HUD apparatus is shown. The schematic diagram explaining an example of the display in the front windshield of the image display system for motor vehicles which concerns on this invention is shown. The schematic diagram explaining another example of the display in the front windshield of the image display system for motor vehicles which concerns on this invention is shown. FIG. 1 is a block diagram showing the overall configuration of a driving support system in which an image display system for a vehicle according to the present invention is combined. The flowchart explaining an example of operation|movement of the image display for vehicles is shown. The schematic diagram explaining an example of the display in the front windshield in a driving assistance system is shown. The schematic diagram explaining another example of the display in the front windshield in a driving assistance system is shown. The schematic diagram explaining another example of the display on the front windshield in a driving assistance system is shown. The schematic diagram explaining the lane change of the virtual vehicle displayed on the front windshield in a driving assistance system is shown in order of time progress.

FIG. 1 is a block diagram showing a schematic configuration of an automobile image display system according to the present invention. As shown in FIG. 1, an automobile image display system 1 includes a detection unit 2, a display control unit 3, and a head-up display (HUD) device 4 which is a display unit.

The detection unit 2 includes a camera unit 2A and a sensor unit 2B that monitor the surroundings of the automobile. The image captured by the camera unit 2A and the detection information by the sensor unit 2B serve as the information provided to the occupants of the automobile. Therefore, the detection unit 2 is a detection unit that detects the provided information.

The camera unit 2A is provided with three types of cameras 17, 18, and 19 as shown in FIG. 2, for example, in order to capture an image of the surroundings of an automobile.

The camera 17 is installed on the upper part of the front windshield 15 and captures a forward view of the automobile 10. The camera 18 is arranged on the back windshield to capture the rear view of the automobile 10.

In addition, when visually observing the surroundings from the inside of the automobile 10, there is a structure that forms a blind spot where the line of sight from the inside of the automobile is blocked and the outside is not visible due to the structure of the automobile. For example, it is a structure such as a pillar that supports the roof of the vehicle body. In the example of FIG. 2, in order to secure the view of the blind spot by the A pillar 16 that holds the front windshield 15 on the roof of the vehicle body, A camera 19 is arranged outside.

The sensor unit 2B includes sensors 20 and 21 for detecting the presence of an automobile, a bicycle, a person, an animal, or the like (hereinafter, referred to as “detection target”) around the automobile. The sensors 20 and 21 are, for example, radar sensors, and as shown in FIG. 3, the sensor 20 is attached to the lower part of the left and right door mirrors 35 and detects the presence of a detection target object in the rear left and right. The sensor 21 is attached to the center of the vehicle body at the front of the automobile 10 and detects the presence of a detection target existing in front. As the radar sensor, a millimeter wave radar, a microwave radar, a laser radar, an infrared sensor, an ultrasonic sensor or the like is used.

Further, the steering angle sensor 24, the vehicle speed sensor 25, the gear position sensor 26, etc. provided in the traveling system of the automobile 10 also constitute the detection unit 2 here.

The display control unit 3 displays the information acquired by the detection unit 2 on the HUD device 4 as a stereoscopic image. At this time, the images to be displayed include the images taken by the cameras 17, 18, and 19 and the detection information from the sensors 20, 21, the steering angle sensor 24, and the vehicle speed sensor 25 in the form of pictures such as figures and icons, or letters and numbers. There are two types: a drawing image that is expressed as a target. Then, the display control unit 3 includes an image memory M in which these various drawn images are stored in advance.

In order to stereoscopically display a captured image, a stereo control camera in which lenses for both eyes corresponding to human left and right eyes are housed in one housing is used for each of the cameras 17, 18, and 19. 3 outputs the right-eye image and the left-eye image input from the stereo camera to the HUD device 4. The cameras 17, 18 and 19 are not limited to stereo cameras, and a monocular camera that can capture a stereo image by computer processing can also be used.

In order to display the drawn image, the display control unit 3 has an appropriate pattern or character for visually displaying the detection information from the sensors 20 and 21, the steering angle sensor 24, the vehicle speed sensor 25, and the gear position sensor 26. Select a drawing image of a number from those stored in the image memory M in advance. Then, a right-eye image and a left-eye image are created from the selected drawn image and output to the HUD device 4. Therefore, in this case, the display control unit 3 functions as a drawing unit that converts the provided information into a visually displayable drawing image.

The HUD device 4 here has a conventionally known structure and mechanism. For example, as shown in FIG. 4, a right-eye image display unit 6, a left-eye image display unit 7, and a mirror 8 are used. It is configured to include and. The HUD device 4 is arranged on the upper surface of the dashboard 39 and projects an image toward the front windshield 15. Further, the HUD device 4 may be incorporated in the dashboard 39, or an image may be projected from the ceiling portion of the driver's seat to the front windshield 15.

The image display unit 6 for the right eye and the image display unit 7 for the left eye of the HUD device 4 are displays for displaying the image RP for the right eye and the image LP for the left eye, for example, an ELD (electroluminescence display), a fluorescent display tube. Display devices such as an FED (field emission display), and an LCD (liquid crystal display) are used.

The mirror 8 of the HUD device 4 is arranged so as to face the right-eye image display unit 6 and the left-eye image display unit 7, and the right-eye image display unit 6 and the left-eye image display unit 7 display the images respectively. The right-eye image RP and the left-eye image LP that are present are reflected toward the front windshield 15. Further, the mirror 8 projects the right-eye image RP and the left-eye image LP displayed on the right-eye image display unit 6 or the left-eye image display unit 7 by enlarging them to a magnification predetermined by the mirror shape.

The front windshield 15 is formed of a laminated glass, an IR cut glass, a UV cut glass, or the like into a curved surface shape curved in the vertical direction and the horizontal direction according to the outer shape of the automobile 10. Then, the front windshield 15 transmits the line of sight of the driver of the occupant and reflects the right-eye image RP and the left-eye image LP projected from the HUD device 4 in the line-of-sight direction.

In the image display system 1 for an automobile configured in this way, the HUD device 4 projects the right-eye image RP and the left-eye image LP output from the display control unit 3, whereby the right-eye image RP and the left-eye image RP. The LP is reflected by the front windshield 15 on both eyes of the driver.

As a result, the driver can visually recognize the stereoscopic image TD on the line of sight in the depth direction in front of the vehicle with the front windshield 15 sandwiched by the binocular parallax. Therefore, the driver can recognize the provided information detected by the detection unit 2 in a stereoscopic image such as a captured image or a drawn image while viewing the situation in front of the automobile 10 through the front windshield 15.

As long as the human eye is focused several meters in front, it will not be significantly out of focus even for objects at infinity. Therefore, when the driver focuses his eyes in the depth direction in front of the front windshield 15 to confirm the field of view in front of the automobile 10, when the image is projected on the front windshield 15, it is overlooked. More likely. However, by configuring the driver so that the provided information can be viewed stereoscopically, the provided information can be visually recognized in the state of being focused in the depth direction in front of the front windshield 15, and the provided information can be easily and reliably obtained. No more recognizing and overlooking.

In the vehicle image display system 1 having the above-described configuration, the display control unit 3, for example, when the steering angle sensor 24 detects that the driver operates the steering wheel 22 to turn the vehicle 10 to the right, the camera 18 at that time is detected. The image taken by can be stereoscopically displayed on the HUD device 4. Therefore, as shown in FIG. 4, the driver can see the stereoscopic image TD while the eyes are focused in the front depth direction through the front windshield 15, so that the driver can cross the right side of the automobile 10. Brake operation can be performed by recognizing the presence of a pedestrian who is present. Moreover, since the blind spot scene is displayed as a stereoscopic image, there is no fear that it will be confused with the actual scene in front of the vehicle seen through the front windshield 14, and the driver can accurately grasp the situation. Become.

At this time, since the depth of the stereoscopic image visually recognized by the driver in the depth direction from the front windshield 15 is determined by the binocular parallax, the display control unit 3 reproduces the stereoscopic image at this depth as necessary. The right-eye image RP and the left-eye image LP captured by the camera 18 can be processed.

Similarly, when the vehicle 10 is moved backward, when the gear position sensor 24 detects that the gear position sensor 24 has entered the reverse gear, the display control unit 3 obtains the right-eye image RP and the left-eye image LP captured by the camera 18 from the HUD device 4. To project by. Therefore, the driver can visually recognize the presence of a pedestrian or the like in the rear while viewing the situation in front of the automobile 10 through the front windshield 15.

Further, when the current vehicle speed value is acquired from the vehicle speed sensor 25, the display control unit 3 reads out an image of numbers and characters for visually displaying the current vehicle speed value from the image memory M, and uses this image as the right-eye image RP. The image LP for the left eye is created, and the drawn image of the speed display 34 (“60 km/h” in the illustrated example) shown in FIG. 5 is displayed. At this time, the display control unit 3 recognizes the right-eye image RP of the drawn image so that the stereoscopic image in which the binocular parallax according to the depth in the depth direction from the front windshield 15 of the drawn image indicating the vehicle speed is reproduced can be visually recognized. An image LP for the left eye is created. In addition to the display of the speed, a drawn image or the like showing the state of the gear detected by the gear position sensor 26 is also displayed as a stereoscopic image based on the binocular parallax.

Then, when the display control unit 3 detects by the steering angle sensor 24 that the driver turns the steering wheel 22 to the right or left in order to change the lane, the presence of an object to be detected in the rear of the direction is detected from the sensor 20. Detect with information. At this time, the sensor 20 determines that the detection target is approaching when the detected detection target is larger than a predetermined size, and outputs a detection signal of the detection target to the display control unit 3.

As a result, as shown in FIG. 6, the display control unit 3 reads the drawn image of the prohibition mark 13 indicating that there is a danger of changing lanes from the drawing memory M, and displays it stereoscopically by the HUD device 4. In the illustrated example, the prohibition mark 13 is displayed on the left side because there is another vehicle approaching from the left rear when trying to change the lane to the left lane.

Therefore, since the prohibition mark 13 is displayed three-dimensionally, the driver can recognize the prohibition mark 13 while focusing on the front windshield 15 with the eyes focused in the front depth direction. Can detect danger. It is also possible to dispose a camera in place of the sensor 20 that detects the detection object on the rear left and right, and to display another vehicle behind the camera captured by the HUD device 4 as a stereoscopic image.

The vehicle image display system 1 according to the present invention described above is effectively applied to the vehicle 10 including the driving support system for supporting safe driving. The embodiment will be described below.

First, regarding the driving support system, in Japan, the automation level of the driving support system is defined in four stages from level 1 to level 4. Level 1 is called a safe driving assistance system in which the vehicle performs acceleration, steering or braking. Level 2 is called a semi-automatic driving system, where the car simultaneously performs a plurality of operations including acceleration, steering, and braking. Level 3 is a state in which the car performs all acceleration, steering, and braking, and the driver responds only in an emergency. This is also called a semi-automatic driving system. Level 4 is called a fully automated driving system in which all drivers, except the driver, perform acceleration, steering, and braking without any involvement. In the present specification, “automatic operation” refers to traveling at the level 3 or 4.

FIG. 7 schematically shows the entire configuration of a driving support system 20 that is a combination of the vehicle image display system 1 of this embodiment.

The traveling support system 20 includes a central processing unit 21 including a CPU, a ROM, and a RAM. The central processing unit 21 includes the detection unit 2 described in FIG. 1 and an HMI (Human).
The interface unit 23, the traveling control device 25, and the navigation device 33 are connected to each other.

The HMI unit 23 is configured to include the touch panel 30 and a monitor 31 (shown in FIG. 3) together with the HUD device 4 described in FIG. The monitor 31 displays, for example, the backward view taken by the camera 18 when the vehicle 10 moves backward. Further, the touch panel 30 is formed by software on the screen of the monitor 31.

The travel control device 25 includes a brake control device 26 and an accelerator control device 27. The brake control device 26 controls a brake operating device (not shown) to automatically decelerate the automobile 10. The accelerator control device 27 controls a throttle operating device (not shown) that adjusts the throttle opening to automatically accelerate the automobile 10.

The navigation device 33 reads out map data around the automobile 10 from the map database based on GPS information received from the GPS (Global Positioning System) 32.

The central processing unit 21 is realized by the CPU executing a control program stored in the ROM. The pattern matching processing unit 21a, the stop control unit 21b, the follow-up traveling control unit 21c, and the constant speed traveling control are realized. The unit 21d, the target preceding vehicle determination unit 21e, the automatic driving control unit 21f, and the display control unit 3 described in FIG.

The pattern matching processing unit 21a performs pattern matching processing on the images respectively sent from the cameras 17, 18, and 19 to detect the contours of a pedestrian, a bicycle, a motorcycle, etc. from the images, and Determine the presence of. In order to perform this processing, the ROM includes a pattern data file F in which contours, which are image characteristics of these detection targets, are registered in advance. Further, the ROM includes the above-mentioned image memory M in which a drawing image visually represented by a picture such as a figure or an icon displayed by the display control unit 3 on the HUD device 4 or a character/number is stored in advance. There is.

Then, the pattern matching processing unit 21a also detects the lane line based on the white line or the yellow line based on the brightness information of the image at the same time as the pattern matching process.

When the pattern matching processing unit 21a detects a detection target object such as a pedestrian, a bicycle, or a motorcycle from the image, the stop control unit 21b determines the distance from the parallax between the left and right images of the camera 17 to the detection target object. It is a stop control unit that calculates and controls the stop of the automobile 10 according to the distance.

In the follow-up running mode, the follow-up running control unit 21c calculates the inter-vehicle distance to the preceding vehicle in order for the vehicle 10 to follow the preceding vehicle, and compares the result data with a preset inter-vehicle distance to provide the running control device. Output to 25.

In the constant-speed traveling mode, the constant-speed traveling control unit 21d calculates the difference between the current speed and a preset vehicle speed and outputs it to the traveling control device 25 in order to drive the vehicle at a constant speed.

The target preceding vehicle determination unit 21e determines the preceding vehicle that is the target of the following traveling mode.

The automatic driving control unit 21f performs control for automatically driving the automobile 10 at the level 3 or 4 described above.

In the driving support system 20 described above, a specific operation when the vehicle 10 turns right and automatically stops when the presence of a person is detected will be described based on the flowchart of FIG. 8. This automatic stop can be applied not only to all the driving support from level 1 to level 4 described above, but also to the safety support during manual driving by the driver.

The central processing unit 21 determines whether or not the driver has performed a steering operation for turning the automobile 10 to the right (step S1). At this time, the central processing unit 21 detects whether the steering wheel 22 is rotationally operated by the steering angle sensor 24. When the steering operation for right turn is not performed and the result is “NO”, the operation of displaying the image of the scene that is a blind spot is not performed.

When the right turn steering operation is performed and “YES”, the pattern matching processing unit 21a of the central processing unit 21 acquires the image sent from the camera 19 at this time (step S2), and then, The HUD device 4 displays the right-eye image RP and the left-eye image LP captured by the camera 19. Therefore, the image is stereoscopically displayed by the binocular parallax, so that the driver can visually recognize the image at the position in the depth direction across the front windshield 15 on the driver's line of sight in front of the vehicle (step S3).

Then, the central processing unit 21 performs pattern matching processing between the acquired image and the contour of the detection target registered in the pattern data file F, and the detection target such as a pedestrian, a bicycle, or a motorcycle is included in the image. By detecting whether or not the image feature of the object is included, the existence thereof is determined (step S4). In the case of "NO" in which the detection target object is not included, the operation of displaying the image of the scene that is a blind spot is terminated.

When the image includes a detection target object (“YES” in step S4), the central processing unit 21 determines the distance from the parallax between the left and right images of the camera 19 to the detection target object by the stop control unit 21b. It is calculated to determine whether or not the distance is within a predetermined range (step S5). Then, when the distance to the detection target exceeds the range of, for example, 10 meters (“NO” in step S5), the operation of displaying the image of the scene that is a blind spot is ended.

On the other hand, when the distance to the detection target is within the range of 10 meters (“YES” in step S5), the central processing unit 21 controls the brake control device 26 to stop the automobile 10 (step S6). .. After that, the central processing unit 21 repeats the process from step 2 to determine whether the detection target object or another detection target object does not exist (“NO” in step 4), or even if they exist, When the distance exceeds 10 meters (“NO” in step S5), the operation of displaying the image of the blind scene is terminated. In this case, the stop control unit 21b releases the stopped state of the vehicle 10 by the brake control device 26.

As a result, the vehicle 10 is maintained in the stopped state while the camera 19 is detecting the detection target at a distance within the predetermined range. Therefore, according to the safe driving support system 20, since the automatic stop is controlled according to the situation in the blind spot range where the driver's eyes cannot reach, it is possible to reliably support the safe driving of the driver when making a right turn.

The driving support system 20 in which the image display system 1 for an automobile according to the present invention is integrated includes a pillar portion typified by the A-pillar 6 and other automobile structures such as frames and outer members from the inside of the vehicle. In some cases, a camera may be attached to these surfaces and a stereo image thereof may be displayed on the front windshield 15 by the HUD device 4.

The driving support system 20 displays various information other than the blind spot image by the A-pillar 6 as a stereoscopic image on the driver as a photographed image or a drawn image, but some display examples will be described below.

[Display when running at constant speed or controlling the distance between vehicles]
The constant-speed traveling/inter-vehicle distance control of the automobile 10 by the traveling support system 20 will be described. The constant-speed traveling/inter-vehicle distance control in the present embodiment is performed as follows, for example, but is not limited to this, as a matter of course.

The central processing unit 21 starts constant-speed traveling/inter-vehicle distance control of the own vehicle when the driver turns on a traveling support switch (not shown). The vehicle speed for constant speed traveling and the inter-vehicle distance to the preceding vehicle may be set by inputting desired values to the touch panel 30 of the HMI unit 23 immediately before the driver turns on the traveling support switch. The previously set value stored in the memory 20 (not shown) can be used as it is. The constant speed traveling/inter-vehicle distance control is performed by switching between the following traveling modes when the preceding vehicle is detected and the constant traveling mode when the preceding vehicle is not detected.

First, the target preceding vehicle determination unit 21e detects all the preceding other vehicles using the camera 17 and the sensor 21 on the front side. Then, the target preceding vehicle determination unit 21e detects and stores the detected inter-vehicle distances, relative speeds, directions with respect to the own vehicle, etc., with respect to all the preceding vehicles, and runs in the same lane as the own vehicle among them. Among the preceding vehicles that are present, the one that is closest to the own vehicle is determined as the target preceding vehicle for following the vehicle. Whether or not the preceding vehicle is traveling in the same lane as the own vehicle can be determined by detecting the lane by the pattern matching processing unit 21a.

It is preferable that the sensor 21 always scans regardless of whether the driving support switch is on or off. This makes it possible not only to quickly determine the target preceding vehicle with respect to the ON operation of the travel support switch, but also to detect the preceding vehicle for the rear-end collision prevention function.

The presence of the preceding vehicle can also be obtained by performing near field communication with the preceding vehicle. By adding the information obtained by so-called vehicle-to-vehicle communication, which is performed by the vehicle-to-vehicle information communication with the preceding vehicle, to the detection result by the sensor 21 and the front camera 17, the preceding vehicle detection unit 11 causes the preceding vehicle detection. The accuracy of position information can be further improved.

The presence of the preceding vehicle is carried out by various information communication between the vehicle and equipment around the road, that is, by so-called wireless communication directly with a communication device such as a sensor or an antenna installed on the road or through a server in the surrounding area, It is also available through road-to-vehicle communication. For such vehicle-to-vehicle communication and/or road-to-vehicle communication, the driving support system 20 includes a communication device for communicating with the outside. Therefore, the communication device that acquires the position information and the traveling condition of the surrounding vehicles through the vehicle-to-vehicle communication and the road-to-vehicle communication serves as a detection unit that detects the information provided to the driver.

In the following traveling mode, the central processing unit 21 controls the traveling control device 25 so that the following traveling control unit 21c maintains the determined inter-vehicle distance with the target preceding vehicle at the set inter-vehicle distance. That is, if the current actual inter-vehicle distance to the target preceding vehicle is longer than the set inter-vehicle distance, the accelerator control device 27 is controlled so as to increase the vehicle speed of the automobile 10 to reduce the inter-vehicle distance to the target preceding vehicle. , The brake control device 26 is controlled so as to reduce the vehicle speed of the own vehicle to increase the inter-vehicle distance to the target preceding vehicle, and if the same, the brake control device 26 and the accelerator control are performed so as to maintain the current vehicle speed of the own vehicle. And the device 27.

FIG. 9 shows an example of the screen display of the HUD device 4 in the following traveling mode. In this example, the preceding vehicle 41 that is following is displayed by drawing the marking 42. In front of the vehicle on the road of the real landscape 40 seen through the front windshield 15, one preceding vehicle 41 is in the same overtaking lane 43 as the own vehicle, and another preceding vehicle 41 is in the adjacent traveling lane 44. The vehicle 45 is running. In this scene, both the preceding vehicles 42 and 45 are detected by the sensor 21 and the camera 17, but the target preceding vehicle determination unit 21e targets the preceding vehicle 41 that is closest to the own vehicle on the same lane 43 as the own vehicle. It is decided to be the preceding vehicle. Therefore, in this case, the preceding vehicle 41 is a detection target that the driver particularly gazes in a real scene.

At this time, the target preceding vehicle determination unit 21e stores the detected data such as the inter-vehicle distance, the relative speed, and the direction with respect to the target preceding vehicle 42 in the memory, and the sensor 21 and the camera 17 continuously scan continuously. By doing so, the target preceding vehicle 42 is automatically tracked, and data such as the inter-vehicle distance, relative speed, and direction with respect to the target preceding vehicle 42 are continuously collected and stored.

Then, the display control unit 3 displays the image of the marking 42 indicating that the preceding vehicle 41 is the following target vehicle by the target preceding vehicle determination. As shown in FIG. 9, the marking 42 is a picture of a frame that substantially overlaps with the rear contour of the preceding vehicle 41, and the display control unit 3 reads the drawn image of the frame from the image memory M and uses this as a stereoscopic image. The creation processing of the image RP for the right eye and the image LP for the left eye is performed so as to be displayed at.

The processing at this time will be described. The area where the camera 17 fixedly installed on the upper part of the front windshield 15 and the surface of the front windshield 15 overlap is fixed, and the display controller 3 presets this overlapping area. .. Therefore, when the target preceding vehicle is determined, the display control unit 3 uses the XY axis passing through the center point O of the front windshield 15 as a reference, and selects the contour of the preceding vehicle 41 from the image captured by the camera. The coordinate position of the center point P of is calculated. At this time, the size of the preceding vehicle 41 can be determined from the ratio of its contour to the image, and thus the drawn image of the marking 42 having a size corresponding to the size is read from the image memory M.

Then, the display control unit 3 creates a right-eye image RP and a left-eye image LP having binocular parallax from the drawn image of the marking 42 and displays them on the HUD device 4. In this case, the display control unit 3 allows the driver to visually recognize the marking 42 at the above-described coordinate position whose center is calculated and at the depth in the depth direction from the front windshield 15 to the preceding vehicle 41. An image RP for the right eye and an image LP for the left eye having

Therefore, the driver can visually recognize the marking 42 at the same time while the eyes of the preceding vehicle 41 are focused through the front windshield 15. As a result, the driver sees the target preceding vehicle 42 as if the target preceding vehicle 42 is actually surrounded by a frame, so that the driver can surely visually recognize the target preceding vehicle that is the target of tracking by the own vehicle.

As shown in the figure, the image of the marking 42 in the present embodiment is formed by a thick square frame line surrounding the outline of the target preceding vehicle 41, but various shapes and displays other than the thick square frame line are used. be able to. For example, it may be indicated by an arrow instead of being surrounded by a thick circular frame line or a frame, and may be displayed in a prominent color such as red or orange to further draw the driver's attention. ..

Then, at the time of constant speed traveling under the control of the constant speed traveling control unit 21d, the display control unit 3 stereoscopically displays the traveling speed set by the input to the touch panel 30 of the HMI unit 23 by a drawing image of a number. Further, when changing the constant speed traveling speed, the display control unit 3 acquires the vehicle speed in the middle stage until reaching the set speed from the vehicle speed sensor 25 and sequentially displays the acquired vehicle speed.

Show navigation
When the navigation device 33 sets the route guidance to the destination of the automobile 10, the HUD device 4 stereoscopically displays the navigation information.

The navigation information displayed by the HUD device 4 is, for example, an arrow along the traveling direction of the automobile 10, and a straight arrow is displayed when going straight at an intersection, and a bent arrow corresponding to the direction when turning left or right is displayed. To do.

FIG. 10 shows an example of a screen display by the HUD device 4 at the time of guidance for turning left on the route during navigation traveling. The display control unit 3 reads out the drawn image of the arrow 46 indicating the left turn from the image memory M and stereoscopically displays it on the HUD device 4. At this time, when the information on the distance from the navigation device 33 to the intersection 47 that makes a left turn is transmitted, the display control unit 3 sets the coordinate position on the Y-axis set on the surface of the front windshield 15 based on the distance. Is determined, and a right-eye image RP and a left-eye image LP having binocular parallax such that the arrow 46 is stereoscopically displayed at that position are created and displayed.

Therefore, the driver can also see the arrow 46 while the eyes are focused on the intersection 47 through the front windshield 15. As a result, the driver can prepare for the left turn because the left turn arrow is visually recognized in an overlapping manner at the intersection 47. Therefore, in this case, the intersection 47 is a detection object that the driver particularly gazes in the real landscape.

[Display during automatic operation]
In the vehicle image display system according to the present invention, in a fully autonomous vehicle in which the driver is not directly involved in driving, the driver can monitor the autonomous driving situation by the stereoscopic image projected by the HUD device 4. An example of display by the vehicle image display system according to the present invention during automatic driving will be described.

In the automatic driving, the navigation device 33 calculates one or a plurality of driving route candidates to the destination input in advance by the occupant, and the automatic driving control unit follows the driving route determined by the driver's confirmation or approval. 21f is performed by controlling the traveling control device 25.

During this automatic driving, the display control unit 3 creates the right-eye image RP and the left-eye image LP of the drawn image by the arrow 48 representing the traveling course of the vehicle, as shown in FIG. indicate. In this case, the display control unit 3 detects the coordinate position of the traveling path 49 of the own vehicle by processing the image captured by the camera 17 by the pattern matching processing unit 21a, and the driver shows the arrow 48 with the binocular parallax at the coordinate position. The right-eye image RP and the left-eye image LP are created so that they can be visually recognized as a stereoscopic image.

Therefore, the driver can see the arrow 48 through the front windshield 15 while focusing his/her eyes on the depth direction, so that the driver can see on the road 49 of the real scenery 36 as if he/she sees through the front windshield 15. It can be recognized as the arrow 48 is shown.

Further, the display control unit 3 allows the driver and other occupants to intuitively understand the next driving action to be taken before the automatically driving vehicle 10 deviates from the current driving course or changes the lane. A notice will be given by displaying clearly and instantaneously. In the present embodiment, the determination of whether or not to take a traveling behavior different from the planned traveling behavior is performed by the central processing unit 21 in order to realize the automatic driving control unit 21f. Determine according to the control program.

FIG. 12 schematically shows a screen for notice displayed on the HUD device 4 prior to changing the lane from the overtaking lane 52 to the driving lane 53 while the vehicle is in automatic driving. .. At this time, in addition to the drawn image by the arrow 50 indicating the planned traveling course, the drawn image by the arrow 51 indicating the traveling course for changing lanes is displayed.

The arrow 51 has a shape that curves from a position on the overtaking lane 38 of the real landscape 36 to a position on the adjacent traveling lane 45 of the destination along a trajectory predicted to automatically travel by the vehicle. .. These arrows 50 and 51 are also read from the image memory M and displayed as a stereoscopic image, so that the driver can visually recognize the arrows while the eyes of the vehicle ahead are focused. Therefore, the driver can more clearly, intuitively and concretely recognize that the driving operation for changing the lane is about to be performed from the overtaking lane 52 that is currently traveling to the traveling lane 53.

Further, in the example of FIG. 12, the drawn images by the icons 54 for virtualizing the own vehicle are displayed together with the drawn images by the arrows 50, 51. Then, the display control unit 3 virtually displays the progress of the lane change that the vehicle is going to perform by the movement of the icon 54 by the HUD device 4. 12A to 12D, the displays at this time are schematically shown in time order.

First, in FIG. 12A, the icon 54 indicates a state as if the own vehicle is traveling just in front of the front windshield 15. At this time, the display control unit 3 obtains the right-eye image RP and the left-eye image so that the position where the icon 54 is visually recognized by the driver is a visual difference as if the position is behind the preceding vehicle 55 in the same lane 52. An LP is created and displayed on the HUD device 4.

Then, the display control unit 3 changes the lane of the icon 54 from the overtaking lane 52 to the driving lane 53 and separates from the overtaking lane 52, as sequentially shown in the virtual images of FIGS. An image RP for the right eye and an image LP for the left eye are created and displayed on the HUD device 4 so that the driver can visually recognize each of the moving states as a stereoscopic image. At this time, the appearance shape of the host vehicle also changes as the lane is changed, but the display control unit 3 draws from the drawing memory M a drawing image of the icon 54 representing the appearance shape of the own vehicle according to the elapsed time. Read and display.

It is preferable that the simulation display of the lane change by the icon 54 of the virtual vehicle is displayed at a speed higher than the actual traveling speed of the vehicle. As a result, after notifying the lane change, before the timing of the lane change scheduled by the automatic driving control unit 21f, the driver is made to decide whether to stop the lane change or switch to the manual driving, and prepare if necessary. It is possible to have a time allowance.

Further, the icon 54 can be disappeared immediately after the virtual lane change is completed, or can be maintained for a certain short time while being displayed at the same position on the front windshield 15. Further, the moving images in (a) to (d) of FIG. 12 can be repeatedly displayed.

As described above in detail, the vehicle image display system according to the present invention is capable of performing automatic driving at any stage from level 1 to level 4 when a driving operation is directly performed by a driver or a driving support system. Also in this case, the information necessary for safe driving can be provided to the driver by displaying.

1 Automotive image display system 2 Detection unit (detection means)
3 Display control unit (drawing means)
4 HUD device (display means)
17, 18, 19 Camera 20, 21 Sensor

Claims (2)

A vehicle image display system for displaying information provided to passengers of a vehicle in an image,
A stereo camera that outputs a right-eye image and a left-eye image by photographing a surrounding situation in a blind spot formed by the structure of the automobile,
A steering angle sensor that detects the rotating operation of the steering wheel,
Display means for projecting the right-eye image and the left-eye image forming a stereoscopic image for stereoscopically displaying the image captured by the stereo camera, toward the front windshield of the automobile,
Using the pattern data file the contour which is the image feature of the captured image of the detection object by the stereo camera is registered in advance, specifying the sense target contour information included in the three-dimensional image pattern matching process to Means and
Said parallax between the images of the left and right stereo cameras calculates the distance to the detection object, and means for determining whether the distance is within a predetermined range,
With
When the steering angle sensor detects a rotation operation of the steering wheel in the blind spot direction, the detection target is specified and the distance to the detection target is within a predetermined range, the right-eye image and the image An image display system for a vehicle, which displays a left-eye image as a stereoscopic image at a focus position in the back direction in front of the front windshield of the vehicle. The vehicle image display system according to claim 1, further comprising a stop control unit that sets the vehicle to a stopped state when the distance is within a predetermined range.

Images