JP2020121704A - Display control device, head-up display device, method and computer program - Google Patents

Abstract

To suppress deterioration in augmented reality feeling when vibration occurs.SOLUTION: When estimating that own vehicle vibrates in a magnitude larger than a certain magnitude, a display control device reduces a sense of perspective of a perspective image 210 having a sense of perspective that is perceived such as positioned along a travel lane of own vehicle.SELECTED DRAWING: Figure 3B

Description

The present disclosure relates to a display control device, a head-up display device, a method, and a computer program that are used in a vehicle and that superimpose an image on the foreground of the vehicle for visual recognition.

Patent Document 1 is a head-up display device having a flat or curved display area on which a virtual image of an image is displayed, and the display area is visually recognized so as to be along a horizontal plane (for example, a road surface on which the vehicle travels). It is described that the upper side of the display area viewed from the viewer is located farther from the viewer than the lower side. In such a head-up display device, by arranging the display area along the horizontal plane, the virtual reality (AR) is displayed as if the image displayed in the display area actually exists along the road surface of the traveling lane. : Augmented Reality) can be perceived.

JP, 2016-45252, A

However, in the head-up display device of Patent Document 1, the angle between the host vehicle and the road surface is not constant due to the vibration of the host vehicle due to small ups and downs in the traveling lane and the change in the pitching of the host vehicle due to acceleration/deceleration. Since the angle between the display area and the road surface is different from usual, the sense of augmented reality (augmented reality) felt due to the image becoming familiar with the road surface (real object on the real landscape) is lost (or Is expected to decrease). Therefore, the viewer may feel uncomfortable, and there is room for improvement in the display that suppresses the deterioration of the augmented reality even when vibration occurs.

The following is a summary of the specific embodiments disclosed herein. It should be understood that these aspects are presented only to provide an overview of the reader to these particular embodiments and are not intended to limit the scope of this disclosure. Indeed, the present disclosure may include various aspects not described below.

SUMMARY OF THE DISCLOSURE The present disclosure relates to providing a display with reduced discomfort to a viewer. More specifically, it also relates to suppressing a decrease in augmented reality when vibration occurs.

Therefore, the display control device described in the present specification weakens the perspective of the displayed perspective image 210 when it is estimated that the vehicle is vibrating at a certain size or more. In some embodiments, the perspective image 210 may be viewed by being superimposed on the road surface of the driving lane 310 in the foreground. Further, when the perspective image 210 displayed is weakened, the perspective image 210 may be changed to a non-perspective image 230 having no perspective.

It is a figure which shows the example of application to the vehicle of the display system for vehicles which concerns on some embodiment. FIG. 3 is a block diagram of a vehicular display system according to some embodiments. FIG. 6 is a diagram illustrating a display example of a perspective image according to some embodiments. FIG. 6 is a diagram illustrating a display example of a non-perspective image according to some embodiments. FIG. 9 is a diagram for explaining a perspective image with a weak perspective according to some embodiments. FIG. 6 is a flow diagram illustrating a perspective image display process according to some embodiments. It is a flowchart of the display process which changes the display mode of an image according to the vibration of the own vehicle which concerns on some embodiment.

In the following, FIGS. 1 and 2 provide a description of the configuration of an exemplary vehicle display system. 3A, 3B, and 4 provide a display example of an image displayed for perceiving a virtual object. Further, in FIGS. 5 and 6, a flow of display processing is provided. The present invention is not limited to the following embodiments (including the contents of the drawings). Of course, changes (including deletion of components) can be added to the following embodiments. In addition, in the following description, in order to facilitate understanding of the present invention, description of known technical matters will be appropriately omitted.

Please refer to FIG. The image display unit 11 in the vehicle display system 10 is a head-up display (HUD) device provided in the dashboard 5 of the vehicle 1. The HUD device emits the display light 11 a toward the front windshield 2 (which is an example of a member to be projected) and displays the image 200 in the display area 100, so that the HUD device is visually recognized through the front windshield 2. The image 200 is superimposed on the foreground 300, which is the physical space, and the driver 4 visually recognizes it. The display area 100 itself has low visibility so that it is not visible to the driver 4 or is hardly visible to the driver 4. In the description of the present embodiment, the horizontal direction when the driver 4 seated in the driver's seat of the host vehicle 1 faces the front of the host vehicle 1 is the X-axis (the left direction is the positive direction of the X-axis), and the vertical direction is the vertical direction. The Y-axis (the upward direction is the positive Y-axis direction) and the front-back direction is the Z-axis (the forward direction is the positive Z-axis direction).

Further, the image display unit 11 may be a head mounted display (hereinafter, HMD) device. The driver 4 mounts the HMD device on his/her head and sits on the seat of the host vehicle 1 to visually recognize the displayed image 200 by superimposing it on the foreground 300 through the front windshield 2 of the host vehicle 1. To do. The display area 100 in which the vehicle display system 10 displays a predetermined image 200 is fixed (adjustably arranged) at a specific position with respect to the coordinate system of the host vehicle 1, and when the driver 4 faces that direction. The image 200 displayed in the display area 100 fixed at the specific position can be visually recognized.

The display area 100 of the image 200 is a portion above the display area 100 viewed from the driver 4 (a portion in the Y-axis positive direction) so as to be along a horizontal plane (for example, the road surface of the traveling lane 310 in which the vehicle 1 travels). Is a plane or a curved surface arranged at a position farther from the driver 4 than the lower side (the portion in the Y-axis negative direction). The acute angle θ formed by the display area 100 of the image 200 and the road surface of the driving lane 310 when viewed from the left-right direction (X-axis direction) of the driver 4 is 0 [degree] (the display area 100 and the road surface are parallel). However, it is preferable that it is set to less than 20 [degree]. In addition, the distance between the display area 100 and the road surface of the traveling lane 310 (distance in the Y-axis direction) is preferably set in the range of −1 to +1 [meter].

Based on the control of the display control device 13, the image display unit 11 includes obstacles (pedestrians, bicycles) existing in the foreground 300, which is a real space (real scene) visually recognized through the front windshield 2 of the vehicle 1. , Motorcycles, other vehicles, etc.), road surfaces of the driving lane 310, road signs, and the vicinity of real objects such as features (buildings, bridges, etc.) (an example of a specific positional relationship between an image and a real object), real objects By displaying the image 200 at a position (an example of a specific positional relationship between the image and the real object) that overlaps with, or a position (an example of a specific positional relationship between the image and the real object) set with reference to the real object. A visual augmented reality (AR) can be perceived by a viewer (typically, the driver 4 sitting in the driver's seat of the vehicle 1). The image display unit 11 can display an image 200 including an AR image whose display position is changed according to the position of the real object, and/or a non-AR image whose display position is not changed according to the position of the real object. ..

FIG. 2 is a block diagram of a vehicular display system 10 according to some embodiments. The vehicular display system 10 includes an image display unit 11 and a display control device 13 that controls the image display unit 11. The display controller 13 comprises one or more I/O interfaces 14, one or more processors 16, one or more memories 18, and one or more image processing circuits 20. The various functional blocks depicted in FIG. 2 may be implemented in hardware, software, or a combination of both. FIG. 2 is only one embodiment of an implementation, and the illustrated components may be combined into fewer components or there may be additional components. For example, image processing circuitry 20 (eg, a graphics processing unit) may be included in one or more processors 16.

As shown, the processor 16 and the image processing circuit 20 are operably coupled to the memory 18. More specifically, the processor 16 and the image processing circuit 20 operate the vehicle display system 10, for example, by generating or/and transmitting image data by executing a program stored in the memory 18. be able to. The processor 16 or/and the image processing circuit 20 may be at least one general purpose microprocessor (eg, central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA). , Or any combination thereof. The memory 18 includes any type of magnetic medium such as a hard disk, any type of optical medium such as CD and DVD, any type of semiconductor memory such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVROM.

As shown, the processor 16 is operably coupled to the I/O interface 14. For example, the I/O interface 14 uses the vehicle display system 10 as a personal area network (PAN) such as a Bluetooth (registered trademark) network or a local area network (LAN) such as an 802.11x Wi-Fi (registered trademark) network. A wireless communication interface for connecting to a wide area network (WAN) such as a 4G or LTE cellular network can be included. The I/O interface 14 may also include a wired communication interface such as, for example, a USB port, a serial port, a parallel port, an OBDII, and/or any other suitable wired communication port.

As shown in the figure, the processor 16 is operably connected to the I/O interface 14 to provide information on various other electronic devices connected to the vehicle display system 10 (I/O interface 14). Can be given and received. The I/O interface 14 includes, for example, a vehicle ECU 401 provided in the host vehicle 1, a road information database 403, a host vehicle position detector 405, a vehicle exterior sensor 407, a line-of-sight direction detector 409, an eye position detector 411, and portable information. The terminal 413 and the vehicle exterior communication connection device 420 are operably connected. The image display unit 11 is operably connected to the processor 16 and the image processing circuit 20. Therefore, the image displayed by the image display unit 11 may be based on the image data received from the processor 16 and/or the image processing circuit 20. The processor 16 and the image processing circuit 20 control the image displayed by the image display unit 11 based on the information obtained from the I/O interface 14. The I/O interface 14 may include a function of processing (converting, calculating, analyzing) information received from another electronic device or the like connected to the vehicle display system 10.

The host vehicle 1 is in the state of the host vehicle 1 (for example, mileage, vehicle speed, accelerator pedal opening, engine throttle opening, injector fuel injection amount, engine speed, motor speed, steering angle, shift position, drive mode). It also includes a vehicle ECU 401 that detects various warning states, postures (including roll angle and/or pitching angle), vehicle vibrations (including magnitude, frequency, and/or frequency of vibration). The vehicle ECU 401 collects and manages (may also include control) the above-described state of the host vehicle 1, and as a part of the function, the degree of the state of the host vehicle 1 (for example, the current state of the host vehicle 1). Vibration magnitude, pitching angle) can be transmitted to the processor 16 of the display controller 13. The vehicle ECU 401 may send the determination result based on the numerical value detected by the sensor and/or the analysis result to the processor 16 in addition to the numerical value detected by the sensor or the like. it can. For example, a determination result indicating whether the host vehicle 1 is vibrating in a memory (not shown) of the vehicle ECU 401 that satisfies a predetermined vibration condition or a pitching angle may be transmitted to the processor 16. ..

Further, the vehicle ECU 401 may transmit an instruction signal for instructing the image 200 displayed by the vehicle display system 10 to the I/O interface 14, in which case the coordinates, size, type, display mode, image of the image 200 are displayed. The necessity degree related information of 200 or/and the necessity degree related information which is a basis for determining the necessity degree of notification may be added to the instruction signal and transmitted.

The host vehicle 1 may include a road information database 403 including a navigation system and the like. The road information database 403 is based on the position of the own vehicle 1 acquired from the own vehicle position detection unit 405 described later, and is road information (lane, white line, stop line, Crosswalk, width of road, number of lanes, intersections, curves, forks, traffic restrictions, etc.), presence/absence of feature information (buildings, bridges, rivers, etc.), position (including distance to own vehicle 1), direction , Shape, type, detailed information, etc. may be read and transmitted to the processor 16. Further, the road information database 403 may calculate an appropriate route from the starting point to the destination and send it to the processor 16 as navigation information.

The host vehicle 1 may include a host vehicle position detection unit 405 including a GNSS (Global Navigation Satellite System) or the like. The road information database 403, the portable information terminal 413, which will be described later, and/or the extra-vehicle communication connection device 420 acquires the position information of the own vehicle 1 from the own vehicle position detection unit 405 continuously, intermittently, or at every predetermined event. Thus, the information around the vehicle 1 can be selected and/or generated and transmitted to the processor 16.

The host vehicle 1 may include one or more vehicle exterior sensors 407 that detect real objects existing around the host vehicle 1 (front, side, and rear). The real object detected by the vehicle exterior sensor 407 is, for example, a pedestrian, a bicycle, a motorcycle, another vehicle (such as a preceding vehicle), a road surface of the traveling lane 310, a marking line, a roadside object, and/or a feature (such as a building). May be included. As the vehicle exterior sensor, for example, there are a millimeter wave radar, an ultrasonic radar, a radar sensor such as a laser radar, and a camera sensor including a camera and an image processing device. It may be configured with only one of them. A conventionally known method is applied to the object detection by the radar sensor and the camera sensor. By detecting the objects by these sensors, the presence/absence of a real object in the three-dimensional space, and the position of the real object (relative distance from the own vehicle 1 and the traveling direction of the own vehicle 1 when the real object exists) are detected. Position in the left-right direction in the front-back direction, vertical position, etc.), size (size in the horizontal direction (left-right direction), height direction (vertical direction), etc.), movement direction (lateral direction (left-right direction), The depth direction (front-rear direction), the moving speed (lateral direction (left-right direction), depth direction (front-back direction)), and/or the type may be detected. One or more out-of-vehicle sensors 407 detect a real object in front of the own vehicle 1 for each detection period of each sensor, and detect real object-related information (presence or absence of real object, which is an example of real object-related information). When a real object exists, information such as the position, size, moving direction, moving speed, and/or type of each real object) can be transmitted to the processor 16. Note that the real object-related information is transmitted from the other electronic device to the processor 16 after the information from the vehicle exterior sensor 407 is analyzed via the other electronic device (for example, the vehicle ECU 401). Good. Further, when a camera is used as a sensor, an infrared camera or a near infrared camera is desirable so that a real object can be detected even when the surroundings are dark such as at night. Further, when a camera is used as a sensor, a stereo camera that can acquire a distance and the like by parallax is desirable.

The host vehicle 1 may include a line-of-sight direction detection unit 409 that detects the gaze direction of the driver 4 (hereinafter, also referred to as “line-of-sight direction”) and that includes an infrared camera that captures an image of the face of the driver 4. The processor 16 can specify the line-of-sight direction of the driver 4 by acquiring an image captured by the infrared camera (an example of information that can estimate the line-of-sight direction) and analyzing the captured image. Note that the processor 16 may acquire the line-of-sight direction of the driver 4 specified by the line-of-sight direction detection unit 409 (or another analysis unit) from the image captured by the infrared camera from the I/O interface 14. The method of acquiring the driver's 4 line-of-sight direction of the own vehicle 1 or the information capable of estimating the driver's 4 line-of-sight direction is not limited to these, and the EOG (Electro-oculogram) method, corneal reflex Method, sclera reflection method, Purkinje image detection method, search coil method, infrared fundus camera method, and other known gaze direction detection (estimation) techniques may be used.

The host vehicle 1 may include an eye position detection unit 411 including an infrared camera that detects the position of the eyes of the driver 4. The processor 16 can specify the position of the eyes of the driver 4 by acquiring an image (an example of information that can estimate the position of the eyes) captured by the infrared camera and analyzing the captured image. The processor 16 may acquire the information on the position of the eyes of the driver 4 identified from the image captured by the infrared camera from the I/O interface 14. The method for acquiring the position of the eyes of the driver 4 of the vehicle 1 or the information capable of estimating the position of the eyes of the driver 4 is not limited to these, and known eye position detection (estimation) It may be acquired using a technology. The processor 16 adjusts at least the position of the image 200 based on the position of the eyes of the driver 4 so that the image 200 in which the image 200 superimposed on a desired position of the foreground 300 is detected is the viewer (driver 4). May be visually confirmed.

The mobile information terminal 413 is a smartphone, a laptop computer, a smart watch, or another information device that can be carried by the driver 4 (or another occupant of the vehicle 1). The I/O interface 14 can communicate with the mobile information terminal 413, and acquires the data recorded in the mobile information terminal 413 (or the server via the mobile information terminal). The mobile information terminal 413 has, for example, the same function as the road information database 403 and the vehicle position detection unit 405 described above, acquires the road information (an example of the real object related information), and transmits it to the processor 16. Good. The mobile information terminal 413 may also acquire commercial information (an example of real object-related information) related to a commercial facility in the vicinity of the own vehicle 1 and send it to the processor 16. The mobile information terminal 413 transmits schedule information of the owner (for example, the driver 4) of the mobile information terminal 413, incoming information at the mobile information terminal 413, mail reception information, etc. to the processor 16, and the processor 16 and The image processing circuit 20 may generate or/and transmit image data regarding these.

The extra-vehicle communication connection device 420 is a communication device that exchanges information with the host vehicle 1, and for example, another vehicle connected to the host vehicle 1 through vehicle-to-vehicle communication (V2V: Vehicle To Vehicle), inter-vehicle communication (V2P: A pedestrian (a portable information terminal carried by a pedestrian) connected by a Vehicle To Pedestrian, a network communication device connected by a road-to-vehicle communication (V2I: Vehicle To Road Infrastructure), and in a broad sense, the own vehicle 1 and It includes all things connected by V2X (Vehicle To Everything). The extra-vehicle communication connection device 420 acquires the position of, for example, a pedestrian, a bicycle, a motorcycle, another vehicle (such as a preceding vehicle), a road surface of the traveling lane 310, a lane marking, a roadside object, and/or a feature (such as a building). However, it may be transmitted to the processor 16. In addition, the vehicle exterior communication connection device 420 has the same function as the own vehicle position detection unit 405 described above, and may acquire the position information of the own vehicle 1 and send it to the processor 16, and further, the road information database described above. It also has the function of 403, and may acquire the road information (an example of real object-related information) and send it to the processor 16. The information acquired from the vehicle exterior communication connection device 420 is not limited to the above.

The software components stored in the memory 18 are the real object related information detection module 502, the vibration determination module 504, the eye position detection module 506, the notification necessity determination module 508, the image position setting module 510, the image size setting module 512, and the perspective. A feeling setting module 514 and a graphic module 516 are included.

The real object related information detection module 502 acquires information (also called real object related information) including at least the position (area) of the real object (for example, the traveling lane 310) of the host vehicle 1 via the I/O interface 14. .. The real object-related information detection module 502 detects the position of the real object existing in the foreground 300 of the host vehicle 1 (from the driver 4 in the driver seat of the host vehicle 1 to the traveling direction (forward) of the host vehicle 1) from the vehicle exterior sensor 407. The position in the height direction (vertical direction) and the horizontal direction (horizontal direction) when viewed, and the position in the depth direction (front direction) may be added to these), and the size of the real object (height Direction, lateral size).

Further, the real object related information detection module 502 detects, from the vehicle exterior sensor 407, the position of the left lane marking 311 (see FIG. 3A) and the right lane marking 312 (see FIG. 3A) of the traveling lane 310 of the vehicle 1. And the area between the lane markings 311 and 312 (the position of the traveling lane 310) may be calculated.

In addition, the real object related information detection module 502 is based on the type, the direction indicator of the real object (other vehicle), the steering angle, and/or the driving support system from the real object (other vehicle) via the vehicle exterior communication connection device 420. Information indicating the planned progress route and the progress schedule may be acquired.

The vibration determination module 504 determines the vibration of the vehicle 1. The vibration determination module 504 acquires, for example, the magnitude, frequency, frequency, and/or the pitching angle of the own vehicle 1 of the vibration of the own vehicle 1 from the vehicle ECU 401, and the own vehicle 1 is stored in the memory 18 in advance. It is determined whether or not it is possible to infer that vibration is being performed, and the determination result is transmitted to the processor 16.

The eye position detection module 506 detects the position of the eyes of the driver 4 of the vehicle 1. The eye position detection module 506 determines where in the vertical region (or/and the horizontal region) the eye position of the driver 4 is provided in a plurality of stages, the eye vertical direction of the driver 4 ( Various operations related to the detection of the position in the Y-axis direction or/and the position in the left-right direction (the X-axis direction) and the position of the eyes of the driver 4 (the position in the X-, Y-, and Z-axis directions) are performed. It includes various software components for executing. The eye position detection module 506, for example, acquires the position of the eyes of the driver 4 from the eye position detection unit 411, or receives information that can estimate the position of the eyes of the driver 4 from the eye position detection unit 411, Estimate the position of the eyes of the driver 4. The information capable of estimating the position of the eyes includes, for example, the position of the driver's seat of the host vehicle 1, the position of the driver's 4 face, the height of the sitting height, the input value of the driver 4 on an operation unit (not shown), and the like. Good.

The notification necessity degree determination module 508 determines whether or not each image 200 displayed by the vehicle display system 10 is the content to be notified to the driver 4. The notification necessity degree determination module 508 may acquire information from various other electronic devices connected to the I/O interface 14 and calculate the notification necessity degree. Further, the electronic device (including the display control device 13) connected to the I/O interface 14 in FIG. 2 transmits information to the vehicle ECU 401, and the notification necessity degree determined by the vehicle ECU 401 based on the received information is required. The degree determination module 508 may detect (acquire). The "necessity of notification" is, for example, the degree of danger derived from the degree of seriousness of the possibility itself, the degree of urgency derived from the length of the reaction time required for taking a reaction action, the own vehicle 1 or the driver 4 ( Alternatively, it may be determined based on the effectiveness derived from the situation of other occupants of the own vehicle 1 or a combination thereof (the indicator of the notification necessity degree is not limited to these). That is, the notification necessity degree determination module 508 may determine whether to notify the driver 4 and may select not to display a part or all of the image 200. The vehicle display system 10 may not have the function of estimating (calculating) the notification necessity degree, and some or all of the functions of estimating the notification necessity degree may be displayed by the vehicle display system 10. It may be provided separately from the control device 13.

The image position setting module 510 determines the position of the image 200 within the display area 100 and/or the position of the display area 100. The image position setting module 510 is based on the position of the real object detected by the real object related information detection module 502 so that the image 200 is visually recognized in a specific positional relationship with the real object if the image 200 is the AR image. , The coordinates of the image 200 (including at least the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) when the driver 4 views the direction of the display area 100 from the driver's seat of the vehicle 1) are determined. .. For example, the image position setting module 510 determines the position of the image 200 in the left-right direction and the vertical direction so that the center of the image 200 is visually recognized so as to overlap with the center of the real object. The “specific positional relationship” can be adjusted depending on the situation of the real object or the host vehicle 1, the type of the real object, the type of the displayed image, and the like. In addition, the image position setting module 510 may predict the position of the real object for each display update cycle, and determine the coordinates of the image 200 based on the predicted position. In this case, the image position setting module 510 uses the observation positions of one or more real objects acquired in the past including at least the position (observation position) of the real object acquired immediately before by the real object related information detection module 502. The position of the image 200 (the AR image) may be set based on the predicted position of the real object in the display update cycle of the image 200 (the AR image) predicted. There is no particular restriction on the method of calculating the predicted position of the real object by the image position setting module 510, as long as the prediction is performed based on the observation position acquired in the past from the display update cycle targeted by the image position setting module 510. Any method may be used. The image position setting module 510 uses, for example, a least squares method, a Kalman filter, an α-β filter, or a prediction algorithm such as a particle filter, and uses the past one or more observation positions to calculate the next value. You may make it predict.

Further, if the image 200 is the non-AR image, the image position setting module 510 may set the position of each image 200 to a predetermined position within the display area 100 that is predetermined for each type.

The image size setting module 512 determines the size of the image 200. If the image 200 is the AR image, the image size setting module 512 may change the size of the image 200 according to the position and/or the size of the real object to be associated. For example, the image size setting module 512 can reduce the size of the AR image if the position of the real object to be associated is distant. Further, the image size setting module 512 can increase the size of the AR image if the size of the real object to be associated is large. The image size setting module 512 may have a function of predicting and calculating the display size of the image (AR image) 200 to be displayed in the current display update cycle, based on the size of the real object a predetermined number of times in the past. .. As a first method, the image size setting module 512 uses, for example, the Lucas-Kanade method to track pixels of a real object between two past captured images captured by a camera (an example of the sensor 407 outside the vehicle). Then, the size of the real object in the current display update cycle may be predicted, and the size of the AR image may be determined according to the predicted size of the real object. As a second method, the rate of change of the size of the real object is obtained based on the change of the size of the real object between the past two captured images, and the size of the AR image is determined according to the rate of change of the size of the real object. You may. Note that the method of estimating the size change of the real object from the viewpoint that changes in time series is not limited to the above, and includes, for example, known optical flow estimation algorithms such as the Horn-Schunkk method, the Buxton-Buxton, and the Black-Jepson method. You may use the method of.

Further, if the image 200 is the non-AR image, the image size setting module 512 may set the size of each image 200 to a size predetermined for each type.

Further, the image size setting module 512 is detected by the type, number, and/or notification necessity determination module 508 of the real object that displays the image 200 detected by the real object related information detection module 502 in association with each other ( The size of the image 200 may be determined based on the estimated size of the notification need.

The perspective setting module 514 changes the perspective of the image 200 based on the determination result of whether or not the vehicle 1 is vibrating under a predetermined condition. Specifically, the perspective setting module 514 sets the image 200 to the perspective image 210 with perspective when the own vehicle 1 is determined not to vibrate under a predetermined condition, and the own vehicle 1 sets the predetermined image. When it is determined that the image is vibrating under the condition of, the image 200 is set as the non-perspective image 230 without perspective.

FIG. 3A is a diagram showing a display example of the perspective image 210. The perspective image 210 of FIG. 3A has an arrow shape indicating the guide route of the host vehicle 1, and is arranged at a position overlapping the traveling lane 310 of the foreground 300 of the host vehicle 1 when viewed from the driver 4 of the host vehicle 1 and travels. It is visually recognized as a virtual object along the lane 310. Then, the perspective image 210 is visually recognized such that the size of the portion which is viewed in the vicinity of the own vehicle 1 and overlaps with the travel lane 310 is larger than the size of the part which is overlapped in the far travel lane 310 of the own vehicle 1. Specifically, the width in the left-right direction (X-axis direction) of the portion that is visually recognized as being overlapped with the traveling lane 310 near the host vehicle 1 is the width in the left-right direction of the portion that is overlapped with the traveling lane 310 that is far from the host vehicle 1. Seen as larger. In such a case, the viewer (driver 4) can perceive the perspective image 210 as if he or she is looking down on the virtual object along the traveling lane 310.

On the other hand, FIG. 3B is a diagram showing a display example of the non-perspective image 230. The non-perspective image 230 of FIG. 3B is, like the perspective image 210 of FIG. 3A, in the shape of an arrow indicating the guide route of the host vehicle 1. When viewed from the driver 4 of the host vehicle 1, the foreground 300 of the host vehicle 1 travels. The size of the portion that is arranged so as to overlap the lane 310 and that is visually recognized as being overlapped with the traveling lane 310 in the vicinity of the own vehicle 1 is visually recognized as being the same as the size of the portion that overlaps with the far traveling lane 310 of the own vehicle 1. It Specifically, the width in the left-right direction (X-axis direction) of the portion that is visually recognized as being overlapped with the traveling lane 310 near the host vehicle 1 and the width in the left-right direction of the portion that is overlapped with the traveling lane 310 that is far from the host vehicle 1. And are seen in the same way. In such a case, the viewer (driver 4) can perceive the non-perspective image 230 as if it were a virtual object along a plane (XY plane) formed by the vertical and horizontal directions. That is, when the non-perspective image 230 is defined as an image of a character, a symbol, a figure, an illustration, a still image (photograph), a moving image, or the like viewed from the front, the perspective image 210 is defined as the image component. Can also be defined as an image viewed from an angle rather than the front.

In addition, the perspective setting module 514 sets the image 200 to the perspective image 210 with perspective when the own vehicle 1 is determined not to vibrate under the predetermined condition, and the own vehicle 1 meets the predetermined condition. When it is determined that the image is vibrating, the image 200 may be set to the perspective image 220 having a weak perspective. FIG. 4 is a diagram for explaining a display example of the perspective image 220 having a weak perspective. FIG. 4A is a perspective image 210 displayed when it is determined that the host vehicle 1 is not vibrating under a predetermined condition. In this perspective image 210, a portion 211 arranged on the upper side (Y axis positive direction) viewed from the driver 4 who may overlap on the far side of the driving lane 310 is viewed from the driver 4 who may overlap on the near side of the driving lane 310. It is drawn considerably smaller than the portion 212 arranged on the lower side (Y-axis negative direction). FIG. 4C is a non-perspective image 230 corresponding to the perspective image 210 of FIG. This non-perspective image 230 is from a portion 231 arranged on the upper side (Y-axis positive direction) viewed from the driver 4 that may overlap on the far side of the driving lane 310, and a driver 4 that may overlap on the side near the driving lane 310. A portion 232 arranged on the lower side (Y-axis negative direction) viewed is drawn in the same size. That is, the display mode between the perspective image 210 displayed when it is determined that the host vehicle 1 is not vibrating under the predetermined condition and the non-perspective image 230 corresponding to this perspective image 210 is as shown in FIG. It can be defined as a perspective image 220 having a weak perspective as shown in (b). Specifically, the perspective image 220 with a weak perspective shown in FIG. 4B includes a portion 221 arranged on the upper side (Y-axis positive direction) viewed from the driver 4, which may overlap with the far side of the driving lane 310. The difference in size from the portion 222 arranged on the lower side (Y-axis negative direction) viewed from the driver 4 that may overlap the side near the driving lane 310 is denoted by reference numerals 221 and 222 in the perspective image 210 of FIG. 4A. It is drawn smaller than the size difference of the image shown by.

The graphics module 516 of FIG. 2 provides the image display 11 with components for setting visual effects (eg, brightness, transparency, saturation, contrast, or other visual characteristic) of the displayed image 200. It includes various known software components for rendering and displaying image 200. The graphic module 516 includes software components for controlling hardware components such as a light source (not shown) other than the display (not shown) of the image display unit 11 that may affect the visual effect of the image 200. May be included.

Further, the graphic module 516 may hide the image 200 if the notification necessity degree determined by the notification necessity degree determination module 508 is lower than a predetermined notification necessity degree threshold stored in the memory 18.

The processor 16 and the image processing circuit 20 display the coordinates set by the image position setting module 510 (the left-right direction (X-axis direction) when the driver 4 sees the direction of the display area 100 from the driver's seat of the vehicle 1 and the vertical direction). Direction (including at least the Y-axis direction), the size set by the image size setting module 512, the perspective set by the perspective setting module 514, and the property set by the graphic module 516, which are visually recognized by the driver 4. Thus, the image 200 is displayed on the image display unit 11.

FIG. 5 is a flowchart of a perspective image display process. First, the processor 16 executes the real object related information detection module 502 to acquire the position (area) of the real object (running lane 310) in the foreground 300 (step S11), and executes the eye position detection module 506. , The position of the eyes of the viewer (driver 4) is acquired (step S12), the image position setting module 510 is executed, the position of the real object acquired in step S11, and the eyes of the viewer acquired in step S12. Based on the position, the image 200 is set at a position having a specific positional relationship with the real object in the foreground 300 (step S13), and the image 200 is displayed at the position of the image 200 set at step S13 (step S14). ).

FIG. 6 is a flowchart of a display process for changing the display mode of the image according to the vibration of the own vehicle. This process starts when the perspective image 210 is displayed by the display process of FIG. First, the processor 16 executes the vibration determination module 504 to determine whether it can be estimated that the host vehicle 1 is vibrating at a certain size or more (step S21), and proceeds to step S22. In step S22, when it is not estimated that the host vehicle 1 is vibrating above a certain size, the processor 16 displays the perspective image 210 on the image display unit 11, and the host vehicle 1 vibrates above a certain size. If it is estimated that the perspective image is present, the processor 16 displays the perspective image 220 having a weak perspective or the non-perspective image 230 having no perspective on the image display unit 11.

The operations of the processing processes described above can be performed by executing one or more functional modules of an information processing device such as a general-purpose processor or an application-specific chip. All of these modules, combinations of these modules, and/or combinations with known hardware capable of substituting their functions are included in the scope of protection of the present invention.

The functional blocks of the vehicular display system 10 are optionally implemented by hardware, software, or a combination of hardware and software to implement the principles of the various described embodiments. The functional blocks described in FIG. 2 may optionally be combined or one functional block may be separated into two or more sub-blocks to implement the principles of the described embodiments. As will be appreciated by those skilled in the art. Thus, the description herein optionally supports any possible combination or division of the functional blocks described herein.

As described above, the display control device of the present embodiment weakens the perspective of the displayed perspective image 210 when it is estimated that the vehicle 1 is vibrating at a certain size or more. According to this, when the vibration of the vehicle is detected such that the relative angle between the traveling lane and the image 200 that is displayed substantially along the road surface of the traveling lane 310 (substantially parallel) is broken. Since the image 200 has a display mode in which the perspective is not as if it is not along the road surface of the driving lane 310, it is difficult to give a sense of discomfort even when the relative angle between the image 200 and the road surface of the driving lane 310 changes due to vibration. You can

The image display unit 11 may employ a stereoscopic display and display the image 200 in the display area 100, which is a three-dimensional area.

Further, when it is determined that there is vibration, the display control device 13 (processor 16) may weaken the perspective of the perspective image 210, and if the vibration determination result can be obtained from the I/O interface 14, the display control device 13 (processor 16) reduces the vibration. The determination function (vibration determination module 504) may not be included. In this case, the display control device 13 (processor 16) may obtain the vibration determination result from the vehicle ECU 401, for example.

DESCRIPTION OF SYMBOLS 1... Own vehicle, 2... Front windshield, 4... Driver, 5... Dashboard, 10... Vehicle display system, 11... Image display part, 11a... Display light, 13... Display control device, 14... I/O Interface, 16... Processor, 18... Memory, 20... Image processing circuit, 100... Display area, 200... Image, 210... Perspective image, 220... Perspective image, 230... Non-perspective image, 300... Foreground, 310... Driving lane, 311... Marking line, 312... Marking line, 401... Vehicle ECU, 403... Road information database, 405... Own vehicle position detecting unit, 407... Outside sensor, 409... Line-of-sight direction detecting unit, 411... Eye position detecting unit, 413... Portable information terminal, 420... External communication connection device, 502... Real object related information detection module, 504... Vibration determination module, 506... Eye position detection module, 508... Notification necessity determination module, 510... Image position setting module, 512... Image size setting module, 514... Perspective setting module, 516... Graphic module, θ... Acute angle

Claims (11)

In a display control device (13) for controlling an image display unit (11) that displays a perspective image (210) that is viewed smaller in the distance, in the foreground seen by the driver of the vehicle,
One or more processors (16),
A memory (18),
One or more computer programs stored in the memory (18) and configured to be executed by the one or more processors (16),
Said one or more processors (16)
If it is estimated that at least the vehicle is vibrating above a certain magnitude,
Reduce the perspective of the displayed perspective image (210), execute a command,
Display controller. The perspective image (210) is viewed by the driver sitting in the driver's seat of the own vehicle in a superimposed manner on the road surface in the foreground.
The display control device according to claim 1. The command for weakening the perspective of the perspective image (210) being displayed is
Changing the perspective image (210) to a non-perspective image (230) without perspective.
The display control device according to claim 1. An image display unit (11) for displaying a perspective image (210) that is visually recognized to be smaller in the distance, in the foreground seen by the driver of the own vehicle;
A display control device (13) for controlling the image display unit (11),
The display control device (13) is
If it is estimated that at least the vehicle is vibrating above a certain magnitude,
Reduce the perspective of the displayed perspective image (210), execute a command,
Head-up display device. The perspective image (210) is viewed by a driver seated in the driver's seat of the own vehicle so as to be superimposed on the road surface in the foreground.
The head-up display device according to claim 4. The display control device (13) is
When weakening the perspective of the displayed perspective image (210),
Changing the perspective image (210) to a non-perspective image (230) without perspective.
The head-up display device according to claim 4. The display area (100) in which the image display section (11) displays the perspective image (210) is the display area (100) and the road surface of the traveling lane of the own vehicle when viewed from the left and right direction of the driver. The acute angle (θ) formed by is set to be in the range of 0 to 45 [degree],
The head-up display device according to claim 4. In a method of controlling an image display unit (11) that displays a perspective image (210) that is visually recognized to be smaller in the distance, in the foreground viewed from the driver of the own vehicle,
If it is estimated that at least the vehicle is vibrating above a certain magnitude,
Weakening the perspective of the displayed perspective image (210).
Method. The perspective image (210) is viewed by the driver sitting in the driver's seat of the own vehicle in a superimposed manner on the road surface in the foreground.
The method of claim 8. To reduce the perspective of the displayed perspective image (210)
Changing the perspective image (210) to a non-perspective image (230) without perspective.
The method of claim 8. A computer program comprising instructions for performing the method according to any one of claims 8-10.

Images