JP2020046892A - Vehicle display device, method, and computer program - Google Patents

Abstract

To make a vehicle driver effectively recognize information about light irradiated by a lamp unit.SOLUTION: A processor 16 acquires a state of a light pattern 200 to which a real object 310 present in front of a self-vehicle 1 is exposed, and a position of the light pattern 200 from a peripheral interface 14; and determines a display position of a notification image 120 including information about the state of the light pattern 200 in such a manner that the notification image 120 is visually recognized near the position of the light pattern 200 by a driver 4. An image display part 11 displays the notification image 120 at the determined display position.SELECTED DRAWING: Figure 3C

Description

  The present disclosure relates to a vehicle display device, a method, and a computer program that are used in a vehicle and superimpose an image on a foreground of the vehicle to visually recognize the image.

  Patent Literature 1 discloses a vehicle headlight control system that sets an area illuminated by a lamp unit of a vehicle so as to avoid a person (pedestrian) existing in front of the vehicle.

  Further, in Patent Document 2, a pattern that prompts a pedestrian to cross the road or a pattern that informs the user of the prohibition of crossing is provided on a road surface near the pedestrian by a pattern irradiating means (for example, a lamp unit of the vehicle) provided on the vehicle. A projection technique is described.

JP 2013-199263 A JP 2015-143093 A

  However, in the vehicle lighting units of Patent Documents 1 and 2, the pedestrian in front of the vehicle is avoided from the pedestrian, turned to the pedestrian, or presents the intention of the own vehicle (or the driver) to the pedestrian. However, there is a problem that it is difficult for the driver of the vehicle to recognize what kind of light the lamp unit emits to the pedestrian.

  A summary of certain embodiments disclosed herein is provided below. It should be understood that these aspects are presented merely to provide the reader with an overview of these particular embodiments, and are not intended to limit the scope of this disclosure. Indeed, the present disclosure may encompass various aspects not described below.

  The summary of the present disclosure relates to making a driver of a vehicle effectively recognize information about light emitted by a lamp unit. More specifically, by making it easy to recognize the state of the light pattern emitted by the lamp unit, it is possible to encourage the operation of the vehicle that matches this light pattern.

  Therefore, the display device for a vehicle described in this specification acquires the state and position of the light pattern emitted by the vehicle, and the driver of the vehicle moves in the vicinity of the light pattern emitted to the real object existing in the actual scene. A notification image representing the state of the light pattern is displayed at a position where is visible. In some embodiments, the notification image may include an image representing the positional relationship between the real object and the light pattern, and further, the left and right sides of the light pattern and the real object when facing forward from the driver's seat of the vehicle. The first image representing the light pattern and the second image representing the real object may be displayed side by side in the left-right direction when facing forward from the driver's seat of the vehicle so as to show the positional relationship in the direction.

FIG. 2 is a diagram illustrating an example of a vehicle display device according to some embodiments. 1 is a block diagram of a vehicle display device according to some embodiments. FIG. 9 is a diagram illustrating a display example of an image displayed by the vehicle display device according to some embodiments. FIG. 3B is a diagram illustrating a display example of an image when time further elapses from the situation illustrated in FIG. 3A. FIG. 3C is a diagram showing a display example of an image when time has further elapsed from the situation shown in FIG. 3B. FIG. 10 is a diagram illustrating a display example of a notification image displayed by the vehicle display device according to some embodiments. FIG. 10 is a diagram illustrating a display example of a notification image displayed by the vehicle display device according to some embodiments. FIG. 4 is a flow diagram illustrating a process for displaying a notification image according to some embodiments.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited by the following embodiments (including the contents of the drawings). Modifications (including deletion of components) can be made to the following embodiments. In the following description, well-known technical items are not described in order to facilitate understanding of the present invention.

  In the following, FIGS. 1 and 2 provide a description of an exemplary vehicle display device. 3A to 3C, 4A, and 4B provide display examples. FIG. 5 is a flowchart showing a process of moving an image.

  Please refer to FIG. The image display unit 11 that displays an image in the vehicle display device 10 of the first embodiment is a head-up display (HUD) device provided in the dashboard 5 of the host vehicle 1. The HUD device emits the display light 11a toward the front windshield 2 (which is an example of a projection target member), and displays an image in the virtual display area 100 to be visually recognized through the front windshield 2. The image is visually recognized so as to be superimposed on the foreground 300 which is a real space to be displayed.

  The image display unit 12 that displays an image on the vehicle display device 10 according to the second embodiment is a head-mounted display (hereinafter, HMD) device. The driver 4 wears the HMD device on his / her head and sits down on the seat of the host vehicle 1, so that the displayed image is visually superimposed on the foreground 300 via the front windshield 2 of the host vehicle 1. . The display area 100 on which the vehicle display device 10 displays a predetermined image is fixed at a specific position based on the coordinate system of the host vehicle 1, and is fixed at the specific position when the driver 4 faces in that direction. An image can be visually recognized in the displayed display area 100.

  The image display units 11 and 12 include a pedestrian 310, a road surface 320, a road sign (not shown), a building, an obstacle (a bicycle, an obstacle) existing in the foreground 300 which is a real space visually recognized through the front windshield 2 of the vehicle 1. Visual augmented reality (AR, Augmented Reality) can also be formed by visually recognizing an image near a real object such as a motorcycle or another vehicle or at a position overlapping with the real object.

  FIG. 2 is a block diagram of the vehicle display device 10 according to some embodiments. The vehicle display device 10 includes an image display unit 11 (12), a peripheral interface 14, one or more processors 16, a storage unit 18, and an image processing circuit 20. Various functional blocks described in FIG. 2 may be configured by hardware, software, or a combination of both. FIG. 2 is only one example of an embodiment, and the components shown may be combined into fewer components or there may be additional components. For example, an image processing circuit 20 (eg, a graphics processing unit) may be included in one or more processors 16.

  As shown, the processor 16 and the image processing circuit 20 are operatively connected to the storage unit 18. More specifically, the processor 16 and the image processing circuit 20 execute the program stored in the storage unit 18 to control the operation of the vehicle display device 10 such as generating or transmitting image data. It can be carried out. Processor 16 or / and image processing circuit 20 may include at least one general purpose microprocessor (eg, a central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA). Or any combination thereof. The storage unit 18 includes any type of magnetic medium such as a hard disk, any type of optical medium such as a CD and a DVD, any type of semiconductor memory such as a volatile memory, and a nonvolatile memory. Volatile memory includes DRAM and SRAM, and non-volatile memory may include ROM and NVROM.

  As shown, processor 16 is operatively coupled to peripheral interface 14. For example, the peripheral interface 14 converts the vehicle display device 10 into a personal area network (PAN) such as a Bluetooth (registered trademark) network, a local area network (LAN) such as an 802.11x Wi-Fi (registered trademark) network, and 4G. Or, it may include a wireless communication interface that connects to a wide area network (WAN), such as an LTE® cellular network. Also, peripheral interface 14 may include a wired communication interface, such as, for example, a USB port, a serial port, a parallel port, an OBDII, and / or any other suitable wired communication port.

  As shown in the figure, the processor 16 is operatively connected to the peripheral interface 14 so that the processor 16 can exchange information with various other electronic devices and the like connected to the vehicle display device 10 (the peripheral interface 14). Become. The peripheral interface 14 can operate, for example, a vehicle ECU 401 provided in the host vehicle 1, a road information database 403, a host vehicle position detection unit 405, an external sensor 407, an eye point detection unit 409, and an external communication connection device 420. Linked to The image display units 11 and 12 are operatively connected to the processor 16 and the image processing circuit 20. Therefore, the images displayed by the image display units 11 and 12 may be based on the image data received from the processor 16 and / or the image processing circuit 20. The processor 16 and the image processing circuit 20 control the images displayed by the image display units 11 and 12 based on information obtained from the peripheral interface 14. The peripheral interface 14 may include a function of processing (converting, calculating, and analyzing) information received from another electronic device or the like connected to the vehicle display device 10.

  The own vehicle 1 has the driving state of the own vehicle 1 (running, stopped, accelerating, decelerating, running at high speed), and other obtainable information (for example, running distance, speed, engine speed, The vehicle ECU 401 detects the accelerator opening, the steering angle, various warning states, and the like. The vehicle ECU 401 controls each unit of the own vehicle 1, and can transmit, for example, a current speed of the own vehicle 1 (an example of information that can estimate a driving state) to the processor 16. The vehicle ECU 401 may transmit an instruction signal indicating an image to be displayed by the vehicle display device 10 to the peripheral interface 14. At this time, the coordinates of the image, the necessity of notification of the image, and / or the necessity of notification May be added to the instruction signal and transmitted.

  The vehicle ECU 401 is operatively connected to a control unit (not shown) of a lamp unit (not shown) that illuminates the front of the vehicle 1. The lamp unit has, for example, a DMD (Digital Micro-Mirror Device) having a plurality of pixels, and not only illuminates the front of the vehicle 1 but also a light pattern that avoids a specific real object, a specific real object. It is possible to irradiate a light pattern to be directed or a road image 320 of a specific real object with a projected image (an example of a light pattern). The vehicle ECU 401 may acquire the state of the light pattern emitted by the lamp unit and the position of the light pattern from the control unit and transmit the acquired light pattern to the peripheral interface 14. The state of the light pattern is a light pattern that avoids a specific real object, a light pattern that is directed to a specific object, or the content of a projected image displayed near a specific real object (for example, a road surface 320). For example, a state different from the normal forward irradiation of the lamp unit may be included. Further, the position of the light pattern may include coordinates at which the lamp unit avoids light irradiation, coordinates at which light is directed, or coordinates at which a projected image is displayed. Further, the coordinates may be a three-dimensional vehicle reference coordinate system based on a predetermined position of the host vehicle 1.

  The host vehicle 1 may include a road information database 403 (an example of a road environment acquisition unit) including a navigation system and the like. The road information database 403 stores, based on the position of the host vehicle 1 acquired from the host vehicle position detection unit 405 described later, road information (lane, white line, stop line, crossing, Includes sidewalks, width of roads, number of lanes, intersections, curves, forks, presence / absence of traffic regulations, etc., location (including distance to own vehicle 1), shape, type (eg, highway, street in the city), etc. ) Can be read and sent to the processor 16.

  The host vehicle 1 may include a host vehicle position detection unit 405 including a GNSS (Global Navigation Satellite System) or the like. The road information database 403 and / or the external communication connection device 420 to be described later continuously or intermittently acquire the position information of the own vehicle 1 from the own vehicle position detection unit 405, and thereby obtain the information around the own vehicle 1. It can be selected and generated and sent to the processor 16.

  The host vehicle 1 may include one or more out-of-vehicle sensors 407 that detect a real object existing around the host vehicle 1 (particularly in the foreground 300 in the present embodiment). The real objects detected by the exterior sensor 407 include, for example, a pedestrian 310, a road surface 320, lane markings 321, 322, and 323 (see FIGS. 3A to 3C), a bicycle (not shown), a motorcycle, another vehicle (such as a preceding vehicle), It may include a roadside object or / and a building. As an external sensor, for example, a millimeter-wave radar, an ultrasonic radar, a radar sensor such as a laser radar, a camera sensor including a camera and an image processing device, may be configured by a combination of both a radar sensor and a camera sensor, You may comprise only either one. For the object detection by the radar sensor or the camera sensor, a conventionally known method is applied. By detecting an object by these sensors, the presence or absence of a real object in the three-dimensional space, and if there is a real object, the position of the real object (relative distance from the own vehicle 1, the traveling direction of the own vehicle 1 In the case of the front-rear direction, the position in the left-right direction, the position in the up-down direction, etc.), the size (size in the horizontal direction (left-right direction), the height direction (up-down direction), etc.), and / or the type are detected Is also good. One or more out-of-vehicle sensors 407 detect a real object in front of the host vehicle 1 in each detection cycle of each sensor and execute real object information (existence of a real object, real object If there is an object, information such as the position, size, and / or type of each real object can be transmitted to the processor 16. The real object information may be transmitted to the processor 16 via another device (for example, the vehicle ECU 401). In addition, when a camera is used as a sensor so that a real object can be detected even when the surroundings are dark, such as at night, an infrared camera or a near-infrared camera is desirable. When a camera is used as a sensor, a stereo camera that can acquire a distance or the like by parallax is desirable.

  The host vehicle 1 may include an eye point detection unit 409 including an infrared camera or the like that detects the position of the eyes of the driver 4. The processor 16 acquires an image captured by the infrared camera (an example of information that can estimate the position of the eyes), and analyzes the captured image to specify the position of the eyes of the driver 4. Note that the processor 16 may acquire, from the peripheral interface 14, information on the position of the eyes of the driver 4 specified from the image captured by the infrared camera. Note that the method of acquiring the position of the eyes of the driver 4 of the host vehicle 1 or the information capable of estimating the positions of the eyes of the driver 4 is not limited thereto, and a known eye point detection (estimation) is used. It may be obtained using technology. The processor 16 may make the image superimposed on the desired position of the foreground 300 visible by adjusting at least the position of the notification image 120 based on the position of the eyes of the driver 4.

  The external communication connection device 420 is a communication device for exchanging information with the own vehicle 1. For example, the other vehicle connected to the own vehicle 1 by vehicle-to-vehicle communication (V2V: Vehicle To Vehicle), communication between pedestrians (V2P: Vehicle). Pedestrians (portable information terminals carried by pedestrians) and network communication devices connected by vehicle-to-vehicle communication (V2I: Vehicle To roadside Infrastructure). Includes everything connected by communication (V2X: Vehicle To Everything). The out-of-vehicle communication connection device 420 has, for example, a function similar to that of the host vehicle position detection unit 405 described above, and may acquire position information of the host vehicle 1 and transmit the position information to the processor 16. It may also have a function of 403 and acquire the road information (an example of real object information) and transmit it to the processor 16. Note that the information acquired from the external communication connection device 420 is not limited to the above.

  The software components stored in the storage unit 18 include a light pattern detection module 502, a real object detection module 504, a real object position estimation module 506, an image position determination module 508, a driving state determination module 510, an eye point detection module 512, and A graphics module 514 is included.

  The light pattern detection module 502 acquires the state of the light pattern emitted by the lamp unit of the host vehicle 1 and the position of the light pattern from which the contents of the notification image 120 described later and the display position are determined. The light pattern detection module 502 obtains, for example, the position of the notification image 120 (FIGS. 3A to 3C, 4A, and 4B) projected on the foreground 300 (the road surface 320) of the vehicle 1 from the vehicle ECU 401. Good. Further, the light pattern detection module 502 estimates the position of the light pattern based on a combination of the position of the light pattern acquired from the vehicle ECU 401 and the position of the light pattern irradiated on the foreground 300 acquired by the outside sensor 407. Good.

  The real object detection module 504 detects a position of a real object existing in the foreground 300, which is a source of determining coordinates of a notification image 120 described later. The real object detection module 504 detects, for example, the position of the real object (the pedestrian 310 in FIGS. 3A to 3C, 4A, and 4B) existing in the foreground 300 of the own vehicle 1 from the outside sensor 407 or the outside communication connection device 420. (Example of real object information) may be obtained. The real object detection module 504 may detect the size (the size in the horizontal direction and the height direction) or / and the type of the real object 310.

  The real object position estimating module 506 estimates a position of a real object existing in the foreground 300, which is a source of determining coordinates of a notification image 120 described later. The real object position estimation module 506 estimates the position of the real object from the state and position of the light pattern (reference numeral 200 in FIGS. 3A to 3C, 4A, and 4B) acquired by the light pattern detection module 502. Good. For example, when the light pattern is in a state of avoiding or pointing to a specific real object, the real object position estimation module 506 may estimate that the real object exists at the position of the light pattern. If it is projected on the nearby road surface 320, it may be estimated that a real object exists on the left or right side of the light pattern. Depending on whether the position of the light pattern is on the left or right side of the lane, it may be estimated whether the real object exists on the left or right of the light pattern.For example, if the position of the light pattern is on the left side of the lane For example, it may be estimated that a real object exists on the left side of the light pattern.

  The image position determination module 508 determines the coordinates of the notification image 120 based on the position of the light pattern detected by the light pattern detection module 502 (when the driver 4 views the direction of the display area 100 from the driver's seat of the vehicle 1). The horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) are determined. The image position determination module 508 determines the coordinates of the notification image 120 so as to have a predetermined positional relationship with the light pattern 200 emitted by the lamp unit. For example, the positions in the left-right direction and the up-down direction of the notification image 120 are determined so that the notification image 120 is visually recognized above the light pattern 200 (the Y-axis positive direction). The image position determination module 508 determines the coordinates of the notification image 120 based on the real object information detected by the real object detection module 504 or the position of the real object estimated by the real object position estimation module 506. Is also good. That is, the image position determination module 508 may determine the coordinates of the notification image 120 so as to have a predetermined positional relationship with the specific real object 310. For example, the positions in the left-right direction and the up-down direction of the notification image 120 may be determined so as to be visually recognized on the right side (negative X-axis direction) of the notification image 120. Further, the image position determination module 508 determines whether the left lane marking 321, the central lane marking 322, or the right lane marking 323 (see FIGS. 3A to 3C, 4 </ b> A, and 4 </ b> B) of the road surface 320 on which the vehicle 1 is traveling. The coordinates of the notification image 120 may be determined based on ()). The “predetermined positional relationship” can be adjusted according to the situation of the real object 310 or the own vehicle 1.

  The driving state determination module 510 acquires the driving state of the own vehicle 1 from the vehicle ECU 401 or a sensor provided in the own vehicle 1 (not shown), or acquires information from which the driving state of the own vehicle 1 can be estimated. It is determined whether the driving state of the vehicle 1 satisfies a predetermined condition. The driving condition of the host vehicle 1 determined by the driving state determination module 510 includes, for example, stopping and deceleration of the host vehicle 1. The driving state of the host vehicle 1 for determining this is, for example, information that can estimate the stop or deceleration of the host vehicle 1. The information from which the driving state of the host vehicle 1 can be estimated includes, for example, the traveling distance, speed, engine speed, accelerator opening, and the like of the host vehicle 1. In addition, the conditions of the driving state of the own vehicle 1 determined by the driving state determination module 510, the driving state of the own vehicle 1 acquired by the driving state determination module 510, or the information that can estimate the driving state of the own vehicle 1 are described above. It is not limited to one.

  The eye point detection module 512 detects the position of the eyes of the driver 4 of the vehicle 1. The eye point detection module 512 determines where the eye height of the driver 4 is in the height area provided in a plurality of stages, detects the eye height of the driver 4 (position in the Y-axis direction), Detection of the height (position in the Y-axis direction) and depth position (position in the Z-axis direction) of the eyes of the driver 4 and detection of the position of the eyes of the driver 4 (position in the X, Y, and Z-axes directions) , Including various software components for performing various operations. The eye point detection module 512 can, for example, obtain the eye position of the driver 4 from the eye point detection unit 409, or can estimate the eye position including the eye level of the driver 4 from the eye point detection unit 409. And the position of the eyes including the height of the eyes of the driver 4 is estimated. The information from which the position of the eyes can be estimated includes, for example, the position of the driver's seat of the vehicle 1, the position of the face of the driver 4, the height of the sitting height, the input value of the driver 4 at an operation unit (not shown), and the like. Is also good.

  The graphic module 514 displays a visual effect (e.g., brightness, transparency, saturation, contrast, or other visual characteristics), a size, a display position, and a display distance of the displayed notification image 120 (when the driver 4 determines that the driver's vehicle 1 And various known software components for changing the distance from the driver 4 to the notification image 120 in the left-right direction (X-axis direction) when the direction of the display area 100 is viewed from the driver's seat. The graphic module 514 displays the coordinates set by the image position determination module 508 (the left-right direction (X-axis direction) when the driver 4 views the direction of the display area 100 from the driver's seat of the vehicle 1) and the up-down direction (Y-axis direction). The display position of the notification image 120 is displayed so as to be visually recognized by the driver 4 in at least the direction (including the direction). When the foreground 300 seen from the driver 4 approaches as the host vehicle 1 advances, the graphic module 514 follows the foreground 300 so that the relative position of the notification image 120 to the foreground 300 does not change. May be continuously adjusted. It should be noted that the graphic module 514 has a position in the depth direction (Z-axis direction) of the light pattern detected by the light pattern detection module 502 or a specific real object to which the light pattern of the lamp unit detected by the real object detection module 504 is directed. The size of the notification image 120 may be adjusted according to the distance between the object 310 and the host vehicle 1. Specifically, the graphic module 514 may increase the size of the notification image 120 stepwise or continuously as the position of the light pattern or the real object 310 approaches the host vehicle 1. Conversely, the graphic module 514 may reduce the size of the notification image 120 stepwise or continuously as the position of the light pattern or the real object 310 approaches the host vehicle 1. Further, the graphic module 514 may adjust the timing at which the image (the notification image 120) is displayed according to the driving state that can be obtained or estimated by the driving state determination module 510.

  Further, the storage unit 18 may include a notification necessity detection module (not shown). The notification necessity detection module detects the necessity of notifying the driver 4 of the real object information together with the real object information detected by the real object detection module 504 (necessity of notification). The notification necessity detection module may detect the notification necessity from various other electronic devices connected to the peripheral interface 14. In addition, the electronic device connected to the peripheral interface 14 in FIG. 2 may transmit information to the vehicle ECU 401, and the notification necessity detection module may detect the notification necessity determined by the vehicle ECU 401 based on the received information. The “notification necessity” is, for example, a danger derived from the degree of possible seriousness, an urgency derived from the length of a reaction time required to perform a reaction action, the own vehicle 1 and the driver 4 ( Alternatively, it can be determined based on the degree of effectiveness derived from the situation of the other occupant of the host vehicle 1), or a combination thereof.

  The storage unit 18 may include a notification necessity estimation module (not shown). The notification necessity estimating module detects the necessity information serving as a source for estimating the notification necessity of the notification image 120, and estimates the notification necessity from the information. The necessity information serving as a basis for estimating the necessity of notification of the notification image 120 may be estimated based on, for example, the position or type of the light pattern of the real object 310 or the lamp unit. It may be estimated on the basis of other information input from another electronic device or in consideration of other information.

  Each of the above modules corresponds to executable instructions for performing one or more of the above functions, and methods (eg, computer executed programs) described in this application.

  Some or all of the above modules are provided at predetermined intervals via the peripheral interface 14 via the vehicle ECU 401, the road information database 403, the vehicle position detection unit 405, the vehicle outside sensor 407, the eye point detection unit 409, and the vehicle communication connection device. A request is sent to 420 to send various information. In response to this, the peripheral interface 14 obtains various information from the vehicle ECU 401, the road information database 403, the own vehicle position detection unit 405, the outside sensor 407, the eye point detection unit 409, and the outside vehicle communication connection device 420, and sends the information to the processor 16. Send out. In some other embodiments, some or all of the modules described above are scheduled to perform a regular or predetermined event (for example, detection of the start of traveling of the vehicle 1 or irradiation of the predetermined light pattern 200). May be requested every time the information is detected).

  3A, 3B, and 3C are diagrams showing changes in images displayed by the vehicle display device 10 as the host vehicle 1 advances. The vehicle display device 10 displays a caution image 110 that calls attention to a specific real object 310 existing in the foreground 300 of the host vehicle 1 as viewed from the driver 4. In the example of FIG. 3A, the attention image 110 includes the text “There is a crosser, please stop”. Note that the attention image 110 is not limited to text, and may include a graphic, an illustration, a still image, a moving image, or the like. As shown in FIG. 3A, the attention image 110 may have a content that prompts the driver 4 to make the driving state of the host vehicle 1 satisfy a specific condition. After the state shown in FIG. 3A, the vehicle display device 10 displays the notification image 120. The notification image 120 in FIG. 3B includes a first image 121 representing the content of the light pattern 200, a second image 122 representing the real object 310, and a balloon-shaped third image 123 indicating the direction of the light pattern 200. After the state shown in FIG. 3B, the lamp unit of the vehicle 1 irradiates the specific light pattern 200 to the road surface 320 near the real object 310. In FIG. 3C, the notification image 120 is a position shifted to the right (negative X-axis direction) of the real object 310 when facing forward from the driver's seat of the host vehicle 1 and in the upward direction of the light pattern 200 ( At the position shifted in the Y-axis positive direction), the driver 4 visually recognizes the position. As shown in FIG. 3C, the light pattern 200 may have a pedestrian crossing shape that intersects a center line 322 of the road surface 320 from the left end line 321 to the right end line 323 of the road surface 320. In this case, as shown in FIGS. 3B and 3C, the first image 121 may be an image imitating the light pattern 200 (crosswalk shape) viewed from above (positive Y-axis direction), and / or The second image 122 may be an image imitating the real object 310 viewed from above (the positive direction of the Y axis).

  4A and 4B are diagrams illustrating a modification of the light pattern 200 and a modification of the notification image 120. FIG. As shown in FIG. 4A, the light pattern 200 is an image simulating a sign including a text visually recognized as “STOP” as viewed from a pedestrian (specific real object 310). May be included. Also in this case, the first image 121 may be an image imitating the light pattern 200 (crosswalk shape) viewed from above (positive direction in the Y-axis). Also, as shown in FIG. 4B, the light pattern 200 includes a text visually recognized as “GO” as viewed from a pedestrian (specific real object 310), an arrow image indicating a direction in which the pedestrian wants to proceed, May be included.

  FIG. 5 is a flow diagram illustrating a process for displaying a notification image, according to some embodiments. For example, when detecting that the vehicle 1 has turned on the lamp unit, the vehicle display device 10 may start this process and repeatedly execute the process until the lamp unit is turned off.

  The vehicle display device 10 acquires information (state and position) related to the specific light pattern 200 emitted by the lamp unit from the vehicle ECU 401 (or the lamp unit) provided in the host vehicle 1 (step S01), and A caution image 110 for directing the driver 4's attention to a real object (for example, a pedestrian) 310 as a target of the pattern 200 is displayed (step S02), and the driving state of the host vehicle 1 satisfies a predetermined condition from the vehicle ECU 401. Is determined (step S03). If the driving state of the own vehicle 1 does not satisfy the predetermined condition (NO in step S03), the process proceeds to step S07, and the lamp unit performs the specific light pattern 200 without displaying the notification image 120 on the vehicle display device 10. Is irradiated.

  When the driving state of the host vehicle 1 satisfies the predetermined condition (YES in step S03), in the vehicle display device 10, the real object detection module 504 acquires the position of the real object 310 or the real object position estimation module 506. Obtains information from which the position of the real object 310 can be estimated (step S04), determines the content of the notification image 120 from the state of the light pattern 200 obtained in step S01, and sets the information to the position of the real object 310 obtained in step S04. The position of the notification image 120 is determined based on the information (step S05), and the notification image 120 is displayed (step S06). Thereafter, the lamp unit emits light including the specific light pattern 200 (step S07).

  As described above, in the present embodiment, the state of the light pattern 200 emitted from the one or more peripheral interfaces 14 to the real object 310 existing in front of the own vehicle 1 by the own vehicle 1, The position of the light pattern 200 is obtained, and the display position of the notification image 120 including information on the state of the light pattern 200 is determined so that the driver can visually recognize the notification image 120 near the position of the light pattern 200. The display unit 11 (12) displays the notification image 120 at the determined display position.

  In some embodiments, the notification image 120 is located near the first image 121 so as to indicate the positional relationship between the first image 121 indicating the state of the light pattern 200 and the real object 310 and the light pattern 200. And the second image 122 to be displayed.

  In some embodiments, the notification image 120 includes the first image 121 so as to show the left-right positional relationship between the light pattern 200 and the real object when the vehicle 1 faces forward from the driver's seat. And the second image 122 may be arranged side by side in the left-right direction when facing forward from the driver's seat of the vehicle 1.

  Further, in some embodiments, the first image 121 is an image imitating the light pattern 200 viewed from above, and the second image 122 is an image imitating the real object 310 viewed from above. It may be.

  Further, in some embodiments, the display position of the notification image 120 is a position shifted in the left-right direction of the real object 310 when facing forward from the driver's seat of the host vehicle 1, and the vertical position of the light pattern 200. The position may be shifted in the direction.

  In some embodiments, the notification image 120 may include a third image 123 indicating the direction of the light pattern 200. The third image 123 may have a balloon shape or a leader line shape indicating the direction of the light pattern 200.

In some embodiments, the notification image 120 may be displayed before the light pattern 200 is displayed.
[Modification]

  In FIG. 1, the display area 100 is provided perpendicular to the traveling direction (Z-axis direction) of the host vehicle 1, but the display area 100 is not limited to this. In some embodiments, for example, the display area 100 rotates around the vehicle width direction (X-axis direction) of the host vehicle 1 and does not become perpendicular to the traveling direction of the host vehicle 1 (Z-axis direction). May be provided. Further, the display area 100 may be provided not at one place but at two or more places, for example, at positions separated in the traveling direction (Z-axis direction) of the host vehicle 1. The display area 100 may be a three-dimensional volume area instead of an area area. That is, the images 110 and 120 may be movable in the depth direction in the three-dimensional space. In other words, the display distance of the images 110 and 120 may be changeable.

  In some embodiments, the position of the light pattern 200 also includes a position in the depth direction when facing forward from the driver's seat of the vehicle 1, and is notified so as to approach the position of the light pattern 200 in the depth direction. The position of the image 120 in the depth direction may be adjusted. In other words, the position of the notification image 120 in the depth direction (Z-axis direction) when the driver 4 faces forward from the driver's seat of the host vehicle 1 matches or is close to the position of the light pattern 200 in the depth direction. May be adjusted as follows.

  In the above embodiment, the graphic module 514 continuously changes the coordinate data and the other property data (size) so that the notification image 120 is visually recognized as approaching the vehicle 1. The graphic module 514 in some embodiments may further gradually change the property data (depression angle / elevation angle) as the host vehicle 1 advances. Further, the property data (azimuth) may be gradually changed.

DESCRIPTION OF SYMBOLS 1 ... Own vehicle, 2 ... Front windshield, 4 ... Driver, 5 ... Dashboard, 10 ... Display device for vehicles, 11 ... Image display part, 12 ... Image display unit, 14: peripheral interface, 16: processor, 18: storage unit, 20: image processing circuit, 100: display area, 110: attention image, 120 ... Information image, 121: first image, 122: second image, 123: third image, 124: guidance image, 200: light pattern, 300: foreground, 310 ... -Pedestrian (real object), 320 ... road surface (real object), 321 ... division line, 322 ... division line, 323 ... division line, 401 ... vehicle ECU, 403 ... Road information database, 405 ... own vehicle position detection 407: Outside sensor, 409: Eye point detection unit, 420: Outside communication connection device, 502: Optical pattern detection module, 504: Real object detection module, 506: Real object Position estimation module, 508: Image position determination module, 510: Operation state determination module, 512: Eye point detection module, 514: Graphic module

Claims (10)

In a vehicle display device for displaying an image in an area overlapping a foreground (300) of a vehicle (1),
Image display units (11, 12);
One or more peripheral interfaces (14);
One or more processors (16);
A memory (18);
One or more computer programs stored in the memory (18) and configured to be executed by the one or more processors (16);
The one or more computer programs comprises:
A state of a light pattern (200) emitted by the vehicle (1) from the one or more peripheral interfaces (14) to a real object (310) existing in front of the vehicle (1); Obtain the position of the light pattern (200),
The display position of the notification image (120) including information on the state of the light pattern (200) is determined so that the driver can visually recognize the notification image (120) near the position of the light pattern (200). ,
The image display unit (11, 12) displays the notification image (120) at the display position.
A display device for a vehicle, including instructions. The notification image (120) includes a first image (121) indicating a state of the light pattern (200) and the second image (121) indicating a positional relationship between the real object (310) and the light pattern (200). A second image (122) displayed near the first image (121).
The vehicle display device according to claim 1. The notification image (120) is configured to display the first image (120) so as to show a left-right positional relationship between the light pattern (200) and the real object when facing forward from the driver's seat of the vehicle (1). 121) and the second image (122) are arranged side by side in the left-right direction when facing forward from the driver's seat of the vehicle (1).
The vehicle display device according to claim 2. The first image (121) is an image imitating the light pattern (200) viewed from above, and the second image (122) is imitating the real object (310) viewed from above. Is an image like
The vehicle display device according to claim 3. The display position of the notification image (120) is a position shifted in the left-right direction of the real object (310) when facing forward from the driver's seat of the vehicle (1), and the light pattern (200). ) In the vertical direction,
The vehicle display device according to claim 1. The notification image (120) includes a third image (123) indicating a direction of the light pattern (200).
The vehicle display device according to claim 1. An instruction for the image display unit (11, 12) to display the notification image (120) at the display position is:
Executed before the light pattern (200) is displayed;
The vehicle display device according to claim 1. The position of the light pattern (200) also includes a position in a depth direction when facing forward from a driver's seat of the vehicle (1),
The image display units (11, 12) are capable of adjusting a position in a depth direction of the notification image (120) when facing forward from a driver's seat of the vehicle (1),
The one or more computer programs comprises:
Further comprising an instruction for adjusting the position of the notification image (120) in the depth direction so as to approach the position of the light pattern (200) in the depth direction;
The vehicle display device according to claim 1. In a vehicle display device having an image display unit (11, 12) and one or more peripheral interfaces (14),
The image display units (11, 12) displaying an image (200) in a predetermined display area (100) overlapping the foreground (300) of the vehicle (1);
The content of a light pattern (200) projected by the vehicle (1) from the one or more peripheral interfaces (14) to a real object (310) existing in front of the vehicle (1); Obtaining the position of the light pattern (200);
The display position of the notification image (120) including information on the content of the light pattern (200) is determined so that the driver can visually recognize the notification image (120) near the position of the light pattern (200). That
The image display unit (11, 12) displaying the notification image (120) at the display position;
Including, methods. A computer program comprising instructions for performing the method of claim 9.

Images