CN117406440A - Virtual image display device - Google Patents

Virtual image display device Download PDF

Info

Publication number
CN117406440A
CN117406440A CN202310837654.9A CN202310837654A CN117406440A CN 117406440 A CN117406440 A CN 117406440A CN 202310837654 A CN202310837654 A CN 202310837654A CN 117406440 A CN117406440 A CN 117406440A
Authority
CN
China
Prior art keywords
display
distance
display mode
target
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310837654.9A
Other languages
Chinese (zh)
Inventor
妹尾俊明
樋口実
村上真一
団功司
山崎贵也
饭高良
佐佐木文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honda Motor Co Ltd
Original Assignee
Honda Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honda Motor Co Ltd filed Critical Honda Motor Co Ltd
Publication of CN117406440A publication Critical patent/CN117406440A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0125—Traffic data processing
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0185—Displaying image at variable distance

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Atmospheric Sciences (AREA)
  • Traffic Control Systems (AREA)
  • Instrument Panels (AREA)

Abstract

Provided is a virtual image display device which can distinguish and recognize the existence of each object even when a plurality of objects are densely present in a range close to the relative position of a host vehicle. The virtual image display device includes: a surrounding state recognition unit 6 that recognizes traffic participants present around the vehicle as a target, and acquires target position information including information on the relative position between the host vehicle and the target; and a display control unit 8 for displaying the object icon images I1, I2 at the display positions on the display screen 51 specified based on the object position information. The display control section 8 is capable of switching between image display in a normal display mode in which the display position is determined based on the object position information so that the driver can recognize the relative position between the host vehicle and the object, and image display in an enlarged display mode in which the display position is determined in such a manner that the interval between the two object icon images on the display screen 51 is larger than in the normal display mode.

Description

Virtual image display device
Technical Field
The present invention relates to a virtual image display device. More specifically, the present invention relates to a virtual image display device that displays a target image related to a target existing around a moving body on a display screen that can be visually confirmed by a driver.
Background
In recent years, in order to improve traffic safety, a virtual image display device has been proposed in which a virtual image that contributes to assisting safe driving is displayed on a monitor that can be visually confirmed by a driving driver. For example, in the device shown in patent document 1, an image acquired by an off-vehicle camera facing the rear of the vehicle is displayed on a monitor, for example. Thus, the driver who is driving can recognize the presence of an object existing behind the vehicle by visually checking the monitor when the vehicle is moving backward, so that the traffic safety can be improved.
[ Prior Art literature ]
(patent literature)
Patent document 1: japanese patent laid-open No. 2009-226978
Patent document 2: japanese patent laid-open No. 2020-013493
Disclosure of Invention
[ problem to be solved by the invention ]
However, in a general virtual image display device, in order to allow a driver to quickly check a landscape that can be visually checked directly by the eyes of the driver with a virtual image displayed on a monitor, in many cases, a virtual image that causes a sense of distance to the viewer is displayed on the monitor.
However, in a virtual image drawn by the near-far method, the size of a target image becomes smaller as the distance from the host vehicle in the depth direction (i.e., the traveling direction) becomes longer. Further, the display position of the target image gradually approaches the vanishing point as the distance from the host vehicle in the depth direction increases. Therefore, in a virtual image drawn by the near-far method, for example, when a plurality of objects are present in a distance from the host vehicle, it is difficult for the driver to recognize the distance between the objects in the distance (in particular, the distance along the depth direction) in the distance.
In contrast, in the invention described in patent document 1, when the size on the display screen of the predetermined notification target is equal to or smaller than the threshold value, the enlarged image obtained by enlarging the size on the display screen of the notification target is additionally displayed. According to the invention described in patent document 1, a target existing in a distance from the host vehicle can be easily recognized.
However, when a plurality of objects are densely present, the objects are superimposed and displayed on the display screen, and therefore, it may be difficult to distinguish the objects from each other by enlarging the size of the display screen. In addition, if a plurality of objects are present in the display screen, if an enlarged image is additionally displayed on the display screen as in the invention described in patent document 1, there is a possibility that other objects are blocked by the enlarged image.
The present invention provides a virtual image display device capable of distinguishing and identifying the existence of each object even when a plurality of objects are densely present in a range close to the relative position of the vehicle.
[ means of solving the problems ]
(1) A virtual image display device (for example, a virtual image display device 1 described later) of the present invention recognizes a traffic participant existing around a moving object (for example, a vehicle V described later) as an object, and displays an object image (for example, object icon images I1 and I2 described later) related to the object on a display screen (for example, a display screen 51 described later) that can be visually confirmed by a driver of the moving object, the virtual image display device comprising: a surrounding state recognition unit (for example, a surrounding state recognition unit 6 described later) that acquires surrounding state information on a surrounding state of the mobile body, and acquires object position information including information on a relative position between the mobile body and the object based on the surrounding state information; and a display control unit (for example, a display control unit 8 described later) for displaying the target image at a display position within the display screen specified based on the target position information; the display control unit is configured to switch between image display in a normal display mode in which the display position is determined based on the target position information so that the driver can recognize the relative position between the moving body and the target, and image display in an enlarged display mode in which the display position is determined so that the distance between the two target images on the display screen is larger than in the normal display mode.
(2) In this case, it is preferable that the display control unit displays, in the enlarged display mode, the object image related information related to each object on the vicinity of the object image when a plurality of object images are displayed on the display screen.
(3) In this case, the target related information preferably includes at least one of position information, moving direction information, moving speed information, and category information of the target.
(4) In this case, it is preferable that the display control unit determines the display position so that, when the object images are displayed on the display screen in such a manner that the object images are perceived as being far and near, and the image display is performed in the enlarged display mode, either or both of a longitudinal interval along a longitudinal direction of the screen and a lateral interval along a lateral direction of the screen of the two object images are larger than in the normal display mode.
(5) In this case, it is preferable that the display screen is divided into a short-distance display area (for example, a short-distance display area R1 described later) for displaying the target image of the short-distance target whose travel direction distance from the moving body to the target is smaller than a distance threshold value, and a long-distance display area (for example, a long-distance display area R2 described later) for displaying the target image of the long-distance target whose travel direction distance is equal to or longer than the distance threshold value in the long-distance display area, and the display control unit determines the display position so that the distance between the two target images displayed in the long-distance display area is larger than that in the normal display mode when the plurality of target images are displayed on the display screen in the enlarged display mode.
(6) In this case, it is preferable that the display control unit displays the image in the enlarged display mode when the surrounding state recognition unit recognizes that the first object and the second object are separated from each other by a distance between the first object and the second object along the traveling direction being smaller than a distance threshold.
(7) In this case, it is preferable that the virtual image display device further includes a visual line direction acquisition unit (for example, a visual line direction acquisition unit 7 described later) that acquires a visual line direction, which is a visual line or a face direction of the driver, and that the display control unit displays the image in the enlarged display mode when the target object is present in the visual line direction.
(8) In this case, the display control unit preferably switches between the image display in the normal display mode and the image display in the enlarged display mode based on the surrounding state information.
(9) In this case, it is preferable that the surrounding state recognition unit recognizes, based on the surrounding state information, a traffic participant existing on the traveling direction side from the moving body as the target, and the display control unit performs the image display in the enlarged display mode when the target is recognized for a predetermined time or longer, when an area having a density of the target that is equal to or greater than a predetermined value is recognized, when a specific facility or a specific device is recognized on the traveling direction side from the moving body, or when a road having a specific form is recognized on the traveling direction side from the moving body.
(effects of the invention)
(1) In the present invention, the surrounding state recognition unit recognizes traffic participants existing around the host vehicle, that is, the mobile body, as the target, and acquires target position information including information on the relative position between the host vehicle and the target. The display control unit causes the target image to be displayed at a display position within the display screen specified based on the target position information. In addition, the display control section is capable of switching between image display in a normal display mode in which the display position is determined based on the object position information so that the driver can recognize the relative position between the moving body and the object, and image display in an enlarged display mode in which the display position is determined in such a manner that the interval between two object images on the display screen is larger than in the normal display mode. Thus, according to the present invention, by performing the image display in the enlarged display mode, even when a plurality of objects are densely present in a range close to the relative position of the host vehicle, the object images are displayed at a distance from each other on the display screen, and thus the driver can recognize the presence of each object by distinction, and the traffic safety can be improved.
(2) In the present invention, when a plurality of object images in the enlarged display mode are displayed on a display screen, a display control unit causes object-related information associated with each object to be displayed in the vicinity of the object image. That is, in the present invention, in the enlarged display mode, by displaying the object images of two objects at a distance on the display screen more apart than in the normal display mode and displaying the object related information in the vicinity thereof, it is possible to display more information while avoiding cluttering the viewer, so that it is possible to improve traffic safety.
(3) In the present invention, the object-related information includes at least any one of position information, movement direction information, and category information of the object. Thus, according to the present invention, it is possible to quickly recognize information such as the position, moving direction, moving speed, and type of a target, which is difficult to grasp only by the target image.
(4) In the present invention, the display control unit displays the object image on the display screen in such a manner that a sense of distance is generated. In addition, in the case of performing image display in the enlarged display mode, the display control section determines the display position such that either or both of a longitudinal interval of the two object images along the longitudinal direction of the screen (i.e., a distance between the two objects along the traveling direction) and a lateral interval of the two object images along the lateral direction of the screen (i.e., a distance between the two objects along the width direction) are greater than in the normal display mode. Thus, according to the present invention, it is possible to make the interval of the target images on the display screen wider than that in the conventional display mode while avoiding the damage to the sense of distance.
(5) In the present invention, the display control unit displays the object image on the display screen in such a manner as to generate a sense of distance, and divides the display screen into a near-distance display region for displaying the object image of the near-distance object and a far-distance display region for displaying the object image of the far-distance object in a region above the near-distance display region in the longitudinal direction of the screen. In this case, since the display position of the target image of the remote target displayed in the remote display area converges on the horizontal line virtually specified in the display screen as the distance from the host vehicle in the traveling direction increases, the two target images are less likely to overlap. In the present invention, therefore, when a plurality of target images are displayed on a display screen in the enlarged display mode, the display control unit determines the display position such that the interval between two target images displayed in the remote display area is larger than that in the normal display mode. Thus, according to the present invention, the presence of a plurality of remote objects existing at such a position that the object images would overlap if in the normal display mode can be distinguished, and the driver can be made to recognize.
(6) In the present invention, the display control unit performs image display in the enlarged display mode when the first and second objects are recognized and the interval between the first and second objects in the traveling direction is equal to or less than the interval threshold. Thus, in the case where two object images overlap if in the normal display mode, by performing image display in the enlarged display mode, the two object images can be prevented from overlapping, so that the presence of the respective objects can be distinguished to be recognized by the driver.
(7) In the present invention, the visual line direction acquisition unit acquires the visual line direction, which is the visual line or the face direction of the driver, and the display control unit performs the image display in the enlarged display mode when the object is present in the visual line direction, that is, when the driver looks at the object. Thus, the driver can distinguish between a plurality of objects present in the direction of gaze.
(8) In the present invention, the display control section switches the image display in the normal display mode and the image display in the enlarged display mode based on surrounding state information about the surrounding state of the host vehicle. Thus, according to the present invention, the normal display mode and the enlarged display mode can be switched at appropriate timings according to the change in the state around the host vehicle.
(9) In the present invention, the display control unit performs the image display in the enlarged display mode when a predetermined number or more of objects are recognized within a predetermined time based on the surrounding state information, when an area in which the density of the objects is equal to or greater than a predetermined value is recognized, when a specific facility or a specific device is recognized to exist on the traveling direction side from the host vehicle, or when a road of a specific form is recognized to exist on the traveling direction side from the host vehicle. Thus, according to the present invention, by performing image display in the enlarged display mode in the case where a large number of objects are already present on the traveling direction side or in the case where a large number of objects are predicted to appear on the traveling direction side, it is possible to distinguish between these multiple objects and identify them by the driver.
Drawings
Fig. 1 is a diagram schematically showing the configuration of a virtual image display device according to an embodiment of the present invention and an automobile in which the virtual image display device is mounted.
Fig. 2 is a plan view of a vehicle traveling on a road with a single three-lane, as viewed from above.
Fig. 3 is a diagram showing an example of a virtual image displayed on a display screen in the case where the display control unit performs image display in the normal display mode in the situation shown in fig. 2.
Fig. 4 is a diagram showing an example of a virtual image displayed on a display screen in the case where the display control unit performs image display in the enlarged display mode in the situation shown in fig. 2.
Fig. 5 is a flowchart showing a flow of displaying a virtual image in the display control section.
Detailed Description
Hereinafter, a virtual image display device according to an embodiment of the present invention will be described with reference to the drawings.
Fig. 1 is a diagram schematically showing the configuration of a virtual image display device 1 according to the present embodiment and a vehicle V on which the virtual image display device 1 is mounted.
The virtual image display device 1 recognizes traffic participants present around the mobile body, particularly traffic participants present on the front side in the traveling direction of the mobile body from a viewpoint determined based on the mobile body, as a target, and displays a target image associated with the target on a display screen 51 of a display 55 provided at a position visually recognizable by a driver of the mobile body, thereby assisting the driver in recognizing the target around the mobile body. In the following, a case where the virtual image display device 1 is mounted on the vehicle V, which is a four-wheel car, will be described, but the present invention is not limited thereto. The present invention may be used to mount a virtual image display device on a mobile body other than a four-wheel car such as a truck, a saddle-ride type vehicle, or a bicycle. In addition, in the case of a straddle-type vehicle mounted on a mobile body, in particular, the virtual image display device may be mounted on a helmet worn by a rider, which is a user.
The vehicle V includes: an external sensor unit 2 that acquires information on the front in the traveling direction; a navigation device 3; an indoor camera unit 4; and, the virtual image display device 1 displays a virtual image for assisting the safe driving of the driver on the display screen 51.
The external sensor unit 2 is constituted by a front camera unit 21, a front radar unit 22, and the like.
The front camera unit 21 includes, for example, a digital camera using a solid-state imaging element such as a charge coupled device (Charge Coupled Devices, CCD) or a complementary metal oxide semiconductor (Comlementary Metal Oxide Semiconductor, CMOS). The front camera unit 21 is mounted in a position closer to a front window, for example, in a vehicle interior side of a roof of the vehicle V, in a state of facing a front side in a traveling direction. The front radar unit 22 includes a millimeter wave radar that detects an object by measuring a reflected wave from the object for millimeter wave irradiation. The front Fang Leida unit 22 is mounted to, for example, a front bumper of the vehicle V in a state of being directed toward the front side in the traveling direction.
The navigation device 3 includes, for example, a global navigation satellite system (Global Navigation Satelite System, GNSS) receiver for determining the current position of the vehicle based on signals received from the GNSS satellites, a storage device for storing map information, and the like. Here, the map information also includes information related to the road identification. The navigation device 3 transmits information on the current position of the host vehicle to the virtual image display device 1 together with map information of the current position.
The indoor camera unit 4 includes, for example, a digital camera using the solid-state imaging device described above. The indoor camera unit 4 is provided in the vehicle interior so as to face the face of the driver seated in the driver's seat. The image of the driver photographed by the indoor camera unit 4 is transmitted to the virtual image display apparatus 1.
The virtual image display apparatus 1 includes: a display 5 provided at a position where a driver who is driving can visually confirm the display; a surrounding state recognition unit 6 that obtains surrounding state information on the surrounding state of the vehicle V acquired by the external sensor unit 2 or the navigation device 3, and recognizes the surrounding state of the vehicle V; a visual line direction acquisition unit 7 that acquires a visual line direction, which is the visual line of the driver or the direction of the face, based on the image transmitted from the indoor camera unit 4; and, the display control unit 8 displays a virtual image related to the front of the vehicle V in the traveling direction, which is generated based on the recognition result of the surrounding state recognition unit 6, on the display screen 51 of the display 5.
In the present embodiment, the description is given of the case where the display control unit 8 displays the virtual image on the display screen 51 of the display 5 provided opposite to the driver's seat, but the present invention is not limited to this. The display control unit 8 may display the virtual image using a head-up display with a part of the windshield as a display screen.
The surrounding state recognition unit 6 recognizes, as a target, one or more traffic participants present on the front side in the traveling direction from the viewpoint determined for the vehicle V, based on surrounding state information acquired from the external sensor unit 2 or the navigation device 3, and acquires target position information and target related information of each target. The object position information includes information on the relative position between the viewpoint and each object. More specifically, the object position information includes information on a travel direction distance from the viewpoint to each object along the travel direction and information on a width direction distance from the viewpoint to each object along a width direction orthogonal to the travel direction. The object-related information includes at least any one of position information, movement direction information, movement speed information, and category information of the object.
Here, the viewpoint specified for the vehicle V is specified at an arbitrary position in the vehicle interior of the vehicle V (more specifically, the driver's seat) or outside the vehicle interior of the vehicle V (more specifically, a position separated from the rear end portion of the vehicle V toward the rear side in the traveling direction by a predetermined distance), for example. Here, in the case where the viewpoint is determined to the driver's seat, a virtual image of the same viewpoint as the driver may be displayed. In addition, when the viewpoint is determined to be a position away from the rear end portion of the vehicle V toward the rear side, a bird's-eye-view virtual image including the host vehicle in the field of view may be displayed.
The surrounding state identifying section 6 acquires the target position information or target association information as described above, and acquires traffic environment information of the traffic participant including the vehicle V and the identified target based on the surrounding state information acquired from the external sensor unit 2 or the navigation device 3. Here, the traffic environment information includes information on the number of lanes of the road being traveled, the presence or absence of a pedestrian path, the type of facilities or equipment present on the front side in the traveling direction, the form of the road present on the front side in the traveling direction, and the like.
In the present embodiment, the case where the surrounding state recognition unit 6 acquires surrounding state information from the external sensor unit 2 or the navigation device 3, which is an in-vehicle device that moves together with the vehicle V, is described, but the present invention is not limited to this. The surrounding state identifying unit 6 may acquire the surrounding state information by inter-vehicle communication between the host vehicle and another vehicle traveling nearby, communication between a server device monitoring the traffic state of the road on which the host vehicle is traveling, and the host vehicle, or the like.
The display control unit 8 generates a virtual image that reproduces the situation seen from the viewpoint toward the front side in the traveling direction based on the recognition result of the surrounding state recognition unit 6, and displays the virtual image on the display screen 51 of the display 5. More specifically, the display control unit 8 superimposes and draws, at a predetermined position of the background image corresponding to the traffic environment recognized by the surrounding state recognition unit 6, a target image associated with each target recognized by the surrounding state recognition unit 6 in a predetermined size, thereby generating a virtual image that causes the viewer to feel a sense of distance.
The display control unit 8 can generate a virtual image in the normal display mode or the enlarged display mode, and perform image display of the virtual image. In the following, a flow of generating virtual images in each display mode will be described by taking a situation as shown in fig. 2 as an example.
Fig. 2 is a plan view of the vehicle V traveling on the road 9 of the one-sided three-lane, as viewed from above. Fig. 2 shows a case where the host vehicle, that is, the vehicle V, is traveling in the lane 91 at the center, and another vehicle 93 and a pedestrian 94 are present in front of the host vehicle in the traveling direction. Fig. 2 shows a case where another vehicle 93 is stopped in a lane 92 on the side of the pedestrian, and the pedestrian 94 is attempting to enter the lane 92 from the side of the pedestrian from the front in the traveling direction of the other vehicle 93. In the following, a case will be described in which the viewpoint 95 is set to the rear side in the traveling direction of the vehicle V, and the surrounding state recognition unit 6 recognizes the other vehicle 93 as a first target, and recognizes the pedestrian 94 as a second target. Thus, in fig. 2, "y1" represents the travel direction distance of the first object, "y2" represents the travel direction distance of the second object, "x1" represents the width direction distance of the first object, and "x2" represents the width direction distance of the second object.
< regular display mode >)
Fig. 3 is a diagram showing an example of a virtual image displayed on the rectangular display screen 51 in the case where the display control unit 8 performs image display in the normal display mode in the situation shown in fig. 2. As shown in fig. 3, the virtual image is composed of, for example, a background image Ib simulating a scribe line of a road on which the vehicle is traveling, a host vehicle icon image Iv associated with the vehicle V, and object icon images I1, I2 associated with the respective objects 93, 94. Hereinafter, the host vehicle icon image Iv and the object icon image I1 associated with the object for the vehicle are simplified to square marks, and the object icon image I2 associated with the object for the pedestrian is simplified to circular marks.
In the case of displaying an image in the normal display mode, the display control unit 8 causes the host vehicle icon image Iv and the object icon images I1 and I2 to be displayed on the background image Ib generated based on the traffic environment recognized by the surrounding state recognition unit 6 in such a position and size that the viewer can recognize the relative position between the host vehicle and each object and can generate a sense of distance. In the following, description will be made on a case where only the scribe line of the road is displayed as the background image, but the present invention is not limited to this. In addition to the scribing of the road, a structure such as facilities or equipment existing on the road may be displayed as a background image, or grid lines in a grid shape may be displayed as a background image.
When displaying an image in the normal display mode, the display control unit 8 reduces the size of each icon image Iv, I1, I2 on the display screen 51 as the distance in the traveling direction, that is, the traveling direction distance from the viewpoint to the object is longer.
When the image is displayed in the normal display mode, the display control unit 8 causes the display positions of the icon images Iv, I1, I2 on the display screen 51 to approach the horizontal reference line L1 in the screen transverse direction as the width direction distance from the viewpoint to the object is longer, as the horizontal reference line L1 virtually specified in the display screen 51 is longer, and as the distance in the traveling direction is longer. The display control unit 8 displays the object existing on the right hand side in the traveling direction from the viewpoint on the right hand side of the horizontal reference line L1 in the display screen 51, and displays the object existing on the left hand side in the traveling direction from the viewpoint on the left hand side of the horizontal reference line L1 in the display screen 51. In the following, a case will be described in which the horizontal reference line L1 extending in the longitudinal direction of the screen is set in the center of the display screen 51, but the present invention is not limited to this.
In the case of performing image display in the normal display mode, the display control unit 8 causes the display positions of the icon images Iv, I1, I2 on the display screen 51 to be further apart from the screen lower end line L2 of the display screen 51 along the screen longitudinal direction orthogonal to the screen transverse direction as the distance in the traveling direction from the viewpoint to the object is longer.
In the case of performing image display in the normal display mode, the display control unit 8 sets the display positions and sizes of the icon images Iv, I1, I2 on the display screen 51 in accordance with the above-described flow, and thereby can display a virtual image that enables the viewer to recognize the relative position between the host vehicle and each object and to generate a sense of distance. However, in the virtual image displayed on the display screen 51 in the normal display mode, the object icon images I1 and I2 associated with the two objects 93 and 94 existing in the distant place from the viewpoint in the traveling direction are often superimposed as shown in fig. 3, and therefore, it is sometimes difficult for the driver to distinguish between the plurality of objects 93 and 94 existing in the distant place from the viewpoint.
< enlarged display mode >)
Fig. 4 is a diagram showing an example of a virtual image displayed on the display screen 51 in the case where the display control unit 8 performs image display in the enlarged display mode in the situation shown in fig. 2.
As shown in fig. 4, when the image is displayed in the enlarged display mode, the display control unit 8 causes the host vehicle icon image Iv and the object icon images I1 and I2 to be displayed on the background image Ib generated based on the traffic environment recognized by the surrounding state recognition unit 6 in such a position and size that the viewer can recognize the relative position between the host vehicle and each object and can generate a sense of distance.
Here, in the case of performing image display in the enlarged display mode, the display control unit 8 changes at least the display positions of the object icon images I1 and I2 on the display screen 51 from the normal display mode in order to easily distinguish a plurality of objects existing far from the viewpoint by the viewer. More specifically, in the case of performing image display in the enlarged display mode, the display control unit 8 determines the display positions of the respective object icon images I1, I2 such that the interval between the two object icon images I1, I2 on the display screen 51 is larger than the interval between the two object icon images I1, I2 displayed on the display screen 51 in the normal display mode.
More specifically, the display control unit 8 extracts, as the enlargement target object, an object whose object icon image on the display screen in the normal display mode is at a distance equal to or less than a predetermined distance from among the plurality of objects recognized by the surrounding state recognition unit 6, and changes, from the display position in the normal display mode, either or both of the display positions of the two object icon images associated with the two extracted enlargement target objects in the direction of being separated from each other. At this time, the display control unit 8 brings the display position of the extracted enlargement target object whose traveling direction distance from the viewpoint is shorter from the display position specified in the normal display mode closer to the screen lower end line L2 side by a predetermined distance from the display position specified in the normal display mode, and brings the display position of the enlargement target object whose traveling direction distance from the viewpoint is longer from the display position specified in the normal display mode farther from the screen lower end line L2 side by a predetermined distance from the display position specified in the normal display mode. The display control unit 8 also causes the display position of the extracted enlargement target object having a shorter width direction distance from the viewpoint to be closer to the predetermined distance from the display position specified in the normal display mode toward the lateral reference line L1 side along the screen transverse direction, and causes the display position of the enlargement target object having a longer width direction distance from the viewpoint to be farther from the predetermined distance from the lateral reference line L1 side along the screen transverse direction from the display position specified in the normal display mode. In this way, the display control unit 8 determines the display position so that, when the image display in the enlarged display mode is performed, both or either of the vertical interval along the vertical direction of the screen and the horizontal interval along the horizontal direction of the screen of the two target icon images is larger than in the normal display mode.
When changing the display position of the enlargement target object from the display position in the normal display mode as described above, the display control section 8 makes the size of the object icon image closer to the screen lower end line L2 side along the screen longitudinal direction slightly larger than the size in the normal display mode, and makes the size of the object icon image farther from the screen lower end line L2 side along the screen longitudinal direction slightly smaller than the size in the normal display mode. Therefore, the distance between the two object icon images can be increased while avoiding damage to the sense of distance.
By performing image display in the enlarged display mode by the display control unit 8 according to the flow described above, the overlapping of the two target icon images I1 and I2 in the virtual image generated in the normal display mode (see fig. 3) may be eliminated in the image display in the enlarged display mode as shown in fig. 4. Thus, the driver can easily distinguish between the two objects 93,94 existing in the distant place from the viewpoint.
As described above, the display control unit 8 can facilitate recognition of a plurality of distant objects existing in a distant place from the viewpoint by increasing the interval between the two object icon images in the enlarged display mode as compared with the normal display mode, but in this case, it may be difficult to grasp the sense of distance of a close object existing in a close place from the viewpoint. Accordingly, as indicated by a virtual scribe line L3 in fig. 4, the display control unit 8 divides the display screen 51 into a short-distance display region R1 and a long-distance display region R2, the short-distance display region R1 displaying a target icon image of a short-distance target whose travel direction distance from the viewpoint to the target is smaller than a distance threshold (see fig. 2), and the long-distance display region R2 displaying a target icon image of a long-distance target whose travel direction distance is equal to or greater than the distance threshold along the screen longitudinal direction upper side than the short-distance display region R1. In the case where a plurality of target icon images are displayed in the enlarged display mode, the display control unit 8 determines the display position so that the interval between the two target icon images displayed in the long-distance display region R2 is larger than that in the normal display mode, and matches the display position of the target icon image displayed in the short-distance display region R1 with that in the normal display mode. The distance threshold may be a fixed value or may be changed according to the vehicle speed. That is, the distance threshold may be made longer as the vehicle speed is faster.
As described above, in the image display in the enlarged display mode, the interval between the two object icon images I1, I2 is larger than that in the image display in the normal display mode. Therefore, in the case where the plurality of object icon images I1, I2 are displayed in the enlarged display mode, the display control unit 8 may display the object-related information associated with each object in the vicinity of each object icon image I1, I2 by using the space formed between each object icon image I1, I2, as shown in fig. 4. Fig. 4 shows a case where the moving direction of the second object 94 to enter the roadway from the footpath is indicated by an arrow 99 as object-related information.
In the present embodiment, the description has been made with respect to the case where the display control unit 8 makes the form of the background image 1b the same in the normal display mode and the enlarged display mode, but the present invention is not limited to this. The form of these background images can also be changed between the normal display mode and the enlarged display mode within a range that does not destroy the sense of distance.
Fig. 5 is a flowchart showing a flow of displaying a virtual image on the display screen 51 in the virtual image display apparatus 1.
First, in step ST1, the surrounding state recognition unit 6 acquires surrounding state information from the external sensor unit 2 or the navigation device 3, and proceeds to step ST2.
Next, in step ST2, the surrounding state recognition unit 6 acquires the target position information, the target related information, and the traffic environment information based on the surrounding state information acquired in step ST1, and proceeds to step ST3.
Next, in step ST3, the gaze direction acquisition section 7 acquires the gaze direction of the driver based on the image transmitted from the indoor camera unit 4, and proceeds to step ST4.
Next, in step ST4, the display control unit 8 determines whether any one or a combination of six enlarged display conditions (a) to (F) determined to set the display mode of the virtual image to the enlarged display mode is established, based on the surrounding state information acquired in step ST1 and the line-of-sight direction acquired in step ST3. If the determination result in step ST4 is NO (NO), the display control unit 8 proceeds to step ST5, and displays the virtual image generated in the normal display mode on the display screen 51, and returns to step ST1. If the determination result in step ST4 is YES, the display control unit 8 displays the virtual image generated in the enlarged display mode on the display screen 51, and returns to step ST1. As described above, the display control section 8 switches the image display in the normal display mode and the image display in the enlarged display mode based on the surrounding state information.
Enlarged display condition (a): based on the surrounding state information, objects of a predetermined number or more are recognized within a predetermined time.
Enlarged display condition (B): based on the surrounding state information, the presence of a region in which the density of the object is equal to or higher than a predetermined value is identified.
Enlarged display condition (C): based on the surrounding state information, the presence of a specific facility (for example, a station or business facility where a large number of people or moving objects come in and go out) or a specific device (for example, a crosswalk where a large number of people move, a traffic signal, or the like) is recognized on the traveling direction side from the viewpoint.
Enlarged display condition (D): based on the surrounding state information, the presence of a road of a specific form (for example, an intersection, a merging point of two roads, or the like) is recognized on the traveling direction side from the viewpoint.
Enlarged display condition (E): it is recognized that a target is present in the line of sight direction of the driver who is driving.
Enlarged display condition (F): when the first object and the second object are recognized from the pointing direction side from the viewpoint, the distance between the first object and the second object along the traveling direction is equal to or less than a predetermined distance threshold.
The virtual image display apparatus 1 according to the present embodiment has the following effects.
(1) The surrounding state recognition unit 6 recognizes traffic participants existing around the host vehicle, that is, the vehicle V, as a target, and acquires target position information including information on the relative position between the host vehicle and the target. The display control unit 8 causes the target icon image to be displayed at a display position within the display screen 51 specified based on the target position information. In addition, the display control section 8 is capable of switching the image display in the normal display mode that determines the display position based on the object position information so that the driver can recognize the relative position between the vehicle V and the object, and the image display in the enlarged display mode that determines the display position in such a manner that the interval between the two object icon images on the display screen 51 is larger than in the normal display mode. Thus, according to the virtual image display device 1, by performing the image display in such an enlarged display mode, even when a plurality of objects are densely present in a range close to the relative position of the host vehicle, the object images are displayed at intervals on the display screen 51, and thus the presence of each object can be recognized by distinguishing the driver, and the traffic safety can be improved.
(2) In the present invention, when a plurality of object images in the enlarged display mode are displayed on a display screen, a display control unit causes object-related information associated with each object to be displayed in the vicinity of the object image. That is, in the present invention, in the enlarged display mode, by displaying the object images of two objects at a distance on the display screen more apart than in the normal display mode and displaying the object related information in the vicinity thereof, it is possible to display more information while avoiding cluttering the viewer, so that it is possible to improve traffic safety.
(3) The object association information includes at least any one of position information, movement direction information, and category information of the object. Thus, according to the virtual image display apparatus 1, it is possible to quickly recognize information that is difficult to grasp only by the object image, such as the position, moving direction, moving speed, and type of the object.
(4) The display control unit 8 displays the icon image of the object on the display screen 51 in such a manner as to generate a sense of distance. In addition, in the case of performing image display in the enlarged display mode, the display control section 8 determines the display position such that either or both of the longitudinal interval of the two object icon images along the longitudinal direction of the screen (i.e., the distance between the two objects along the traveling direction) and the lateral interval of the two object icon images along the lateral direction of the screen (i.e., the distance between the two objects along the width direction) are greater than in the normal display mode. Thus, according to the virtual image display apparatus 1, the interval of the target icon images on the display screen 51 can be made wider than in the normal display mode while avoiding the damage to the sense of distance.
(5) In the virtual image display device 1, the display control unit 8 displays the object icon image on the display screen 51 in such a manner as to generate a sense of distance, and divides the display screen 51 into a short-distance display region R1 in which the object icon image of the short-distance object is displayed, and a long-distance display region R2 in which the object icon image of the long-distance object is displayed in a region above the short-distance display region R1 in the screen longitudinal direction. In this case, since the display position of the target icon image of the remote target displayed in the remote display region R2 converges on the horizontal line virtually specified on the display screen 51 as the distance from the host vehicle in the traveling direction increases, the two target icon images are less likely to overlap. Therefore, in the virtual image display apparatus 1, when a plurality of target icon images are displayed on the display screen 51 in the enlarged display mode, the display control section 8 determines the display position such that the interval between the two target icon images displayed in the remote display region R2 is larger than in the normal display mode. Thus, according to the virtual image display apparatus 1, it is possible to distinguish the presence of a plurality of remote objects existing at such a position that the object icon images overlap in the normal display mode, and to recognize the driver.
(6) In the virtual image display device 1, when the first and second objects are recognized and the distance between the first and second objects in the traveling direction is equal to or less than the distance threshold, the display control unit 8 performs image display in the enlarged display mode. Thus, in such a case that two object icon images overlap if in the normal display mode, by performing image display in the enlarged display mode, the two object icon images can be prevented from overlapping, so that the presence of the respective objects can be distinguished to make the driver recognize.
(7) In the virtual image display device 1, the visual line direction acquisition unit 7 acquires the visual line direction, which is the visual line direction of the driver or the face direction, and the display control unit 8 performs image display in the enlarged display mode when the object is present in the visual line direction, that is, when the driver looks at the object. Thus, the driver can distinguish between a plurality of objects present in the direction of gaze.
(8) In the virtual image display device 1, the display control section 8 switches between image display in the normal display mode and image display in the enlarged display mode based on surrounding state information about the surrounding state of the host vehicle. Thus, according to the virtual image display apparatus 1, the normal display mode and the enlarged display mode can be switched at appropriate timings according to the change in the state around the host vehicle.
(9) In the virtual image display device 1, the display control unit 8 performs image display in the enlarged display mode when a predetermined number or more of objects are recognized within a predetermined time based on surrounding state information, when an area in which the density of objects is equal to or greater than a predetermined value is recognized, when a specific facility or a specific device is recognized to exist on the traveling direction side from the host vehicle, or when a specific form of road is recognized to exist on the traveling direction side from the host vehicle. Thus, according to the virtual image display device 1, when a large number of objects are already present on the traveling direction side or when a large number of objects are predicted to appear on the traveling direction side, the image display in the enlarged display mode is performed, so that the driver can recognize these large number of objects by distinguishing them.
While the above description has been given of the embodiment of the present invention, the present invention is not limited to this. The construction of the details may be appropriately changed within the gist of the invention.
For example, in the above embodiment, the case where the display control unit 8 displays the virtual image in such a manner that the viewer is perceived as being far and near on the display screen 51 has been described, but the present invention is not limited to this. The display control unit 8 may display a virtual image that does not cause the viewer to feel a sense of distance, such as an overhead image or a planar image, but can recognize the relative position between the host vehicle and the object.
Reference numerals
V: vehicle with a vehicle body having a vehicle body support
1: virtual image display device
2: external sensor unit
21: front camera unit
22: front radar unit
3: navigation device
4: indoor camera unit
5: display device
51: display picture
6: surrounding state recognition unit
7: line-of-sight direction acquisition unit
8: display control unit
L1: transverse datum line
L2: lower end line of picture
I1, I2: object icon image (object image).

Claims (9)

1. A virtual image display device for recognizing traffic participants existing around a moving object as a target and displaying a target image related to the target on a display screen which can be visually confirmed by a driver of the moving object, the virtual image display device comprising:
a surrounding state recognition unit that acquires surrounding state information related to a surrounding state of the mobile body, and acquires object position information including information on a relative position between the mobile body and the object based on the surrounding state information; the method comprises the steps of,
a display control unit that causes the target image to be displayed at a display position within the display screen specified based on the target position information;
the aforementioned display control section is capable of switching between image display in the normal display mode and image display in the enlarged display mode,
The normal display mode is to determine the display position based on the target position information so that the driver can recognize the relative position between the moving body and the target,
the enlarged display mode is to determine the display position such that a distance between the two target images on the display screen is larger than that in the normal display mode.
2. The virtual image display device according to claim 1, wherein the display control unit causes, when a plurality of the object images are displayed on the display screen in the enlarged display mode, object-related information associated with each of the objects to be displayed in the vicinity of the object image.
3. The virtual image display device according to claim 2, wherein the object-related information includes at least any one of position information, moving direction information, moving speed information, and category information of the object.
4. The virtual image display device according to claim 3, wherein the display control unit determines the display position such that, when the object images are displayed on the display screen in such a manner as to generate a sense of distance, and the images are displayed in the enlarged display mode, either or both of a longitudinal interval along a longitudinal direction of the screen and a lateral interval along a lateral direction of the screen are larger than in the normal display mode.
5. The virtual image display device according to claim 4, wherein the display screen is divided into a short-distance display area for displaying the object image of a short-distance object whose travel direction distance from the moving body to the object is smaller than a distance threshold value along a travel direction of the moving body, and a long-distance display area for displaying the object image of a long-distance object whose travel direction distance is equal to or larger than the distance threshold value along an area above the short-distance display area in a longitudinal direction of the screen,
the display control unit determines the display position such that a distance between two of the object images displayed in the remote display area is larger than that in the normal display mode when the plurality of object images are displayed on the display screen in the enlarged display mode.
6. The virtual image display device according to claim 4, wherein the display control unit displays the image in the enlarged display mode when the surrounding state recognition unit recognizes that the first object and the second object are separated from each other by a distance between the first object and the second object in the traveling direction being smaller than a distance threshold.
7. The virtual image display device according to claim 4, further comprising a visual line direction acquisition unit configured to acquire a visual line direction which is a visual line or a face direction of the driver,
when the object is present in the line of sight direction, the display control unit displays an image in the enlarged display mode.
8. The virtual image display device according to claim 4, wherein the display control section switches between the image display in the normal display mode and the image display in the enlarged display mode based on the surrounding state information.
9. The virtual image display device according to claim 8, wherein the surrounding state recognition unit recognizes, as the target, a traffic participant existing on a traveling direction side from the moving body based on the surrounding state information,
the display control unit displays an image in the enlarged display mode when the object is recognized to be a predetermined number or more of objects within a predetermined time based on the surrounding state information, when the object is recognized to be a region having a density of a predetermined value or more of objects, when a specific facility or a specific device is recognized to exist on the traveling direction side from the moving body, or when a road of a specific form is recognized to exist on the traveling direction side from the moving body.
CN202310837654.9A 2022-07-15 2023-07-10 Virtual image display device Pending CN117406440A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022114118A JP2024011841A (en) 2022-07-15 2022-07-15 virtual image display device
JP2022-114118 2022-07-15

Publications (1)

Publication Number Publication Date
CN117406440A true CN117406440A (en) 2024-01-16

Family

ID=89485960

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310837654.9A Pending CN117406440A (en) 2022-07-15 2023-07-10 Virtual image display device

Country Status (2)

Country Link
JP (1) JP2024011841A (en)
CN (1) CN117406440A (en)

Also Published As

Publication number Publication date
JP2024011841A (en) 2024-01-25

Similar Documents

Publication Publication Date Title
US11767024B2 (en) Augmented reality method and apparatus for driving assistance
CN109427199B (en) Augmented reality method and device for driving assistance
EP3705846A1 (en) Object location indicator system and method
US9747800B2 (en) Vehicle recognition notification apparatus and vehicle recognition notification system
US9507345B2 (en) Vehicle control system and method
US10262629B2 (en) Display device
US8094192B2 (en) Driving support method and driving support apparatus
US20160292998A1 (en) Vehicle traveling control device
US20090132161A1 (en) Navigation device and its method
CN102889892A (en) Live-action navigation method and navigation terminal
JP2018101302A (en) Vehicle control system, vehicle control method, and vehicle control program
JP7163748B2 (en) Vehicle display control device
US10369995B2 (en) Information processing device, information processing method, control device for vehicle, and control method for vehicle
CN110574357B (en) Imaging control apparatus, method for controlling imaging control apparatus, and moving body
CN109927629B (en) Display control apparatus, display control method, and vehicle for controlling projection apparatus
US20230135641A1 (en) Superimposed image display device
JP2008097279A (en) Vehicle exterior information display device
JP7315101B2 (en) Obstacle information management device, obstacle information management method, vehicle device
US20210284192A1 (en) Movable object control device, movable object control method, and storage medium storing program
CN117406440A (en) Virtual image display device
CN109070799B (en) Moving body periphery display method and moving body periphery display device
US20240017673A1 (en) Virtual image display device
JP2021149641A (en) Object presentation device and object presentation method
JP4899493B2 (en) Vehicle information providing device and lane keeping control device
JP6759072B2 (en) Image processing equipment, imaging equipment, and display system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination