JP2024011841A - virtual image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a virtual image display device capable of distinguishing and recognizing presence of each target even when a plurality of targets are densely present in a range where a relative position from an own vehicle is close.

SOLUTION: A virtual image display device includes: a surrounding state recognition unit 6 configured to recognize a traffic participant present around a vehicle as a target to acquire target position information including information on a relative position between an own vehicle and the target; and a display control unit 8 configured to display target icon images I1 and I2 at a display position in a display screen 51 determined based on the target position information. The display control unit 8 can switch between an image display under a normal display mode in which a display position is determined based on the target position information so that a driver can recognize a relative position between the own vehicle and the target and an image display under an enlarged display mode for determining the display position so that an interval between two target icon images on the display screen 51 becomes larger than the normal display mode.

SELECTED DRAWING: Figure 4

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to a virtual image display device. More specifically, the present invention relates to a virtual image display device that displays target images of targets around a moving object on a display screen that is visible to a driver.

In recent years, in order to improve traffic safety, virtual image display devices have been proposed that display virtual images useful for supporting safe driving on a monitor that can be viewed by a driver while driving. For example, in the device disclosed in Patent Document 1, an image obtained by, for example, an external camera facing toward the rear of the vehicle is displayed on a monitor. This allows the driver to recognize objects behind the vehicle by visually checking this monitor when backing up the vehicle, improving traffic safety. .

JP2009-226978A Japanese Patent Application Publication No. 2020-013393

By the way, in general virtual image display devices, in order to enable the driver to quickly compare the scenery that can be directly viewed with his or her own eyes and the virtual image displayed on the monitor, In many cases, a virtual image is displayed on a monitor in a manner that gives a sense of perspective.

However, in a virtual image drawn based on perspective, the size of the target object image becomes smaller as the distance from the own vehicle in the depth direction (that is, the traveling direction) becomes longer. Furthermore, the display position of the target object image approaches the vanishing point asymptotically as the distance from the vehicle in the depth direction increases. For this reason, in a virtual image drawn based on perspective, if there are multiple targets far away from the driver's vehicle, the distance between these distant targets (especially the distance along the depth direction) can be calculated by the driver. The problem is that it is difficult to recognize.

On the other hand, in the invention described in Patent Document 1, when the size of a predetermined notification target on the display screen is less than a threshold value and is difficult for the driver to recognize, the size of this notification target on the display screen is enlarged. An enlarged image is additionally displayed. According to the invention described in Patent Document 1, it is possible to easily recognize a target object that is far away from the vehicle.

However, when multiple targets are present closely together, they are displayed overlappingly on the display screen, so it is difficult to distinguish and recognize individual targets simply by enlarging the size on the display screen. It can be difficult. Furthermore, when there are multiple targets on the display screen, if an enlarged image is additionally displayed on the display screen as in the invention described in Patent Document 1, there is a risk that other targets may be covered up by the enlarged image. There is also.

The present invention provides a virtual image display device that can distinguish and recognize the existence of individual targets even when a plurality of targets are densely present within a range whose relative positions from the own vehicle are close. The purpose is to provide.

(1) The virtual image display device according to the present invention (for example, virtual image display device 1 described below) recognizes traffic participants existing around a moving object (for example, a vehicle V described below) as a target, and Displaying a target image related to a target (for example, target icon images I1 and I2 described below) on a display screen (for example, a display screen 51 described below) that can be visually recognized by a driver of the mobile object, a surrounding state recognition unit that acquires surrounding state information regarding the surrounding state of the moving object, and obtains target position information including information regarding a relative position between the moving object and the target object based on the surrounding state information; (for example, the surrounding state recognition unit 6 described later), and a display control unit (for example, the display control unit 8 described later) that displays the target image at a display position within the display screen determined based on the target position information. ), wherein the display control unit determines the display position based on the target object position information so that the driver can recognize the relative position between the moving object and the target object. and image display under an enlarged display mode in which the display position is determined so that the interval between the two target images on the display screen is larger than in the normal display mode. It is characterized by being possible.

(2) In this case, when displaying the plurality of target images on the display screen under the enlarged display mode, the display control unit may display target-related information associated with each target to the target. It is preferable to display it near the image.

(3) In this case, it is preferable that the target related information includes at least one of position information, moving direction information, moving speed information, and type information of the target.

(4) In this case, the display control unit displays the target object image on the display screen in a manner that creates a sense of perspective, and when displaying the image under the enlarged display mode, the display control unit Preferably, the display position is determined so that the vertical interval along the vertical direction of the screen and/or the horizontal interval along the horizontal direction of the screen between the target images are larger than in the normal display mode.

(5) In this case, the display screen displays the target image of a short-distance target whose traveling direction distance from the moving body to the target along the traveling direction of the moving body is less than a distance threshold. a short-distance display area (for example, a short-distance display area R1 described later), and an area located above the short-distance display area along the vertical direction of the screen, and the distance in the traveling direction is equal to or greater than the distance threshold. and a long-distance display area (for example, a long-distance display area R2 described later) in which the target image of a certain long-distance target is displayed, and the display control unit is configured to display a plurality of the target images under the enlarged display mode. When displaying target images on the display screen, it is preferable to determine the display position so that the interval between the two target images displayed in the long-distance display area is larger than in the normal display mode.

(6) In this case, when the first target and the second target are recognized by the surrounding state recognition unit, and the display control unit If the interval along the traveling direction is less than an interval threshold, it is preferable to display the image under the enlarged display mode.

(7) In this case, the virtual image display device further includes a line-of-sight direction acquisition unit (for example, a line-of-sight direction acquisition unit 7 described below) that acquires a line-of-sight direction that is the driver's line of sight or a face direction, and the virtual image display device It is preferable that the control unit displays an image under the enlarged display mode when the target object is present in the line-of-sight direction.

(8) In this case, it is preferable that the display control unit switches between image display under the normal display mode and image display under the enlarged display mode based on the surrounding state information.

(9) In this case, the surrounding state recognition unit recognizes a traffic participant present on the traveling direction side from the moving body as the target object based on the surrounding state information, and the display control unit Based on the state information, if a predetermined number or more of the targets are recognized within a predetermined time, or if an area where the density of the targets is equal to or higher than a predetermined value is recognized, a specific facility or specific When it is recognized that a facility exists, or when it is recognized that a road of a specific type exists on the traveling direction side from the mobile object, it is preferable to display an image under the enlarged display mode.

(1) In the present invention, the surrounding state recognition unit recognizes traffic participants present around the moving body, which is the own vehicle, as targets, and recognizes traffic participants as targets, and recognizes traffic participants that are present around the moving body, which is the own vehicle, as targets, and recognizes traffic participants that are present around the moving body, which is the own vehicle, as targets, and recognizes traffic participants as targets, and Obtain target position information. The display control unit displays the target image at a display position within the display screen determined based on the target position information. The display control unit also displays an image in a normal display mode that determines the display position based on target position information so that the driver can recognize the relative position between the moving object and the target, and displays two images on the display screen. It is possible to switch between image display under an enlarged display mode in which the display position is determined so that the interval between two target images is larger than in the normal display mode. Therefore, according to the present invention, by displaying an image under such an enlarged display mode, it is possible to display images even when two or more targets are closely located within a range with close relative positions from the host vehicle. Even if there are targets, these target images are displayed at intervals on the display screen, so the driver can distinguish and recognize the existence of individual targets, improving traffic safety. Can be done.

(2) In the present invention, when displaying a plurality of target images on the display screen under the enlarged display mode, the display control unit displays target-related information associated with each target in the vicinity of the target image. Display. That is, in the present invention, in the enlarged display mode, the distance between the target images of two targets on the display screen is displayed farther apart than in the normal display mode, and the target related information is displayed in the vicinity. Since it is possible to display a large amount of information without making the viewer feel cluttered, traffic safety can be improved.

(3) In the present invention, the target object related information includes at least one of position information, movement direction information, and type information of the target object. Therefore, according to the present invention, it is possible to quickly recognize information such as the position, moving direction, moving speed, and type of the target that is difficult to grasp only from the target image.

(4) In the present invention, the display control unit displays the target image on the display screen in a manner that creates a sense of perspective. In addition, when displaying an image under the enlarged display mode, the display control unit determines the vertical interval between the two target images along the vertical direction of the screen (that is, the distance between the two target images along the traveling direction). and the display position is set so that either or both of the horizontal distance between the two target images along the horizontal direction of the screen (i.e., the distance along the width direction between the two targets) is larger than in the normal display mode. stipulate. Therefore, according to the present invention, the interval between target images on the display screen can be made wider than in the normal display mode while not impairing the sense of perspective.

(5) In the present invention, the display control unit displays the target image on the display screen in a manner that creates a sense of perspective, and also changes the display screen to a short-distance display area where the target image of the short-distance target is displayed. and a long-distance display area, which is an area above the short-distance display area along the vertical direction of the screen and in which a target image of a long-distance target is displayed. In this case, the display position of the target image of the long-distance target displayed in the long-distance display area converges to a horizontal line virtually defined within the display screen as the distance from the own vehicle along the traveling direction increases. Therefore, the two target images tend to overlap. Therefore, in the present invention, when displaying a plurality of target images on the display screen under the enlarged display mode, the display control unit may set the interval between the two target images displayed in the long-distance display area to be larger than that in the normal display mode. Set the display position so that the size of the image is also large. Therefore, according to the present invention, it is possible to distinguish and make the driver recognize the presence of a plurality of long-distance targets that exist in positions where target images would overlap in the normal display mode.

(6) In the present invention, when the first and second targets are recognized, and the interval between the first and second targets along the traveling direction is equal to or less than the interval threshold If so, display the image under enlarged display mode. As a result, if two target images overlap in the normal display mode, it is possible to prevent these two target images from overlapping by displaying the images in the enlarged display mode. This makes it possible to distinguish the presence of individual targets and make the driver aware of them.

(7) In the present invention, the line-of-sight direction acquisition unit acquires the line-of-sight direction, which is the direction of the driver's line of sight or face, and the display control unit detects the direction of the driver's line-of-sight when there is a target on the line-of-sight direction, that is, when the driver When gazing at a target, the image is displayed in enlarged display mode. This allows the driver to distinguish and recognize a plurality of targets present in the direction in which the driver is gazing.

(8) In the present invention, the display control section switches between image display under normal display mode and image display under enlarged display mode based on surrounding state information regarding the surrounding state of the host vehicle. Therefore, according to the present invention, it is possible to switch between the normal display mode and the enlarged display mode at appropriate timing according to changes in the surrounding conditions of the own vehicle.

(9) In the present invention, based on the surrounding state information, the display control unit automatically controls the If it is recognized that a specific facility or specific equipment exists in the direction of travel from the vehicle, or if it is recognized that a road of a specific type exists in the direction of travel from the own vehicle, the image will be displayed in the enlarged display mode. I do. Therefore, according to the present invention, when there are already many targets in the direction of travel, or when it is predicted that many targets will appear in the direction of travel, images can be displayed in the enlarged display mode. This allows the driver to distinguish and recognize these plurality of targets.

1 is a diagram schematically showing the configuration of a virtual image display device according to an embodiment of the present invention and a vehicle equipped with this virtual image display device. FIG. 2 is a plan view of a vehicle traveling on a road with three lanes on each side, viewed from above. FIG. 3 is a diagram illustrating an example of a virtual image displayed on a display screen when the display control unit displays an image under normal display mode in the situation illustrated in FIG. 2; FIG. 3 is a diagram illustrating an example of a virtual image displayed on a display screen when the display control unit displays an image in an enlarged display mode in the situation illustrated in FIG. 2; 3 is a flowchart showing a procedure for displaying a virtual image in a display control unit.

Hereinafter, a virtual image display device according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram schematically showing the configuration of a virtual image display device 1 according to the present embodiment and a vehicle V in which this virtual image display device 1 is mounted.

The virtual image display device 1 recognizes, as targets, traffic participants existing around a moving object, particularly those existing in the forward direction of the moving object from a viewpoint determined with the moving object as a reference, and displays this object. By displaying a target image associated with a target object on the display screen 51 of a display 55 provided at a position where the driver of the moving object can see it, the driver can recognize the target objects around the moving object. It is intended to support. In addition, although the case where the virtual image display apparatus 1 is mounted on the vehicle V which is a four-wheeled vehicle is demonstrated below, this invention is not limited to this. In the present invention, the virtual image display device may be mounted on a moving object other than a four-wheeled vehicle such as a truck, a saddle-type vehicle, or a bicycle. Furthermore, when mounted on a moving object, particularly a straddle-type vehicle, the virtual image display device may be mounted on a helmet worn by a rider as a user.

The vehicle V includes an external sensor unit 2 that acquires information ahead in the direction of travel, a navigation device 3, an indoor camera unit 4, and a virtual sensor unit that displays a virtual image on a display screen 51 to support safe driving by the driver. An image display device 1 is provided.

The external sensor unit 2 includes a front camera unit 21, a front radar unit 22, and the like.

The front camera unit 21 includes, for example, a digital camera using a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The front camera unit 21 is mounted, for example, on the roof of the vehicle V at a position on the interior side of the vehicle interior near the front window, with the front camera unit 21 facing forward in the direction of travel. The forward radar unit 22 includes a millimeter wave radar that detects an object by measuring reflected waves from the object in response to millimeter wave irradiation. The front radar unit 22 is attached to, for example, the front bumper of the vehicle V, facing forward in the direction of travel.

The navigation device 3 includes, for example, a GNSS (Global Navigation Satellite System) receiver that identifies the current position of the vehicle based on signals received from satellites, a storage device that stores map information, and the like. Here, the map information also includes information regarding road signs. The navigation device 3 transmits information regarding the current location of the own vehicle to the virtual image display device 1 together with map information of the current location.

The indoor camera unit 4 includes, for example, a digital camera using a solid-state image sensor as described above. This indoor camera unit 4 is provided in the vehicle interior so as to face the driver seated in the driver's seat. The image of the driver taken by the indoor camera unit 4 is transmitted to the virtual image display device 1.

The virtual image display device 1 acquires surrounding state information regarding the state of the surroundings of the vehicle V obtained by a display 5 provided at a position visible to the driver while driving, and an external sensor unit 2 and a navigation device 3. , a surrounding state recognition unit 6 that recognizes the surrounding state of the vehicle V, and a line-of-sight direction acquisition unit 7 that obtains the line-of-sight direction, which is the direction of the driver's line of sight or face, based on the image transmitted from the indoor camera unit 4. , a display control unit 8 that displays, on a display screen 51 of the display 5, a virtual image related to the forward direction of the vehicle V generated based on the recognition result by the surrounding state recognition unit 6.

In this embodiment, a case will be described in which the display control unit 8 displays a virtual image on the display screen 51 of the display 5 provided in front of the driver's seat, but the present invention is not limited to this. The display control unit 8 may display a virtual image using a part of the windshield as a display screen by using a head-up display.

The surrounding state recognition unit 6 detects one or more traffic participants present in the forward direction of the vehicle V from a predetermined viewpoint with respect to the vehicle V based on the surrounding state information acquired from the external sensor unit 2 and the navigation device 3. In addition to recognizing each target as a target, the target position information and target related information of each target are acquired. The target position information also includes information regarding the relative position between the viewpoint and each target. More specifically, the target position information includes information about the traveling direction distance from the viewpoint to each target along the traveling direction and the width direction distance from the viewpoint to each target along the width direction perpendicular to the traveling direction. Contains information about. Further, the target object related information includes at least one of position information, moving direction information, moving speed information, and type information of the target object.

Here, the viewpoint determined for the vehicle V is, for example, the interior of the vehicle V (more specifically, the driver's seat) or the exterior of the vehicle V (more specifically, from the rear end of the vehicle V in the direction of travel). (a predetermined distance away from the rear). If the viewpoint is set at the driver's seat, a virtual image from the same viewpoint as the driver can be displayed. Furthermore, when the viewpoint is set at a position away from the rear end of the vehicle V to the rear side, it is possible to display an overhead virtual image that includes the own vehicle in the field of view.

The surrounding state recognition unit 6 acquires the target object position information and target object related information as described above, and also recognizes the vehicle V and the recognized object based on the surrounding state information obtained from the external sensor unit 2 and the navigation device 3. Obtain traffic environment information of traffic participants including traffic signs. Here, the traffic environment information includes information regarding the number of lanes on the road on which the vehicle is traveling, the presence or absence of sidewalks, the types of facilities and equipment that exist in the front direction of travel, the form of the road that exists in the forward direction of travel, and the like.

In this embodiment, a case will be described in which the surrounding state recognition unit 6 acquires surrounding state information from the external sensor unit 2 and the navigation device 3 as in-vehicle devices that move with the vehicle V, but the present invention is not limited to this. . The surrounding condition recognition unit 6 performs vehicle-to-vehicle communication between the own vehicle and other vehicles traveling in the vicinity thereof, and communication between the own vehicle and a server device that monitors the traffic condition of the road on which the own vehicle is traveling. Ambient state information may be acquired by, for example.

The display control unit 8 generates a virtual image that reproduces the appearance seen from the viewpoint forward in the direction of travel based on the recognition result by the surrounding state recognition unit 6, and displays this virtual image on the display screen 51 of the display 5. . More specifically, the display control unit 8 converts the target image associated with each target recognized by the surrounding state recognition unit 6 into a background image according to the traffic environment recognized by the surrounding state recognition unit 6. By overlapping the drawings at a predetermined position and a predetermined size, a virtual image is generated in a manner that gives a sense of perspective to the viewer.

The display control unit 8 is capable of generating a virtual image under normal display mode or enlarged display mode and displaying this virtual image. Below, the procedure for generating a virtual image under each display mode will be explained using the situation shown in FIG. 2 as an example.

FIG. 2 is a plan view of a vehicle V traveling on a road 9 with three lanes on each side, viewed from above. FIG. 2 shows a case where a vehicle V, which is the own vehicle, is traveling in a center lane 91, and another vehicle 93 and a pedestrian 94 are present in front of the vehicle in the direction of travel. Further, FIG. 2 shows a case where another vehicle 93 is stopped in a lane 92 on the sidewalk side, and a pedestrian 94 is about to enter the lane 92 from the sidewalk side in front of the other vehicle 93 in the traveling direction. Further, in the following description, a viewpoint 95 is set on the rear side in the traveling direction of the vehicle V, and the surrounding state recognition unit 6 recognizes another vehicle 93 as a first target and a pedestrian 94 as a second target. Let me explain the case. Therefore, in FIG. 2, "y1" indicates the distance in the traveling direction of the first target, "y2" indicates the distance in the traveling direction of the second target, "x1" indicates the distance in the width direction of the first target, “x2” indicates the distance in the width direction of the second target.

<Normal display mode>
FIG. 3 is a diagram showing an example of a virtual image displayed on the rectangular display screen 51 when the display control unit 8 displays an image in the normal display mode in the situation shown in FIG. As shown in FIG. 3, the virtual image includes, for example, a background image Ib imitating the marking lines of the road on which the vehicle is traveling, an own vehicle icon image Iv associated with the vehicle V, and images associated with each target object 93, 94. It is composed of target icon images I1 and I2. In addition, below, the own vehicle icon image Iv and the target object icon image I1 associated with the target object that is a vehicle are simplified with square marks, and the target object icon image I2 associated with the target object that is a pedestrian is indicated with a circle mark. Simplify.

When displaying an image in the normal display mode, the display control unit 8 displays the host vehicle icon image Iv and the target object icon image on the background image Ib generated according to the traffic environment recognized by the surrounding state recognition unit 6. I1 and I2 are displayed at such a position and size that a viewer can recognize the relative position between the own vehicle and each target object and can create a sense of perspective. Note that although a case will be described below in which only road marking lines are displayed as a background image, the present invention is not limited to this. In addition to road marking lines, structures such as facilities and equipment existing on the road may be displayed as the background image, or a lattice-like grid may be displayed as the background image.

When displaying images in the normal display mode, the display control unit 8 determines the size of each icon image Iv, I1, I2 on the display screen 51 as the distance from the viewpoint to the target object along the traveling direction. The longer the distance in the traveling direction, the smaller it becomes.

When displaying an image under the normal display mode, the display control unit 8 changes the display position of each icon image Iv, I1, I2 on the display screen 51 so that the longer the distance in the width direction from the viewpoint to the target is, the more the display screen changes. The horizontal reference line L1, which is virtually defined within 51, is moved away from the horizontal reference line L1 along the horizontal direction of the screen, and as the traveling direction distance increases, the horizontal reference line L1 is moved closer to the horizontal reference line L1 along the horizontal direction of the screen. Furthermore, the display control unit 8 displays a target existing on the right hand side when viewed from the viewpoint along the traveling direction on the right hand side from the horizontal reference line L1 in the display screen 51, and displays the target on the right hand side when viewed from the viewpoint along the traveling direction. A target existing on the left hand side is displayed on the left hand side from the horizontal reference line L1 within the display screen 51. Note that although a case will be described below in which the horizontal reference line L1 extending along the vertical direction of the screen is set at the center of the display screen 51, the present invention is not limited to this.

In addition, when displaying images under the normal display mode, the display control unit 8 changes the display position of each icon image Iv, I1, I2 on the display screen 51 so that the longer the distance in the traveling direction from the viewpoint to the target object is, The screen is moved away from the screen bottom line L2 of the display screen 51 along the screen vertical direction perpendicular to the screen horizontal direction.

When displaying an image under the normal display mode, the display control unit 8 sets the display position and size of each icon image Iv, I1, I2 on the display screen 51 according to the procedure described above, thereby controlling the viewing angle. It is possible to display a virtual image that allows a person to recognize the relative position between his or her own vehicle and each target object and creates a sense of perspective. However, in the virtual image displayed on the display screen 51 under the normal display mode, the target icon images I1 and I2 associated with the two targets 93 and 94 that are located far away from the viewpoint along the direction of travel are As shown in FIG. 3, there are many cases in which the target objects 93 and 94 overlap, so it may be difficult for the driver to distinguish and recognize the plurality of targets 93 and 94 that are located far away from the viewpoint.

<Enlarged display mode>
FIG. 4 is a diagram showing an example of a virtual image displayed on the display screen 51 when the display control unit 8 displays an image in the enlarged display mode in the situation shown in FIG.

As shown in FIG. 4, when displaying an image in the enlarged display mode, the display control unit 8 displays an own vehicle icon on the background image Ib generated according to the traffic environment recognized by the surrounding state recognition unit 6. Image Iv and target icon images I1 and I2 are positioned and sized so that the viewer can recognize the relative position between the own vehicle and each target and can create a sense of perspective. Display.

Here, when displaying an image under the enlarged display mode, the display control unit 8 displays at least the target object icon image I1, The display position of I2 on the display screen 51 is changed from the normal display mode. More specifically, when displaying an image under the enlarged display mode, the display control unit 8 determines that the interval between the two target icon images I1 and I2 on the display screen 51 is equal to the distance between the two target icon images I1 and I2 on the display screen under the normal display mode. The display position of each target object icon image I1, I2 is determined so as to be larger than the interval between the two target object icon images I1, I2 displayed on the screen 51.

More specifically, the display control unit 8 determines whether the interval between target icon images on the display screen under the normal display mode is equal to or less than a predetermined distance among the plurality of targets recognized by the surrounding state recognition unit 6. is extracted as an enlargement target target, and both or either of the display positions of the two target icon images associated with the two extracted enlargement target targets are changed from the display position under the normal display mode to each other. Change in the direction away from you. At this time, the display control unit 8 sets the display position of the one of the two extracted targets to be enlarged, which has a shorter distance in the traveling direction from the viewpoint, from the display position determined under the normal display mode along the vertical direction of the screen. The display position that is longer in the traveling direction from the viewpoint is a predetermined distance from the display position determined under the normal display mode from the screen bottom line L2 side along the vertical direction of the screen. Just keep it away. In addition, the display control unit 8 determines that the display position of the extracted target to be enlarged, which has a shorter distance in the width direction from the viewpoint, is set along a horizontal reference line along the horizontal direction of the screen from the display position determined under the normal display mode. The display position that is longer in the width direction from the viewpoint is moved a predetermined distance away from the horizontal reference line L1 side along the horizontal direction of the screen from the display position determined under the normal display mode. Accordingly, when displaying an image under the enlarged display mode, the display control unit 8 determines the vertical interval along the vertical direction of the screen and/or the horizontal interval along the horizontal direction of the screen between the two target icon images. Set the display position so that it is larger than in normal display mode.

When changing the display position of the target to be enlarged from the display position under the normal display mode as described above, the display control unit 8 moves the target icon image closer to the screen bottom line L2 along the vertical direction of the screen. For target icon images whose size is slightly larger than the size under the normal display mode and moved away from the screen bottom line L2 along the vertical direction of the screen, the size is set slightly larger than the size under the normal display mode. The size may be slightly smaller than the size shown in . This makes it possible to increase the distance between the two target icon images without impairing the sense of perspective.

The display control unit 8 displays the image under the enlarged display mode according to the procedure described above, thereby eliminating the overlap between the two target icon images I1 and I2 in the virtual image generated under the normal display mode (Fig. 3) may be resolved by displaying the image in the enlarged display mode as shown in FIG. Therefore, the driver can easily distinguish and recognize the two targets 93 and 94 that are located far away from the viewpoint.

As described above, the display control unit 8 makes it easier to recognize multiple long-distance targets located far from the viewpoint by making the interval between two target icon images larger in the enlarged display mode than in the normal display mode. However, in this case, it may be difficult to grasp the sense of distance of a short-distance target that is located close to the viewpoint. Therefore, the display control unit 8 displays the display screen 51 as a short-distance target whose traveling direction distance from the viewpoint to the target is less than the distance threshold (see FIG. 2), as shown by the virtual dividing line L3 in FIG. A short-distance display area R1 in which a target icon image is displayed, and a long-distance target that is an area above the short-distance display area R1 in the vertical direction of the screen and whose traveling direction distance is equal to or greater than a distance threshold. It is divided into a long-distance display area R2 where a target icon image is displayed. In addition, when displaying a plurality of target icon images under the enlarged display mode, the display control unit 8 causes the interval between the two target icon images displayed in the long-distance display area R2 to be larger than in the normal display mode. The display position is determined as follows, and the display position of the target object icon image displayed within the short distance display area R1 is made equal to the display position under the normal display mode. Note that this distance threshold value may be a fixed value or may be changed depending on the vehicle speed. That is, the distance threshold value may be made longer as the vehicle speed becomes faster.

As described above, when displaying an image under the enlarged display mode, the interval between the two target icon images I1 and I2 is larger than when displaying an image under the normal display mode. Therefore, when displaying a plurality of target icon images I1 and I2 in the enlarged display mode, the display control unit 8 uses the space formed between each target icon image I1 and I2 to display the image in FIG. As shown, target-related information associated with each target may be displayed in the vicinity of each target icon image I1, I2. Note that FIG. 4 shows a case where the movement direction of the second target 94 that is about to enter the roadway from the sidewalk is displayed by an arrow 99 as target related information.

In this embodiment, a case has been described in which the display control unit 8 makes the form of the background image Ib the same in the normal display mode and the enlarged display mode, but the present invention is not limited to this. The forms of these background images may be changed between the normal display mode and the enlarged display mode within a range that does not impair the sense of perspective.

FIG. 5 is a flowchart showing a procedure for displaying a virtual image on the display screen 51 in the virtual image display device 1.

First, in step ST1, the surrounding state recognition unit 6 acquires surrounding state information from the external sensor unit 2 and the navigation device 3, and moves to step ST2.

Next, in step ST2, the surrounding state recognition unit 6 obtains target position information, target related information, and traffic environment information based on the surrounding state information obtained in step ST1, and proceeds to step ST3.

Next, in step ST3, the line-of-sight direction acquisition unit 7 acquires the driver's line-of-sight direction based on the image transmitted from the indoor camera unit 4, and proceeds to step ST4.

Next, in step ST4, the display control unit 8 selects the six digits determined to set the display mode of the virtual image to the enlarged display mode based on the surrounding state information obtained in step ST1 and the viewing direction obtained in step ST3. It is determined whether any one or a combination of the enlarged display conditions (A) to (F) is satisfied. When the determination result in step ST4 is NO, the display control unit 8 moves to step ST5, displays the virtual image generated under the normal display mode on the display screen 51, and returns to step ST1. Further, if the determination result in step ST4 is YES, the display control unit 8 displays the virtual image generated under the enlarged display mode on the display screen 51, and returns to step ST1. As described above, the display control unit 8 switches between image display under the normal display mode and image display under the enlarged display mode based on the surrounding state information.

Enlarged display condition (A): A predetermined number or more of targets are recognized within a predetermined time based on surrounding state information.
Enlarged display condition (B): The existence of an area where the density of targets is equal to or higher than a predetermined value is recognized based on the surrounding state information.
Enlarged display condition (C): Based on surrounding state information, a specific facility (e.g., a station or commercial facility where many people or moving objects come in and out) or a specific facility (e.g., where many people move) is displayed in the direction of travel from the viewpoint. (pedestrian crossings, traffic lights, etc.) must be recognized.
Enlarged display condition (D): The presence of a specific type of road (for example, an intersection, a merging point of two roads, etc.) in the direction of travel from the viewpoint is recognized based on the surrounding state information.
Enlarged display condition (E): It is recognized that a target exists in the line of sight of the driver while driving.
Enlarged display condition (F): A case where the first target and the second target are recognized on the direction direction side from the viewpoint, and the distance between the first target and the second target along the traveling direction. is less than or equal to a predetermined interval threshold.

According to the virtual image display device 1 according to this embodiment, the following effects are achieved.
(1) The surrounding state recognition unit 6 recognizes traffic participants existing around the vehicle V, which is the host vehicle, as a target object, and the target position information includes information regarding the relative position between the host vehicle and the target object. get. The display control unit 8 displays the target icon image at a display position on the display screen 51 determined based on the target position information. The display control unit 8 also displays an image in a normal display mode that determines the display position based on target position information so that the driver can recognize the relative position between the vehicle V and the target, and on the display screen 51. It is possible to switch between image display under an enlarged display mode in which the display position is determined so that the interval between two target icon images is larger than in the normal display mode. Therefore, according to the virtual image display device 1, by displaying an image under such an enlarged display mode, it is possible to display a plurality of two or more targets densely located within a range whose relative positions are close to the own vehicle. Even when the target objects are displayed on the display screen 51, the target images are displayed at intervals, so that the driver can distinguish and recognize the existence of individual targets, thereby improving traffic safety. can be improved.

(2) In the present invention, when displaying a plurality of target images on the display screen under the enlarged display mode, the display control unit displays target-related information associated with each target in the vicinity of the target image. Display. That is, in the present invention, in the enlarged display mode, the distance between the target images of two targets on the display screen is displayed farther apart than in the normal display mode, and the target related information is displayed in the vicinity. Since it is possible to display a large amount of information without making the viewer feel cluttered, traffic safety can be improved.

(3) Target-related information includes at least one of position information, movement direction information, and type information of the target. Therefore, according to the virtual image display device 1, it is possible to quickly recognize information that is difficult to grasp only from the target object image, such as the position, moving direction, moving speed, and type of the target object.

(4) The display control unit 8 displays the target icon image on the display screen 51 in a manner that creates a sense of perspective. In addition, when displaying an image under the enlarged display mode, the display control unit 8 controls the vertical interval between the two target icon images along the vertical direction of the screen (i.e., the distance between the two target icon images along the traveling direction). distance) and the horizontal distance between the two target icon images along the horizontal direction of the screen (i.e., the distance along the width direction between the two targets) so that both or either of them is larger than in the normal display mode. Determine the display position. Therefore, according to the virtual image display device 1, the interval between the target object icon images on the display screen 51 can be made wider than in the normal display mode without impairing the sense of perspective.

(5) In the virtual image display device 1, the display control unit 8 displays the target object icon image on the display screen 51 in a manner that creates a sense of perspective, and displays the target object icon image of a short-distance target on the display screen 51. a short-distance display area R1 in which is displayed, and a long-distance display area R2, which is an area above the short-distance display area R1 along the vertical direction of the screen and in which a target icon image of a long-distance target is displayed. Divide into. In this case, the display position of the target icon image of the long-distance target displayed in the long-distance display area R2 is a horizontal line that is virtually determined within the display screen 51 as the distance from the own vehicle along the traveling direction increases. , the two target icon images tend to overlap. Therefore, in the virtual image display device 1, when displaying a plurality of target icon images on the display screen 51 under the enlarged display mode, the display control unit 8 controls two target icon images displayed in the long distance display area R2. The display position is determined so that the interval between images is larger than in the normal display mode. Therefore, according to the virtual image display device 1, the presence of a plurality of long-distance targets that exist in positions where target icon images would overlap under the normal display mode can be distinguished and recognized by the driver. .

(6) In the virtual image display device 1, the display control unit 8 controls when the first and second targets are recognized and the interval between the first and second targets in the traveling direction is an interval. If it is below the threshold, the image is displayed in enlarged display mode. As a result, if two target icon images overlap under normal display mode, displaying the images under enlarged display mode will prevent these two target icon images from overlapping. Therefore, the existence of individual targets can be distinguished and recognized by the driver.

(7) In the virtual image display device 1, the line-of-sight direction acquisition unit 7 acquires the line-of-sight direction, which is the direction of the driver's line of sight or face, and the display control unit 8, when there is a target on the line-of-sight direction, That is, when the driver is gazing at a target object, the image is displayed in the enlarged display mode. This allows the driver to distinguish and recognize a plurality of targets present in the direction in which the driver is gazing.

(8) In the virtual image display device 1, the display control unit 8 controls image display under the normal display mode and image display under the enlarged display mode based on surrounding state information regarding the surrounding state of the host vehicle. Switch. Therefore, according to the virtual image display device 1, it is possible to switch between the normal display mode and the enlarged display mode at appropriate timing according to changes in the surrounding conditions of the own vehicle.

(9) In the virtual image display device 1, if a predetermined number or more of targets are recognized within a predetermined time based on the surrounding state information, the display control unit 8 recognizes an area where the density of targets is equal to or higher than a predetermined value. If you recognize that a specific facility or equipment exists in the direction of travel from your vehicle, or if you recognize that a road of a specific type exists in the direction of travel from your vehicle, the Display the image in . Therefore, according to the virtual image display device 1, when there are already many targets in the direction of travel, or when it is predicted that many targets will appear in the direction of travel, the image display in the enlarged display mode is By doing so, the driver can distinguish and recognize these plurality of targets.

Although one embodiment of the present invention has been described above, the present invention is not limited to this. The detailed structure may be changed as appropriate within the spirit of the present invention.

For example, in the embodiment described above, a case has been described in which the display control unit 8 displays a virtual image on the display screen 51 in a manner that gives a sense of perspective to the viewer, but the present invention is not limited to this. The display control unit 8 displays a virtual image such as an overhead image or a planar image that does not create a sense of perspective to the viewer, but allows the viewer to recognize the relative position between the vehicle and the target. It's okay.

V... Vehicle 1... Virtual image display device 2... External sensor unit 21... Front camera unit 22... Front radar unit 3... Navigation device 4... Indoor camera unit 5... Display 51... Display screen 6... Surrounding state recognition unit 7... Line of sight direction Acquisition unit 8...Display control unit L1...Horizontal reference line L2...Screen bottom line I1, I2...Target icon image (target image)

Claims (9)

A virtual image display device that recognizes traffic participants existing around a moving object as a target and displays a target image related to the target on a display screen that is visible to a driver of the moving object,
a surrounding state recognition unit that acquires surrounding state information regarding the surrounding state of the moving object, and obtains target position information including information regarding a relative position between the moving object and the target object based on the surrounding state information; and,
a display control unit that displays the target image at a display position within the display screen determined based on the target position information,
The display control section includes:
displaying an image under a normal display mode that determines the display position based on the target position information so that the driver can recognize the relative position between the moving object and the target;
Image display under an enlarged display mode in which the display position is determined such that the interval between the two target images on the display screen is larger than in the normal display mode, and image display is switchable. Virtual image display device. When displaying the plurality of target images on the display screen under the enlarged display mode, the display control unit displays target related information associated with each target in the vicinity of the target image. The virtual image display device according to claim 1, characterized in that: The virtual image display device according to claim 2, wherein the target object-related information includes at least one of position information, moving direction information, moving speed information, and type information of the target object. The display control unit displays the target image in a manner that creates a sense of perspective on the display screen, and when displaying the image under the enlarged display mode, displays the target image in a vertical direction of the screen of the two target images. The virtual image display device according to claim 3, wherein the display position is determined so that both or either of a vertical interval along the screen and a horizontal interval along the horizontal direction of the screen is larger than in the normal display mode. . The display screen is a short-distance display area in which the target image of a short-distance target whose traveling direction distance from the moving body to the target along the traveling direction of the mobile body is less than a distance threshold value is displayed. and a long-distance display in which the target image of a long-distance target, which is an area above the short-distance display area along the vertical direction of the screen and whose traveling direction distance is equal to or greater than the distance threshold value, is displayed. divided into areas,
When displaying the plurality of target images on the display screen under the enlarged display mode, the display control unit may adjust the interval between the two target images displayed within the long-distance display area to the normal display. 5. The virtual image display device according to claim 4, wherein the display position is determined to be larger than a mode. When the first target and the second target are recognized by the surrounding state recognition unit, the display control unit is configured to display information along the traveling direction between the first target and the second target. 5. The virtual image display device according to claim 4, wherein the virtual image display device displays an image under the enlarged display mode when the interval is less than an interval threshold. further comprising a line-of-sight direction acquisition unit that acquires a line-of-sight direction that is the direction of the driver's line of sight or face;
The virtual image display device according to claim 4, wherein the display control unit displays the image under the enlarged display mode when the target object is present in the line of sight direction. The virtual image according to claim 4, wherein the display control unit switches between image display under the normal display mode and image display under the enlarged display mode based on the surrounding state information. Display device. The surrounding state recognition unit recognizes a traffic participant present on the traveling direction side from the moving object as the target object, based on the surrounding state information,
Based on the surrounding state information, if the display control unit recognizes a predetermined number or more of the targets within a predetermined time, or if it recognizes an area where the density of the targets is equal to or higher than a predetermined value, If it is recognized that a specific facility or specific equipment exists in the direction of travel, or if it is recognized that a road of a specific type exists in the direction of travel from the moving object, the image under the enlarged display mode The virtual image display device according to claim 8, wherein the virtual image display device performs display.

Images