JP7166711B2 - display system - Google Patents

Description

The present invention relates to a display system using a head-up display, and more particularly to a system that displays virtual markers for guiding or guiding a vehicle.

2. Description of the Related Art In recent years, head-up displays (hereinafter referred to as HUDs) have begun to spread as a technology for displaying information for assisting drivers. A typical HUD projects a marker image from a projection device onto the windshield or a built-in screen to guide the vehicle's course, and displays it as a virtual image on the real space scenery in front of the vehicle that the driver sees. . As a result, the driver can visually recognize the driving support information in the line-of-sight direction, so that the line-of-sight movement of the driver can be reduced compared to when viewing display equipment such as a display installed in the vehicle.

As a technology related to the HUD, there is a technique that displays an image of a virtual vehicle of the host vehicle to visually notify the driver of the route to be taken using the image of the virtual vehicle when providing route guidance (Patent Document 1). Also disclosed is a technique for moving a virtual marker to a position that does not overlap with the obstacle when an obstacle is detected at the position of the displayed virtual marker (Patent Document 2).

JP 2016-182891 A JP 2016-118423 A

A conventional problem when displaying a virtual marker by HUD will be described. When the virtual vehicle is displayed on the windshield, for example, as shown in FIG. 11A, a virtual marker D having the shape of the vehicle is displayed superimposed on the real scene by projection. This virtual marker D is used to display directions of travel and lane directions to the host vehicle in conjunction with the navigation function. However, there is an obstacle such as another vehicle in front of the vehicle, and when the vehicle approaches the obstacle, the driver sees the virtual marker D overlapping the obstacle. A problem arises in that it becomes difficult to recognize D. In this case, if the virtual marker D is an arrow or the like, the problem can be solved by moving the display position of the virtual image to a position avoiding obstacles. 11(B), the virtual vehicle is moved to the side as shown in FIG. can no longer be displayed.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a display system capable of accurately displaying virtual markers to the driver.

A display system according to the present invention includes projection means for projecting an image for a virtual marker in front of a vehicle for guiding a direction of travel and the like, detection means for detecting obstacles in front of the vehicle, and the detection means. measuring means for measuring a vehicle-to-vehicle distance between the obstacle and the vehicle; and control means for controlling projection of the means, wherein the control means switches the image for the virtual marker to another image when the inter-vehicle distance becomes equal to or less than a threshold.

In one embodiment, the other image is an image in which an upper portion of the image of the virtual marker is deleted. In one embodiment, there are a plurality of thresholds, and the control means switches to another image so that the loss from top to bottom increases stepwise each time the inter-vehicle distance falls below each threshold. In one embodiment, the control means switches to another image when the position where the virtual image of the image for the virtual marker is displayed overlaps with the position of the obstacle. In one embodiment, the control means completely hides the virtual image of the image for the virtual marker when the inter-vehicle distance becomes smaller than a limit value. In one embodiment, the image for the virtual marker is a virtual guided vehicle imitating the shape of a vehicle, and the control means gradually reduces the loss from the top to the bottom of the virtual guided vehicle as the inter-vehicle distance decreases. , and when the inter-vehicle distance reaches a certain threshold value or less, the image is switched to an image of only the rear tire portion of the virtual guided vehicle. In one embodiment, the control means switches the images step by step so that the lost portion of the virtual guided vehicle gradually becomes smaller as the inter-vehicle distance increases. In one embodiment, the display system further includes storage means for storing images for a plurality of virtual markers, and navigation means for providing route guidance, and the control means is linked to the navigation means to control the storage means. switch the image of the virtual marker read from . In one embodiment, the control means projects an image of the virtual marker including instruction information for changing the direction of travel.

According to the present invention, since the projection of the virtual marker image is controlled according to the inter-vehicle distance, it is possible to ensure good visibility of the virtual marker image. In addition, when the projection position of the image for the virtual marker overlaps with an obstacle such as a forward vehicle, the overlapping portion is deleted, so that the visibility of the image for the virtual marker is further improved.

1 is a block diagram showing the configuration of a display system according to the present invention; FIG. FIG. 3 is a schematic diagram of a configuration of a HUD according to an embodiment of the present invention; It is a figure which shows an example of a structure of HUD which concerns on the Example of this invention. 3 is a diagram showing a functional configuration of an image projection program according to an embodiment of the present invention; FIG. FIG. 10 is a diagram showing a display example of virtual markers on the HUD according to the embodiment of the present invention; It is a figure explaining the display area of the virtual image of the virtual marker by the inter-vehicle distance. 4 is a flow chart explaining the operation of the display system according to the embodiment of the present invention; FIG. 4 is a side view schematically showing the inter-vehicle distance and display positions of virtual markers; It is a figure which shows the example of a display in each inter-vehicle distance. It is a figure which shows the example of a display in each inter-vehicle distance. It is a figure which shows the example of a display in each inter-vehicle distance. FIG. 11 is an example of displaying virtual markers according to another embodiment of the present invention; FIG. It is a figure explaining the subject when displaying the virtual marker of a prior art.

A display system according to the present invention includes an image projection device typified by a HUD, and has a function of projecting or irradiating various images onto the windshield of a vehicle or its screen by means of the HUD. The driver can visually recognize a virtual image of the image projected on the windshield by the HUD in the line-of-sight direction.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the overall system configuration of a display system 10 according to an embodiment of the invention. FIG. 2 is a diagram schematically showing the configuration of the HUD. The display system 10 includes an imaging unit 100 , a radar unit 110 , a storage unit 120 , a navigation device 130 , an image processing unit 140 , a HUD 150 and a control unit 160 .

The image capturing unit 100 captures an image of at least the front of the vehicle. Like the surveillance camera 101 in FIG. 2, it is mounted near the front inside the vehicle. Depending on the embodiment, it may be attached to the exterior of the vehicle. Image data captured by the imaging unit 100 is output to the control unit 160 . The image data acquired by the imaging unit 100 is used for detecting an obstacle such as a forward vehicle and detecting the size of the obstacle. Also, the imaging unit 100 may be a stereo camera, in which case the distance to an obstacle such as a forward vehicle can be measured.

Further, the imaging unit 100 can include an imaging camera that is mounted inside the vehicle and captures an image of the vehicle interior space. In this embodiment, the driver's face is photographed and used to detect the line-of-sight direction or face orientation. Therefore, the mounting position is provided in the vicinity of the steering wheel like the internal camera 102 in FIG. 2 or in the instrument panel.

The radar unit 110 monitors at least the area ahead of the vehicle, and detects the position of obstacles ahead, the distance between the vehicles, and the relative speed. The radar unit 110 is an infrared laser, a millimeter wave radar, or the like. Note that when a stereo camera is mounted as the imaging unit 100, the radar unit 110 may not be mounted.

The storage unit 120 stores various information necessary for the display system 10 . In one embodiment, storage unit 120 includes map data 122 and image data 124 . Also, programs for the display system 10 to execute can be stored. The map data 122 stores link data, node data, and the like. Link data is road information connecting nodes, and includes general road and expressway types, coordinates of start and end points of links, directions of links, and the like. The node data is information representing the positional coordinates of intersections and branch points, the attributes of intersections, and the like.

Storage unit 120 further stores image data 124 for virtual markers projected by HUD 150 . Depending on the embodiment, image data for various virtual markers having different display positions and sizes may be stored for changing the display form of the virtual markers according to the inter-vehicle distance from the preceding vehicle. good. Furthermore, the image data for the virtual marker can be image data defined by three-dimensional coordinates as well as image data defined by two-dimensional coordinates. The image data for this virtual marker includes at least a two-dimensional image representing the rear portion of the vehicle and a three-dimensional image representing the entire vehicle from the rear portion to the front portion of the vehicle.

When the user sets a destination, the navigation device 130 detects the position of the vehicle based on positioning using GPS signals or positioning using an autonomous navigation sensor, and calculates a route from the position of the vehicle to the destination. It has a function of searching with reference to the map data 122 and the like and providing route guidance based on the searched route. In this embodiment, route guidance for a route searched by the navigation device 130 is projected by the HUD 150 and presented to the driver. In addition, the display by the navigation device 130 may be displayed from a display device such as a liquid crystal provided separately from the HUD 150 .

Image processing unit 140 reads image data 124 from storage unit 120 and provides it to HUD 150 . Furthermore, the image processing unit 140 can process the image data to be projected according to user input or instructions from the control unit 160 . In one embodiment, the image processing unit 140 generates an image in which information related to direction indication such as a blinker is added to the virtual marker from the route guidance information such as a change in the traveling direction acquired from the navigation device 130, and the image is projected from the HUD 150. is also possible.

Next, the HUD 150 projects an image onto the windshield 25 or screen as shown in FIG. 2 to display the projected image in the line of sight of the driver. By projecting the image onto the windshield 25 or the screen, the driver can visually recognize the projected image as if it were superimposed on the scenery of the real space in front of the vehicle. FIG. 3 is a diagram showing an example of the configuration of the HUD 150. As shown in FIG. HUD 150 includes light source 151 , optical system 152 , light modulating means 153 , projection optical system 154 and actuator 155 . The light source 151 is composed of a halogen lamp, a light emitting diode array, a laser diode array, or the like. Optical system 152 converges light from light source 151 and provides light of a predetermined aspect ratio to light modulating means 153 .

The light modulating means 153 is composed of, for example, a liquid crystal device or a digital mirror device in which a plurality of variable mirrors are two-dimensionally arranged. Also, the light modulating means 153 spatially modulates the light incident from the optical system 152 based on the image data provided from the control section 160 . A projection optical system 154 projects the light modulated by the light modulating means 153 onto the windshield 25 or its screen. The projection optical system 154 uses, for example, optical members such as lenses and concave mirrors. The actuator 155 adjusts the optical distance by, for example, adjusting the position of the lens and the angle of the mirror of the projection optical system 154 . In one embodiment, the actuator 155 varies the focal length and optical axis of the projection optical system 154 based on control signals from the controller 160 . This makes it possible to change the position and size of the virtual image of the projected image viewed by the driver. It should be noted that the display system 10 may have a plurality of HUDs 150 and display the projection image of each HUD 150 .

The control unit 160 includes a microcontroller or microprocessor having ROM/RAM, etc., executes predetermined processing based on control programs stored in the ROM and RAM, and controls the operation of each unit of the display system 10. . The control program according to this embodiment includes an image projection program 200 .

FIG. 4 is a diagram showing the functional configuration of the image projection program 200 according to the embodiment of the invention. The image projection program 200 includes an obstacle detection section 210 , an inter-vehicle distance measurement section 220 , a display position determination section 230 , a display size determination section 240 , a projection control section 250 , a threshold determination section 260 and a projection image switching section 270 .

The obstacle detection unit 210 detects an obstacle such as a vehicle in front of the host vehicle based on the image data captured by the imaging unit 100 and the detection result of the radar unit 110 . In one embodiment, the size of the obstacle is determined based on the captured image data, and if it is a vehicle, the type of vehicle (eg, truck, passenger car) is determined.

Inter-vehicle distance measurement unit 220 measures the inter-vehicle distance between the obstacle and the host vehicle in response to detection of the obstacle by obstacle detection unit 210 . The following description will be made by assuming that the obstacle according to the present embodiment is a forward vehicle. For example, as shown in FIG. 5, when a forward vehicle X is detected, a linear inter-vehicle distance L from the host vehicle to the forward vehicle X is measured. The inter-vehicle distance L can be calculated by the radar unit 110 . In one embodiment, when a stereo camera is used for the imaging unit 100, the inter-vehicle distance is measured from the imaging unit 100. FIG.

The display position determining unit 230 determines the display position of the virtual marker based on the inter-vehicle distance measured by the inter-vehicle distance measuring unit 220 . The display position of the virtual marker is the projection position where the driver can see the virtual image of the virtual marker projected on the windshield in the direction of the line of sight in real space, as shown in FIGS. , and the position (generation distance P) at which the virtual image D of the virtual marker is generated. The display position determining unit 230 shifts the display position of the virtual marker forward so that the virtual marker does not overlap with the vehicle ahead when the inter-vehicle distance becomes shorter due to the approach of the vehicle in front. The display position of the virtual marker is determined so as to shift the display position of .

Any method can be used to vary the display position of the virtual marker, but in one embodiment, a plurality of image data for the virtual marker are prepared according to the inter-vehicle distance, and image data corresponding to the inter-vehicle distance is selected from among the image data. select. Each image data depicts a virtual marker image at a predetermined coordinate position in each frame. is drawn, and if the image data is for a long inter-vehicle distance, the image of the virtual marker is drawn at the coordinate position on the far side of the frame. In another embodiment, the control unit 160 controls the projection angle and focal length of the projection optical system 154 via the actuator 155 based on the inter-vehicle distance, thereby changing the display position of the virtual marker. may

A display size determination unit 240 determines the size of the virtual marker according to the inter-vehicle distance. As the vehicle-to-vehicle distance becomes shorter, the display size determining unit 240 increases the size of the virtual marker or, conversely, increases the vehicle-to-vehicle distance because the vehicle in front appears to the driver to be closer and larger. As a result, the vehicle ahead appears farther and smaller for the driver, so the size of the virtual marker is reduced accordingly. Preferably, the display size is controlled so that the size of the virtual marker is increased stepwise as the inter-vehicle distance becomes shorter, and the size of the virtual marker is gradually reduced as the inter-vehicle distance becomes longer.

Any method can be used to vary the display size, but in one embodiment, as described above, a plurality of image data for the virtual marker are prepared according to the inter-vehicle distance, and the image size of the virtual marker drawn in the frame is changed. It is made different according to the inter-vehicle distance. In another embodiment, the control unit 160 controls the projection angle and focal length of the projection optical system 154 via the actuator 155 based on the inter-vehicle distance, thereby changing the display size of the virtual marker. good. Moreover, preferably, the display size determination unit 240 measures the size of the forward vehicle by the obstacle detection unit 210, and determines the display size of the virtual marker accordingly. As a result, it is possible to eliminate the sense of discomfort caused by the size mismatch between the forward vehicle and the virtual marker. In addition, the display size may be varied according to the type of vehicle in front (bus, truck, ordinary vehicle, light vehicle, motorcycle, etc.).

Projection control section 250 controls HUD 130 so that the virtual marker is projected according to the display position determined by display position determination section 230 and the display size determined by display size determination section 240 . That is, the projection control section 250 controls the image data for the virtual marker provided to the light modulation means 153 and controls the projection of the projection optical system 154 via the actuator 155 . By performing the above adjustments, the projection optical system 154 projects the image of the virtual marker onto the windshield 25, and the driver can visually recognize the virtual marker without overlapping the vehicle ahead.

Threshold determination unit 260 determines whether the inter-vehicle distance measured by inter-vehicle distance measurement unit 220 is below a preset threshold. There are multiple thresholds, and the settings can be changed as needed. The threshold determination section 260 detects whether or not the vehicle-to-vehicle distance is below each threshold as needed, and outputs the result to the projection image switching section 270 when it is below the threshold.

Here, the range in which the virtual marker can be displayed based on the inter-vehicle distance will be described with reference to FIG. In FIG. 6, A represents the area where the driver sees the forward vehicle X, C represents the blind spot from the driver even if the virtual image of the virtual marker is displayed on the hood of the vehicle M, and X represents the forward vehicle. Let B be an area that does not overlap and does not cause a blind spot due to the hood. In this area B, the display position of the virtual marker is varied. However, this region B changes according to the position of the vehicle X ahead. For example, as the preceding vehicle X approaches, the inter-vehicle distance X becomes shorter, and the region B narrows accordingly. Region B disappears when it approaches the position of _Ltmax . That is, there is no space in which the virtual marker can be displayed between the vehicles ahead. _{Let Ltmax} be the threshold value of the inter-vehicle distance at which the area B disappears.

The projection image switching unit 270 switches the image of the virtual marker each time the threshold set by the threshold determination unit 260 is exceeded. In this embodiment, an image for each lower threshold is selected from the image data 124 stored in the storage unit 120 and switched. Note that the projection control section 250 may have the function of the projection image switching section 270 .

Next, the operation of the display system according to the embodiment of the present invention will be explained using the flowchart of FIG. Further, description will be made with reference to FIG. 8, which is a side view schematically showing the inter-vehicle distance and the display position of the virtual marker, and FIG.

First, the HUD 150 projects an image for a virtual marker in the line-of-sight direction of the driver to display the virtual marker as a virtual image (S101). A virtual marker in the shape of a vehicle is used in this embodiment. The behavior of this virtual marker is linked with the navigation device 130 to guide the user to the destination. Next, the obstacle detector 210 detects whether there is another vehicle ahead (S103). When the forward vehicle X is detected, the inter-vehicle distance measuring unit 220 measures the inter-vehicle distance to the detected forward vehicle X (S105).

According to the inter-vehicle distance, the display position and display size of the virtual marker are determined by the display position determination unit 230 and the display size determination unit 240, adjusted by the projection control unit 250, and projected (S107). Specifically, when the vehicle-to-vehicle distance is as large as L1 as shown in FIG. After adjustment, a virtual image D1 is projected as shown in FIG. 9(A). Next, when the inter-vehicle distance as shown in FIG. 8(B) becomes shorter than L1 and approaches L2, which is the limit inter-vehicle distance at which the entire virtual marker can be displayed, the optimum position (generated distance P2) is obtained similarly to L1. The projection control unit 250 performs adjustment so that the virtual image D2 is displayed on the screen, and the virtual image D2 is projected as shown in FIG. 9B. Also, a limit inter-vehicle distance such as this L2 that allows the entire virtual marker to be displayed is set as a threshold value _Lt1 . That is, if the inter-vehicle distance is L _t1 or more, the entire virtual marker can be displayed by adjusting the projection control unit 250 .

Next, the threshold determination unit 260 determines whether the inter-vehicle distance has fallen below the first threshold _Lt1 (S109). As shown in FIG. 8C, when the inter-vehicle distance is L3, which is smaller than _Lt1 , and the virtual image is generated at the generation distance P2, a portion Q1 where the virtual marker appears to overlap with the forward vehicle X is generated. In this case, as shown in FIG. 9C, an image in which a part of the upper part of the virtual marker is missing is prepared in advance, and the image is switched by the projection image switching unit 270 and projected to display the virtual image D3 (S111). . A limit inter-vehicle distance at which an image in which a part of the upper part of the virtual marker can be displayed is set as a threshold value _Lt2 .

Next, the threshold determination unit 260 determines whether the second threshold _Lt2 is exceeded (S113). As shown in FIG. 8(D), when the virtual image is generated at the generation distance P2 when the inter-vehicle distance is L4, which is smaller than _Lt2 , a portion Q2 (Q2>Q1) where the virtual marker appears to overlap with the forward vehicle X is generated. do. In this case, as shown in FIG. 9D, the virtual image D4 is displayed by switching the projection image switching unit 270 to an image of only the rear tires by removing the upper portion of the virtual marker and projecting the image. (S115). It should be noted that the image of only the rear tires is the last image that can be displayed while the images are switched step by step according to the approach of the forward vehicle X in this embodiment.

Next, the threshold determination unit 260 determines whether the third threshold (the last displayable threshold) _{L_tmax} is exceeded (S117). As shown in FIG. 8(E), when the inter-vehicle distance is L5, which is smaller than _Ltmax , and the virtual image is generated at the generation distance P2, all the virtual markers appear to overlap with the vehicle X ahead. In this case, no virtual marker is displayed as shown in FIG. 9(E) (S119). If the first threshold, the second threshold, and the third threshold are not less than the respective thresholds, the inter-vehicle distance measuring unit 220 measures whether or not the inter-vehicle distance has changed (S121). ). If there is a change in the inter-vehicle distance, the process proceeds to S107. At this time, as the inter-vehicle distance increases, the display image is switched opposite to the previous one. For example, the projection control unit 250 and the projection image switching unit 270 switch the display step by step, such as an image of only the rear tires, an image with the upper part missing, and the entire display. Next, the obstacle detector 210 detects whether or not the forward vehicle X has disappeared (S123). When the preceding vehicle X disappears, the processing ends.

As described above, the display of the virtual marker shifts to the near side as the preceding vehicle approaches the own vehicle, and when the entire virtual marker cannot be displayed due to the approach, the upper side of the original virtual marker is displayed. By gradually switching to an image in which the missing range gradually increases, the virtual marker does not overlap the vehicle in front, and features such as the tire part and the blinker part are used to lead. It becomes possible to display a different part for as long as possible.

Note that the threshold and the number of virtual marker images to be switched for each threshold can be changed as appropriate depending on the embodiment. In addition, it is also possible to change the setting of the section in which each switched virtual marker can be displayed (setting between thresholds). Also, for each switched virtual marker, the optimal display form is determined by the display position determination unit 230 and the display size determination unit 240, and adjusted and projected by the projection control unit 250, so that the display without discomfort can be performed.

Next, another example of this embodiment will be described. When the vehicle-shaped virtual marker is displayed, for example, by processing the winker portion on the virtual marker to express blinking, etc., it is possible to perform route guidance in a more comprehensible manner. However, in the present invention, since the image is gradually switched from the upper part of the virtual marker to the missing image, the information cannot be provided when the information presentation part such as the turn signal is switched to the missing image. end up

Therefore, for example, as shown in FIG. 10A, guidance is provided by turning on the turn signals 30, and when the display switches to only the rear tires due to the approach of the vehicle X in front, a turn signal auxiliary 33 is provided to provide an auxiliary display. to It should be noted that the display position and display form of the winker assistance 33 can be appropriately changed depending on the embodiment. In addition to the winker, additional information that cannot be displayed due to image switching is displayed in a different form on the virtual image when the image switching is detected. As a result, additional information can be continuously presented even when the image is switched to the missing image of the virtual marker.

Further, this embodiment can be adapted to display a virtual marker of a three-dimensional image at a position corresponding to the direction of the driver's face or line of sight. In the above embodiment, the virtual marker has the shape of a vehicle, but this is an example, and the virtual marker may have a shape other than the vehicle shape.

Although preferred embodiments of the present invention have been described in detail, the present invention is not limited to specific embodiments, and various modifications and changes can be made within the scope of the gist of the invention described in the claims. It is possible.

10 Display system 25 Windshield 101 Front camera 102 Internal camera X Forward vehicle M Own vehicle D Virtual image L Inter-vehicle distance P Generated distance

Claims (9)

Projection means for projecting an image for a virtual marker in front of the own vehicle for guiding the direction of travel;
detection means for detecting an obstacle in front of the own vehicle;
measuring means for measuring the inter-vehicle distance between the detected obstacle and the host vehicle;
a control means for controlling projection of the projection means so that the image for the virtual marker is displayed between the obstacle and the host vehicle according to the inter-vehicle distance;
The control means shifts the display position of the virtual marker image forward when the inter-vehicle distance becomes shorter, shifts the display position of the virtual marker image forward when the inter-vehicle distance increases, and shifts the display position of the virtual marker image forward when the inter-vehicle distance increases. A display system that switches the image for the virtual marker to another image when the distance falls below a threshold. 2. The display system of claim 1, wherein the other image is an image lacking an upper portion of the image for the virtual marker. 3. The method according to claim 1, wherein a plurality of said thresholds exist, and said control means switches to another image so that loss from top to bottom gradually increases each time said inter-vehicle distance falls below each threshold. display system. 4. The display system according to claim 2, wherein said control means switches to another image when the position where the virtual image of the image for said virtual marker is displayed overlaps with the position of said obstacle. 5. The display system according to any one of claims 2 to 4, wherein said control means completely hides the virtual image of said virtual marker image when said inter-vehicle distance becomes smaller than a limit value. The image for the virtual marker is a virtual guided vehicle that imitates the shape of the vehicle, and the control means gradually increases the loss from the top to the bottom of the virtual guided vehicle as the inter-vehicle distance decreases. 6. The display system according to any one of claims 1 to 5, wherein the display system switches to an image step by step, and switches to an image of only the rear tire portion of the virtual guided vehicle when the vehicle-to-vehicle distance reaches a certain threshold or less. 7. The display system according to claim 6, wherein said control means switches images step by step so that the missing portion of said virtual guided vehicle gradually becomes smaller as said inter-vehicle distance increases. The display system further includes storage means for storing images for a plurality of virtual markers and navigation means for providing route guidance,
8. The display system according to any one of claims 1 to 7, wherein said control means switches the image for said virtual marker read out from said storage means in conjunction with said navigation means. 9. The display system according to claim 8, wherein said control means projects an image for said virtual marker including instruction information for changing the direction of travel.

Images