EP4178822A1

EP4178822A1 - Intelligent head-up display

Info

Publication number: EP4178822A1
Application number: EP21745354.7A
Authority: EP
Inventors: Philip Zimmermann
Original assignee: Volkswagen AG
Current assignee: Volkswagen AG
Priority date: 2020-08-06
Filing date: 2021-07-21
Publication date: 2023-05-17
Also published as: CN116056934A; DE102020209971A1; US20230271622A1; WO2022028892A1

Abstract

The invention relates to a device (10) for displaying information (24) for an occupant (30) of a motor vehicle (28) and to a corresponding system (18), method and computer program. The disclosed teaching may in particular be used to achieve the following: The display position of a head-up display is able to be varied flexibly depending on the context. To this end, both a displacement of the X and/or Y position and a colour selection adapted to the background may be implemented and be performed automatically. Information from the steering angle, speed, map data and also head tracking by the interior camera (34) may be combined to determine the X and/or Y position. In order to achieve optimum visibility for any load case, sensor fusion from the position data (32) and an exterior camera (36) may in particular take place: Content or information (24) may thus be displayed depending on the current display position, so as to be more visible. If only some of the information (24) or part of the information symbol (42, 44) is on a background, only some of the colour may also change, such that visibility is ensured for the entire information symbol (42, 44). In addition, using visual warning colours and simultaneously displacing the display in the direction of a source of a hazard (46) makes it possible to divert the gaze of the driver (30) towards the hazardous situation.

Description

description

Intelligent head-up display

The invention relates to a device for displaying information for an occupant of a motor vehicle and a corresponding system, method and computer program.

It is known to use so-called head-up display devices for displaying information in vehicles. In this case, information, for example the current driving speed, is projected onto the windshield and visually superimposes the image located behind the windshield. These display devices usually have a projector that throws the light emitted by the display against the windshield, which acts as a combiner and reflects the light, thus providing a virtual image in front of the windshield that is visible to the driver. A disadvantage of such systems is that part of the image behind the windshield is always at least partially covered, so that relevant information from the image behind the windshield is difficult for a driver to perceive. In addition, the driver's attention and line of sight are directed to the projection onto the windshield. The driver's attention to information from the image behind the windshield is reduced.

Document DE 198 13 300 A1 relates to a display device, in particular a passenger information system for motor vehicles, with a device for projecting at least one virtual image in front of a windshield of the motor vehicle. Provision is made for a position of the at least one virtual image in front of the windshield to be variable as a function of at least one driving condition parameter. The disadvantage here is that only one driving condition parameter is taken into account, a viewing direction of the driver and the condition of the roadway are not taken into account. As a result, for example, a pothole can be covered by the virtual image and only poorly or not at all recognized by the driver, which leads to an increased risk of an accident.

Document DE 103 36 329 A1 relates to a method, a device and a processing unit for improving visibility in a motor vehicle. From the image of the area around the motor vehicle, which records a camera of the motor vehicle, a Image section provided, the size of the image section and additionally or alternatively the position of the image section within the image depending on the course of the road and / or the current and / or desired by the driver direction of travel of the motor vehicle is set. The disadvantage here is that only a predetermined trajectory of the motor vehicle is taken into account. With the teaching disclosed, it is not possible for the driver to direct his gaze in a targeted manner, for example in the direction of the source of danger when a source of danger has been identified. In addition, it cannot be guaranteed that the driver will capture the image section immediately, since the driver's eye position or viewing direction is not determined.

The publication DE 102007 035 505 A1 describes a motor vehicle with a head-up display for displaying information, the information displayed by the head-up display being visible from a spatial element inside the motor vehicle, with an adjustment device for adjusting the position of the Spatial element with a detection device for detecting the position of the eyes or the head of a driver of the motor vehicle, with a control device which is set up and coupled to the adjusting device and the detection device in such a way that the position of the spatial element depends on the detected position of the eyes or the Head of a driver is adjusted. The disadvantage here is that only the driver's eye position is recorded in order to achieve improved visibility of the information. A transmission of further information through the position of the notice on the windscreen and/or a subtle directing of the driver's gaze is not achieved.

The published application DE 102018 004662 A1 relates to an optical detection system, in particular for a driver assistance system for detecting the area of the roadway/driving trajectory in the direction of travel, with a detection area for recording a section of the surroundings, comprising an image sensor chip which has a specific maximum number of pixels, with the recording of the environmental image, a specific number of pixels is active in a pixel area of the image receiver chip to be evaluated, wherein the maximum number of pixels of the image receiver chip is greater than the number of pixels of the pixel area to be evaluated and wherein the detection area is variable, the variation and/or alignment of the detection area on the driving trajectory ahead takes place by changing the pixel area to be evaluated on the image receiver chip, with a) control parameters supplied for this purpose serving as input parameters, which originate from other systems located in the motor vehicle en and/or are made available by means of these systems, and/or b) self-determined control parameters as input parameters are used, which originate from the optical detection system and/or are made available by means of the optical detection system. The disadvantage here is that no smart further processing of the recorded information takes place. In particular, only the output of a notice is disclosed in the publication as one possibility.

The invention is now based on the object of creating a device for displaying information for an occupant of a motor vehicle, which at least partially overcomes the disadvantages mentioned above. In particular, an intelligent device that is at least partially tailored to an occupant is to be created.

The above object is achieved by a device for displaying a message for an occupant of a motor vehicle, having: an input interface for receiving data with information about the message for the occupant; an analysis unit for analyzing the data and generating a control command, the control command causing the information to be output visually in the form of an information symbol by means of an augmented reality display device; and an output interface for transmitting the control command to an augmented reality display device; wherein the control command causes a predefined optical output of the indication at a predefined position in the occupant's field of vision.

The above object is also achieved by a system for displaying information for an occupant of a motor vehicle, having: a unit for determining information which is to be transmitted to the occupant by means of the information symbol; a device as previously defined; and an augmented reality display device for outputting an indication according to the predefined visual output at a predefined position in the occupant's field of vision in response to a control command of the device.

In addition, the above object is achieved by a method for displaying a message for an occupant of a motor vehicle, preferably using a system as defined above, comprising the steps:

receiving data including information about the notice to the occupant; analyzing the data; Generating a control command, wherein the control command causes the information to be output visually in the form of an information symbol by means of an augmented reality display device; and transmitting the control command to an augmented reality display device; where the Control command causes a predefined optical output of the note at a predefined position in the occupant's field of vision.

Finally, the above object is achieved by a computer program with program code means to carry out all the steps of a method as defined above when the computer program is executed on a computer or a corresponding computing unit.

By using an input interface for receiving data with information about the information for the occupant, a technically simple and cost-effective device can be created which, in particular, does not have to be designed to record the information itself. The device can thus be integrated into existing navigation or infotainment systems. A control command for an augmented reality display device (AR display device) enables improved visualization of the information for an occupant. In particular, a driver can direct his gaze to the roadway and does not have to look at an additional display in the motor vehicle. Road safety is increased. Components already installed in the motor vehicle can be used for the display by means of an output interface. It is also conceivable to retrofit a display device and/or to use several display devices in combination. By means of a predefined optical output of an information symbol at a predefined position in the occupant's field of vision, the occupant can grasp the information without having to adjust his field of vision. Furthermore, it is also conceivable to direct the occupant's gaze or field of vision to a predefined position by outputting the information and thus to draw the occupant's attention to a dangerous situation, for example. The visibility of the notice can be increased by means of a predefined optical output, for example a suitable color or a suitable contrast can be selected for the notice based on the situation in front of the motor vehicle, for example black writing can be selected if the predefined position is in front of a white one truck is located.

Furthermore, the system can also be used for entertainment, for example games such as - I see something that you don't see -, - Tetris - and/or - Pong - can be implemented through flexible context-related design. Here, the predefined position and the predefined optical output are determined by parameters of the game.

In a preferred embodiment of the invention, it is provided that the device is designed for installation in a motor vehicle and is preferably part of an on-board computer and/or navigation system is. This can significantly increase comfort when driving a car. In particular, the device can also be offered as a type of retrofit solution for cars that have already been purchased and can be set up, for example, by replacing or reprogramming the on-board electronics.

In a further preferred embodiment of the invention, it is provided that the input interface is designed to receive sensor data from a sensor with information relating to the occupant's field of vision, the analysis unit being designed to determine the predefined position in the occupant's field of vision based on the sensor data. This can significantly improve the visibility of the notice.

In particular, the information can be grasped by the occupant without a significant change in the line of sight.

In a further preferred embodiment of the invention, it is provided that the input interface is designed to receive sensor data from a sensor which has at least one steering angle sensor, a speed sensor, a position sensor, a head tracking sensor, an interior camera, in particular an occupant camera, and/or an exterior camera includes, preferably data from at least two sensors, in particular a position sensor and an outdoor camera, can be combined in a sensor fusion. This allows an improved determination of the predefined position.

In a further preferred embodiment of the invention, it is provided that the analysis unit is designed to determine a source of danger in the field of vision of the occupant, in particular the driver, with the control command causing the information symbol to be shifted to the source of danger in the field of vision of the occupant. As a result, road safety can be improved. In particular, the occupant's attention can be drawn to the source of danger at an early stage. Convenience is increased because preferably no warning tone or other warning signal needs to be emitted to warn the occupant.

In a further preferred embodiment of the invention, it is provided that the analysis unit is designed to generate a predefined optical output, in particular a color and/or a type of information symbol, based on a situation in front of the motor vehicle in the occupant's field of vision and the predefined position in the occupant's field of vision to determine in order to cause an output of the notification symbol with increased visibility for the occupant by means of the control command. For example, a color or structure of the situation before the Motor vehicle can be determined with a sensor, such as an outdoor camera, wherein the predefined optical output can include a modification of contrast, contrasting color and/or a background for the information.

In a further preferred embodiment of the invention, it is provided that the augmented reality display device comprises a head-up display on a motor vehicle, the head-up display preferably being designed to display information at any position on the windscreen of a motor vehicle. In this way, an indication can be output at the predefined position in a technically simple manner. A projection unit of the head-up display and/or a mirror can be moved using actuators in order to enable projection at the predefined position.

Further preferred configurations of the invention result from the remaining features mentioned in the dependent claims.

Unless stated otherwise in the individual case, the various embodiments of the invention mentioned in this application can advantageously be combined with one another.

The invention is explained below in exemplary embodiments with reference to the associated drawings. Show it:

FIG. 1 shows a schematic representation of a device for displaying information for an occupant of a motor vehicle;

Figure 2 is a schematic representation of a variant of an inventive

systems;

FIG. 3 shows a schematic representation of a further variant of a system according to the invention in a vehicle;

FIG. 4 shows a schematic representation of a further variant of a system according to the invention in a vehicle;

Figure 5 shows a schematic representation of an output of the system on an AR

display device; FIG. 6 shows a schematic representation of an output from the system on an AR display device in a driving situation; and

FIG. 7 shows a schematic representation of a method according to the invention.

FIG. 1 shows a schematic representation of a device 10 for displaying information for an occupant of a motor vehicle. The device 10 comprises an input interface 12, an analysis unit 14 and an output interface 16.

The input interface 12 is designed to receive data with information about the information for the occupant. The data can, for example, include a speed or a navigation instruction and can originate, for example, from a speedometer, GPS receiver and/or navigation system. For receiving, the input interface 12 is preferably connected to an in-vehicle transmission network. Furthermore, the input interface 12 can also be designed for wireless communication or be connected to a proprietary transmission network, for example a wired transmission network.

The analysis unit 14 is designed to receive the data and to generate a control command, the control command causing the information to be output visually in the form of an information symbol by means of an augmented reality display device. The control command causes a predefined optical output of the message at a predefined position in the occupant's field of vision.

The output interface 16 is designed to transmit the control command to the augmented reality display device. The output interface 16 can be designed analogously to the input interface 12 for communication. It goes without saying that the input interface 12 and the output interface 16 can also be designed in combination as a communication interface for sending and receiving.

Figure 2 shows a schematic representation of a system 18 according to the invention with a device 10 and an AR display device 20 with a projection unit 22.

The device 10 receives data with information about the indication. This data can be, for example, the speed of the motor vehicle, data from an eye-tracking sensor, Data on the traffic situation, position data of the motor vehicle, data relating to a position of a source of danger, etc.

On the basis of the data, the device 10 creates a control command for the AR display device 20 and thus causes a message 24 to be output with regard to information for an occupant on a windscreen 26 of a motor vehicle. In the example shown, the AR display device 20 comprises a head-up display, with the projection unit 22 being designed to be movable and being able to be moved, for example by means of actuators, so that the information 24 is projected on the windshield 26 at the predefined position in the occupant's field of vision can be. As a result, the information symbol can be displayed directly on the windscreen 26 of the motor vehicle.

Figure 3 shows a schematic representation of a variant of the system 18 according to the invention in a motor vehicle 28. The system 18 according to the invention comprises a device 10 and an AR display device 20. In the example shown, the AR display device 20 comprises a semi-transparent screen which is not attached to the windscreen 26 of the motor vehicle 28 shown in more detail is arranged.

As described above, the device 10 receives data with information about the notice 24 for an occupant 30. In the example shown, the system 18 is designed as part of a navigation system and is connected to a GPS receiver 32 for position determination or speed determination. It goes without saying that the system 18 can also be designed as an independent unit. In the example shown, the navigation system may form an entity that determines information, such as a navigation prompt, to be communicated to the occupant 30 .

Additional data can come from an occupant camera 34, for example to determine a viewing direction of the occupant 30, or from an external camera 36, such as a front camera, to determine a situation in front of the motor vehicle 28. It goes without saying that other sensors known in principle from the prior art can also be used. In particular, a sensor data fusion of data from a number of sensors can take place in order to achieve higher data quality.

The device 10 analyzes the received data and determines a control command that causes the message 24 to be output at a predefined position in the field of vision of the occupant 30 . FIG. 4 shows a schematic representation of a further variant of a system 18 according to the invention in a motor vehicle 28. The same reference symbols relate to the same features and are not explained again. In the following, the differences to the variant shown in FIG. 3 will be discussed.

In contrast to the variant shown in FIG. 3, AR display device 20 is designed in the form of a head-up display, as described with reference to FIG. Such an embodiment is preferably suitable as an initial equipment variant, since major structural changes have to be made in motor vehicle 28 in order to install a correspondingly controllable projection unit 22 . In particular, the front screen 26 of the motor vehicle 28 can be replaced here in a technically simple manner, since the front screen 26 is only a passive part of the AR display device 20 .

In the example shown, the sensors are not shown for reasons of clarity.

FIG. 5 shows a schematic representation of an output 38 of the system 18 on an AR display device 20 in a driving situation. A scenario from the driver's perspective is shown. The driving situation includes driving on a lane 40 of a country road. The reference symbol 42 is shown schematically as a star

In the example shown, it is advantageous for a driver of the motor vehicle 28 to direct his or her gaze towards the apex of the curve, therefore the predefined position of the indicator 42 is the apex of the curve. The position of an information symbol 44 known in the prior art is shown in dashed lines. By displaying the information symbol 42 at the predefined position, the driver can detect the information symbol 42 without having to take his eyes off the apex of the curve. The visibility of the notification icon 42 is increased. Furthermore, road safety is increased since the driver is stimulated to optimally direct his gaze.

FIG. 6 shows a schematic representation of an output 38 of the system 18 on an AR display device 20 in a further driving situation. The same reference symbols refer to the same features and will not be explained again. The differences will be discussed below. In contrast to the situation shown in FIG. 5, there is an obstacle 46 in front of motor vehicle 28. In the example shown, obstacle 46 comprises a piece of wood lying on roadway 40. The piece of wood can lead to a hazard, particularly if there is a collision between the motor vehicle 28 and the piece of wood. Analysis unit 14 is designed to recognize the imminent danger and, for this purpose, analyzes, for example, the image from an external camera 36, in particular a front camera, or other sensor data. In order to draw the driver's attention to the imminent danger, the analysis unit 14 can generate a control command in order to display the information 24 at the source of the danger. In the example shown, the information symbol 44 is displayed in such a way that it is in front of the piece of wood from the driver's point of view. The predefined position in the driver's field of vision is therefore in front of the piece of wood. It goes without saying that the hazardous situation described above was only selected as an example in order to explain the functioning of the present teaching. One skilled in the art will recognize that the teaching can be applied to a variety of other hazardous situations.

In the example shown, a predefined optical output of the information symbol 44 also takes place. In order to increase the visibility or perceptibility of the information symbol 44 for the occupant or driver, the information symbol 44 has a background 48 on it. It is understood that the predefined visual output may also include changing the color or contrast of the indicator 44 .

The steps of a method according to the invention for displaying a message 24 for an occupant 30 of a motor vehicle 28 are shown schematically in FIG. The method is preferably operable using the system 18 as previously described and includes the following steps. In a step S1, data with information about the notice 24 for the occupant 28 is received. A step S2 includes analyzing the data. In a step S3, a control command is generated, the control command causing the information 24 to be output visually in the form of an information symbol 42, 44 by means of an augmented reality display device 20. A step S4 includes the transmission of the control command to an augmented reality display device 20.

The control command causes the information 24 to be output optically at a predefined position in the field of vision of the occupant 28.

It goes without saying that the method can be executed using a computer program with program code means, the program code means being designed to carry out all the steps of the method when the computer program runs on a computer or a corresponding processing unit, such as an on-board computer, a navigation and/or infotainment system.

The invention has been described in detail. With the disclosed teaching, the following advantages can be achieved and/or problems solved in particular by at least one embodiment:

- The display position of a head-up display can be flexibly varied depending on the context.

- To do this, the X and/or Y position can be shifted and the color selected can be adapted to the background and carried out automatically.

- To determine the X and/or Y position, information from the steering angle, speed, map data and head tracking from the interior camera can be combined.

- In particular, in order to achieve optimal visibility for each load case or application, a sensor fusion of the position data and an external camera can take place: This means that content or information can be displayed depending on the current display position in such a way that it is more visible. If only part of the notice or notice symbol is in front of a background, only part can change color, so that visibility for the entire notice symbol is ensured.

- In addition, the use of visual warning colors and a simultaneous shifting of the display in the direction of a source of danger can direct the driver's gaze towards the dangerous situation.

reference number

10 device

12 input interface

14 analysis unit

16 output interface

18 systems

20 Augmented Reality Display Device (AR Display Device)

22 projection unit

24 Note

26 windscreen

28 motor vehicle

30 inmates

32 GPS receivers

34 occupant camera

36 outdoor camera

38 edition

40 lane

42 Notice icon

44 Notice icon

46 obstacle

48 background

S1-S4 procedural steps

Claims

Claims Device (10) for displaying an indication (24) for an occupant (30) of a motor vehicle (28), comprising: an input interface (12) for receiving data with information on the indication (24) for the occupant (30); an analysis unit (14) for analyzing the data and generating a control command, the control command causing the information (24) to be output visually in the form of an information symbol (42, 44) by means of an augmented reality display device (20); and an output interface (16) for transmitting the control command to an augmented reality display device (20); characterized in that the control command causes a predefined optical output of the indication (24) at a predefined position in the field of vision of the occupant (30). Device (10) according to Claim 1, characterized in that the device (10) is designed for installation in a motor vehicle (28) and is preferably part of an on-board computer and/or navigation system. Device (10) according to one of the preceding claims, characterized in that the input interface (12) is designed to receive sensor data from a sensor with information relating to the field of vision of the occupant (30), the analysis unit (14) being designed to be based on this to determine the predefined position in the field of vision of the occupant (30) on the basis of the sensor data. Device (10) according to one of the preceding claims, characterized in that the input interface (12) is designed to receive sensor data from a sensor which has at least one steering angle sensor, a speed sensor, a position sensor, a head tracking sensor, an interior camera, in particular a Passenger camera (34) and/or an exterior camera (36), it being possible for data from at least two sensors, in particular a position sensor and an exterior camera (36), to be combined in a sensor fusion. Device (10) according to one of the preceding claims, characterized in that the analysis unit (14) is designed to determine a source of danger in the field of vision of the occupant (30), in particular the driver, the control command causing a displacement of the information symbol (42, 44) to the source of danger in the field of vision of the occupant (30). Device (10) according to one of the preceding claims, characterized in that the analysis unit (14) is designed to, based on a situation in front of the motor vehicle (28) in the field of vision of the occupant (30) and the predefined position in the field of vision of the occupant (30 ) to determine a predefined optical output, in particular a color and/or a type of the information symbol (42, 44), in order to cause the information symbol (42, 44) to be output with increased visibility for the occupant (30) by means of the control command. System (18) for displaying a notice (24) for an occupant (30) of a motor vehicle (28), comprising: a unit for determining information which is to be transmitted to the occupant (30) by means of the notice symbol (42, 44); a device (10) according to any one of the preceding claims; and an augmented reality display device (20) for outputting an indication (24) at a predefined position in the field of vision of the occupant (30) in response to a control command from the device (10). System (18) according to claim 7, characterized in that the augmented reality display device (20) comprises a head-up display on a motor vehicle (28), wherein the head-up display is preferably designed to information at each position Represent windscreen (26) of a motor vehicle (28). Method for displaying an indication (24) for an occupant (30) of a motor vehicle (28), preferably by means of a system (18) according to one of Claims 7 or 8, comprising the steps:

Receiving (S1) data with information about the notice (24) for the occupant (30);

analyzing (S2) the data; Generating (S3) a control command, the control command causing the information (24) to be output visually in the form of an information symbol (42, 44) by means of an augmented reality display device (20); and

Transmission (S4) of the control command to an augmented reality display device (20); characterized in that the control command causes the indication (24) to be output optically at a predefined position in the field of vision of the occupant (30). Computer program with program code means to carry out all the steps of a method according to the preceding claim when the computer program is executed on a computer or a corresponding computing unit.