KR102527043B1 - System and method for controlling head up display - Google Patents

Abstract

The present invention relates to a HUD control system and method.
The HUD control system considering the driver's gaze according to the present invention includes an input unit for receiving the driver's gaze information, a memory storing a program for determining the display position of the HUD information using the driver's gaze information, and a processor for executing the program, , The processor determines the gazing point using the driver's gaze information, and determines the display position so that the HUD information is displayed at the gazing point.

Description

HUD control system and its method {SYSTEM AND METHOD FOR CONTROLLING HEAD UP DISPLAY}

The present invention relates to a HUD control system and method, and more particularly, to a HUD control system and method considering a driver's line of sight.

Since a head-up display (HUD) according to the related art does not consider a driver's gaze position (focusing point), there is a problem in that the driver cannot perceive information displayed through the HUD according to circumstances.

In addition, according to the prior art, there is a problem in that the HUD is not controlled in consideration of the driver's condition or surrounding information.

The present invention has been proposed to solve the above-described problems, and by determining the driver's gaze and controlling to provide HUD information to a location where the driver's gaze is located, a HUD that supports intuitive perception of HUD information even when the driver's gaze moves. Its purpose is to provide a control system and method.

The HUD control system considering the driver's gaze according to the present invention includes an input unit for receiving the driver's gaze information, a memory storing a program for determining the display position of the HUD information using the driver's gaze information, and a processor for executing the program, , The processor determines the gazing point using the driver's gaze information, and determines the display position so that the HUD information is displayed at the gazing point.

The processor determines whether the change of the driver's gaze is a change of the driver's gaze in a situation in which a change in the expression position is required by monitoring the driver's gaze information received through the input unit in conjunction with the driving information in chronological order.

When it is determined that the change of the expression position is unnecessary, the processor determines that the driver's gazing point is maintained in the front area and maintains the expression position.

If it is determined that the display position needs to be changed, the processor determines that the driver's current visual field is the gazing point, and changes the display position so that the HUD information is displayed at the gazing point.

In addition, when the information provided at the changed display position and the color of the surrounding environment at the driver's viewing position are similar, the information transfer effect may be impaired, so the processor secures visibility and readability by changing the background color of the displayed information. .

The HUD control method considering the driver's gaze according to the present invention comprises the steps of (a) tracking the driver's gaze information, (b) determining the HUD information display position using the driver's gaze information, and (c) the determined display position. It includes the step of expressing HUD information according to.

Step (b) monitors the change in gaze information in conjunction with the driving situation, and determines whether a change in the HUD information display position is necessary according to the change in gaze information.

In step (b), if it is determined that the driver's gazing point has changed according to the change in gaze information, the HUD display position is changed so that the area corresponding to the gazing point becomes the HUD information display position.

Step (b) maintains the HUD information display position without changing it, if the driver's gaze temporarily deviated from the front according to the change in gaze information, but it is expected that the driver's gaze will return to the front according to the driving situation.

According to the present invention, by moving the HUD information to the driver's gazing point, it is possible to support the driver to intuitively perceive the HUD information even when the driver is looking at a place other than the front.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 illustrates a HUD control system considering a driver's gaze according to an embodiment of the present invention.
2 illustrates an example of changing a HUD information display area in consideration of driver's field of view information according to an embodiment of the present invention.
3 illustrates a HUD control method considering a driver's gaze according to an embodiment of the present invention.
4 illustrates a HUD control system using driver status and surrounding information according to another embodiment of the present invention.
5 illustrates a concept of HUD control using driver status and surrounding information according to another embodiment of the present invention.
6 illustrates a HUD control method using a driver's condition and surrounding information according to another embodiment of the present invention.

The foregoing and other objects, advantages and characteristics of the present invention, and a method of achieving them will become clear with reference to the detailed embodiments described below in conjunction with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms, and only the following embodiments provide the purpose of the invention, As only provided to easily inform the configuration and effect, the scope of the present invention is defined by the description of the claims.

Meanwhile, terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, “comprises” and/or “comprising” means the presence of one or more other components, steps, operations, and/or elements in which a stated component, step, operation, and/or element is present. or added.

1 illustrates a HUD control system considering a driver's gaze according to an embodiment of the present invention.

The HUD control system considering the driver's gaze according to an embodiment of the present invention includes an input unit 110 for receiving the driver's gaze information, and a memory 120 storing a program for determining the display position of the HUD information using the driver's gaze information. and a processor 130 executing the program, wherein the processor 130 determines a gazing point using driver's gaze information, and determines an display position to display HUD information at the gazing point.

The processor 130 monitors the driver's gaze information received through the input unit 110 in conjunction with the driving information in chronological order, thereby determining whether the driver's gaze change is a change in the driver's gaze change in a situation where a change in the expression position is required. do.

When it is determined that the change of the expression position is unnecessary, the processor 130 determines that the driver's gazing point is maintained in the front area and maintains the expression position.

When it is determined that the display position needs to be changed, the processor 130 determines that the driver's current field of view is the gazing point, and changes the display position so that the HUD information is displayed at the gazing point.

According to an embodiment of the present invention, the HUD display area provides information not only to a specific area of the front windshield, but also to the entire area of the windshield.

The input unit 110 receives driver's gaze information from a vehicle interior sensor such as a driver's gaze tracker and a camera sensor.

The processor 130 determines the driver's gazing point even when the driver is looking at a place other than the front, and determines the HUD information display position so that the HUD information is displayed at the corresponding gazing point.

Referring to FIG. 2 , when the driver's field of view is in the V direction and does not look forward, HUD information is displayed in the windshield area A on the right side in consideration of the driver's field of view.

At this time, the processor 130 determines whether the change in the driver's gaze corresponds to the change in the driver's gaze to which the display position of the HUD information should be changed by accumulating and monitoring the driver's gaze information in a time sequence.

For example, it is assumed that a vehicle driving on a three-lane one-way road needs to make a right turn 100 meters ahead (using navigation information).

At this time, when the driver enters the rightmost lane and is driving, if the driver looks at the right area instead of the front at the point where the right turn is required, this occurs because the driver has already received navigation information from the HUD information and is ready to turn right. This is a change in gaze (that is, a situation in which the driver's gaze is changed to prevent an accident that may occur with a pedestrian or the like when turning right).

If so, it can be said that there is a high possibility that the driver will look in the forward direction (forward) again after making a right turn. The display position of the HUD information is changed and displayed in the driver's gaze point area (right side) when turning right, and then the driver after turning right again When the gazing point area (front) of the is determined and the HUD information is displayed in the front area accordingly, the HUD information may be displayed by moving the area later in timing, and if so, it may rather cause a sense of heterogeneity to the user.

Therefore, according to an embodiment of the present invention, instead of simply changing the display position of the HUD information in consideration of the driver's view information, the driving situation information and the driver's gaze change information are monitored to display the HUD information at the necessary moment. It is possible to change the display position.

Conversely, in the above example, it is assumed that a vehicle driving on a three-lane one-way road needs to turn right 100 meters ahead (using navigation information).

At this time, even though there is not much time left to turn right, if the driver is driving in the second lane and does not show intention to change lanes, such as turning on the right turn signal, the HUD information is displayed in color, size, and display method. By changing the lights (e.g., blinking), the driver is notified that a right turn is required according to the navigation information.

At this time, if the driver looks at the right area, it is determined that the driver intends to turn right, and as described above, it is expected that the driver will look forward again after turning right, so the display position of the HUD information is not separately changed. .

As described above, even though the HUD information is provided in a different display method for a certain period of time or longer, if the driver's vision information does not change (i.e., if he continues to look forward), he may drive on a route different from the route guided by the navigation. It is determined that there is a driver's intention, and HUD information is displayed in a normal mode as in the previous method.

When the driver is looking at an object of interest (a nearby building, a new vehicle driving in the next lane, etc.) in a traffic signal waiting situation, the processor changes the HUD information display position to the driver's gaze point.

Through this, when a vehicle departure notice in front, traffic light change notice information, etc. is provided through HUD information, even if the driver is not looking at the fixed HUD display area, the HUD information whose display position is determined in response to the driver's gaze point It is possible to increase driving convenience by recognizing .

When the driver's gazing point is changed in a direction other than forward, the processor provides a convenience function to the driver by using the driver's pupil movement or other body gestures.

For example, in a traffic light waiting situation, when a driver watches a building in which a newly opened bakery shop is located, the processor maintains driving-related information in the existing HUD display area and displays it, and the driver's field of view Object information is displayed corresponding to the gazing point area (eg, company name, phone number, menu, etc.).

At this time, when the driver blinks his pupil twice or inputs a hand gesture in the form of a check mark, the processor transmits and stores the information of the object of interest to a device owned by the user or a device inside the vehicle (e.g., OOO Bakery registered as a place of interest on OO, OO, 2020, Location: OO Street, OO Road, OO City, Seoul, OOOO Intersection).

If the place of the object of interest is located on the driver's route later, the processor may provide a notification to the driver (eg, "OOO Bakery previously registered as a POI is on the route. Main menu OOO is available at 10:00 AM, 1:00 PM, and 4:00 PM, and we will pass this place around 1:10 PM. If you want to use the OOO Bakery, please enter an OK gesture and we will register it as a stopover." ).

3 illustrates a HUD control method considering a driver's gaze according to an embodiment of the present invention.

The HUD control method considering the driver's gaze according to an embodiment of the present invention includes tracking the driver's gaze information (S310), determining the HUD information display location using the driver's gaze information (S320), and determining the display location. and displaying HUD information according to (S330).

In step S320, the change in gaze information is monitored in association with the driving situation, and it is determined whether a change in the HUD information display position is necessary according to the change in gaze information.

In step S320, if it is determined that the driver's gazing point has changed according to the change in gaze information, the HUD display position is changed so that the region corresponding to the gazing point becomes the HUD information display position.

In step S320 , when the driver's gaze temporarily deviated from the front according to the change in the gaze information, but the gaze is expected to return to the front according to the driving situation, the HUD information display position is maintained without changing.

In step S310, driver's gaze information is received from an internal vehicle sensor such as a driver's gaze tracker and a camera sensor.

At this time, in step S310 , driving information (eg, navigation information) is received along with the driver's gaze information, and a change history process of the driver's gaze information is monitored.

In step S320, it is determined whether or not the situation according to the change of the driver's gaze point is a situation in which the HUD information display position needs to be changed according to the result of monitoring the change history of the driver's gaze information.

If it is determined in step S320 that the change situation of the driver's gaze point is not a situation in which the HUD information display location needs to be changed, in step S330 the HUD information is continuously displayed at the previously displayed location.

For example, it is assumed that a vehicle driving on a three-lane one-way road needs to make a right turn 100 meters ahead (using navigation information).

At this time, when the driver enters the rightmost lane and is driving, if the driver looks at the right area instead of the front at the point where the right turn is required, this occurs because the driver has already received navigation information from the HUD information and is ready to turn right. This is a change in gaze (that is, a situation in which the driver's gaze is changed to prevent an accident that may occur with a pedestrian or the like when turning right).

If so, it can be said that there is a high possibility that the driver will look in the forward direction (forward) again after making a right turn. The display position of the HUD information is changed and displayed in the driver's gaze point area (right side) when turning right, and then the driver after turning right again When the gazing area (front) of the gazing point is determined and the HUD information is displayed in the front area accordingly, the HUD information may be displayed by moving the area later in timing, and if so, it may rather cause a sense of heterogeneity to the user, step S320. It is determined that the HUD information display position is maintained, and in step S330, the HUD information is displayed as it is in the previously displayed area.

On the other hand, if it is determined that the situation in which the driver's gaze information is changed in step S320 requires the HUD information display position to be changed, for example, in a traffic signal waiting situation, the driver is not looking at the movement of the vehicle in front or the traffic light, but is looking elsewhere. In this case, in step S320, it is determined to display HUD information (eg, notification of departure of a vehicle ahead, notification of change in traffic light) in the driver's gaze area, and step S330 displays the HUD information at the changed display position.

Meanwhile, the HUD control method considering the driver's gaze according to an embodiment of the present invention may be implemented in a computer system or recorded in a recording medium. A computer system may include at least one processor, a memory, a user input device, a data communication bus, a user output device, and a storage. Each of the aforementioned components communicates data through a data communication bus.

The computer system may further include a network interface coupled to the network. The processor may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory and/or storage.

The memory and storage may include various types of volatile or non-volatile storage media. For example, memory may include ROM and RAM.

Accordingly, the HUD control method considering the driver's gaze according to an embodiment of the present invention may be implemented as a computer-executable method. When the HUD control method considering the driver's gaze according to an embodiment of the present invention is executed in a computer device, computer readable instructions may perform the HUD control method considering the driver's gaze according to the present invention.

Meanwhile, the HUD control method considering the driver's gaze according to the present invention described above can be implemented as computer readable codes on a computer readable recording medium. Computer-readable recording media includes all types of recording media in which data that can be decoded by a computer system is stored. For example, there may be read only memory (ROM), random access memory (RAM), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device, and the like. In addition, the computer-readable recording medium may be distributed in computer systems connected through a computer communication network, and stored and executed as readable codes in a distributed manner.

4 illustrates a HUD control system using driver status and surrounding information according to another embodiment of the present invention.

A HUD control system using a driver's state and surrounding information according to another embodiment of the present invention uses at least one of an input unit 410 for receiving driver's state information and illuminance information, and at least one of driver's state information and illuminance information. It includes a memory 420 storing a program that determines an expression method and a processor 430 that executes the program, but the processor 430 determines the color or saturation of the HUD information by using at least one of the driver's state information and illuminance information. Control to display changes.

The processor 430 changes and displays the color or saturation of the HUD information using RGB color information as well as simple vertical and horizontal illuminance information.

The processor 430 changes and displays the color or saturation of the HUD information in consideration of RGB color information (dots) or distribution (surface) at the driver's gaze position.

The input unit 410 receives state information related to the driver's fatigue from the driver monitoring camera, the pressure sensor holding the steering wheel, and the heart rate sensor.

The input unit 410 receives illumination and weather information around the vehicle through a camera sensor outside the vehicle.

At this time, the processor 430 identifies color information using not only the momentary surrounding information received through the input unit 410 (headlight light of an oncoming vehicle, ambient illumination, street lights, etc.) but also accumulated surrounding information, , Control to change and display the color or saturation of HUD information.

In addition, the processor 430 controls only the HUD corresponding to the surrounding information received through the input unit 410 by using the color or saturation change display history information of the HUD information received from other vehicles traveling ahead according to the current driving route. Instead, it is possible to preemptively perform HUD control based on surrounding information expected based on location information.

The input unit 410 processes illuminance information that affects the driver's field of view in consideration of tinting of the vehicle through a camera sensor inside the vehicle, and transmits the processed information to the processor.

The processor 430 changes the HUD information to a color with a weak stimulus and controls the HUD information to be displayed, since external strong light has a strong stimulus intensity to a driver with high fatigue.

When the processor 430 analyzes the illuminance information and determines that the intensity of external light is greater than a preset value, the HUD information is changed to a color with relatively low saturation and is controlled to be displayed.

The processor 430 changes the HUD information by lowering the saturation of a preset color to a certain level to reduce the driver's fatigue when determining that the driver's fatigue is higher than the preset value and the intensity of external light is stronger than the preset value. control the expression.

The aforementioned illuminance information includes not only illuminance according to weather, but also horizontal plane illuminance and vertical illuminance, such as headlights from oncoming vehicles and illuminance from streetlights.

According to another embodiment of the present invention, it is possible to determine the driver's condition through the HMI equipment inside the vehicle, determine the color suitable for the driver's fatigue, correct the color through the HUD, and express the information.

According to another embodiment of the present invention, the saturation of the HUD color is adjusted in consideration of the intensity of light introduced into the vehicle according to the illumination of the vehicle driving environment and weather information.

According to another embodiment of the present invention, the driver's fatigue is reduced according to the display of the HUD change, and the HUD information is more accurately recognized.

5 illustrates a concept of HUD control using driver status and surrounding information according to another embodiment of the present invention.

The processor 430 uses driver's condition information as shown in (a) of FIG. 5 and illuminance information as shown in (b) of FIG. As shown, the HUD display method is determined.

As described above, the processor 430 changes and displays the HUD information in a color that is less stimulating to a driver with high fatigue, and controls the intensity of external light (weather, street lights, headlights of oncoming vehicles, etc.) using the illuminance information. If it is determined that the included illuminance information) is stronger than a predetermined value, the HUD information is changed to a color with relatively low saturation and is controlled to be displayed.

6 illustrates a HUD control method using a driver's condition and surrounding information according to another embodiment of the present invention.

According to another embodiment of the present invention, the HUD control method using the driver's state and surrounding information includes determining the driver's state (S510), determining external environment information (S520), and at least one of the driver's state and external environment information. A step of controlling at least one of HUD color, brightness, and position by using one (S530).

In step S510, state information related to the driver's fatigue is received from the driver's monitoring camera, the steering wheel grip pressure sensor, and the heart rate measurement sensor.

In step S520, illuminance and weather information around the vehicle are received through the vehicle's external camera sensor, and the external environment information is determined by processing the illuminance information that affects the driver's field of view in consideration of vehicle tinting through the vehicle's internal camera sensor. .

In step S530, since strong external light is a strong stimulus to a driver with high fatigue, HUD information is changed to a color with weak stimulus and is controlled to be displayed.

In step S530 , when it is determined that the intensity of external light is stronger than a predetermined value by analyzing the illuminance information, the HUD information is changed to a color with relatively low saturation and is controlled to be displayed.

In step S530, if it is determined that the driver's fatigue is higher than the preset value and the external light intensity is stronger than the preset value, the driver's fatigue is reduced, and the saturation of the preset color is lowered to a certain level to change the HUD information to be displayed. Control.

According to an embodiment of the present invention, when it is determined that the light intensity of a certain portion (refer to horizontal plane illumination) is stronger than a predetermined value by analyzing the illuminance information (eg, a situation caused by the headlights of an oncoming vehicle), the driver's The direction of the face is forward and the HUD information is displayed at a position where it is determined that there will be relatively little visual fatigue where the user can keep an eye on the driving lane as much as possible.

Claims (8)

an input unit that receives driver's gaze information;
a memory storing a program for determining an expression position of HUD information using the driver's gaze information; and
A processor for executing the program;
The processor determines the display position to display the HUD information in the gazing point area in consideration of the driver's gaze information, determines the gazing point using the driver's gaze information received through the input unit, and determines the driver's gaze Information is linked with driving information and monitored according to time sequence, and it is determined whether the change in the driver's gaze while driving is a change in the situation in which the display position of the HUD information needs to be changed, driving lane, route information, Even when the gazing point area deviates from the front in consideration of the driver's gaze information, when the possibility that the gazing point area will soon return to the forward direction is high, the HUD information is continuously displayed in the existing gazing point area;
The processor changes the display position of the HUD information including information on the driver's object of interest to the driver's gazing point in a signal waiting situation;
The processor stores information on an object of interest according to the driver's gesture, and provides an alarm to the driver when a place of the object of interest is located on a future driving route.
In HUD control system.
delete According to claim 1,
When the processor determines that the change of the expression position is unnecessary, determining that the driver's gazing point is maintained in the front area and maintaining the expression position
In HUD control system.
According to claim 1,
When the processor determines that the display position needs to be changed, determining that the current visual field of the driver is the gazing point, and changing the display position so that HUD information is displayed at the gazing point.
In HUD control system.
(a) tracking driver's gaze information;
(b) The change in the gaze information is monitored in conjunction with the driving situation, and it is determined whether or not the HUD information display position needs to be changed according to the change in the gaze information, and the HUD information display position is determined using the driver's gaze information. deciding; and
(c) displaying HUD information according to the determined display location;
In the step (b), the display position is determined to display the HUD information in the gaze point area in consideration of the driver's gaze information, but in a situation where the driver's gaze change during driving needs to change the display position of the HUD information. If it is determined whether it is a gaze change and there is a high possibility that the gazing point area will soon return to the forward direction even if the gazing point area deviates from the front by considering the driving lane, route information, and the driver's gaze information, The HUD information display position is determined so that the HUD information is continuously displayed in the viewpoint area, and the display position of the HUD information including information on the driver's object of interest is changed to the driver's gazing position in a traffic signal waiting situation. , Storing information on an object of interest according to the driver's gesture, and providing an alarm to the driver when the place of the object of interest is located on a future driving route.
In HUD control method.
delete According to claim 5,
In the step (b), when it is determined that the driver's gazing point has changed according to the change in the gaze information, the HUD information display position is changed so that an area corresponding to the gazing point becomes the HUD information display position.
In HUD control method. delete

Images