KR102028757B1 - Method for display in vehicle using head up display and apparatus thereof - Google Patents

Abstract

The present invention relates to a display device and method for a construction machine and a vehicle using a head up display (HUD).
The present invention displays the screen of the HUD according to the user's eye level of the construction machinery or vehicle, and also adaptively moves and displays the HUD screen in the direction of the user's gaze according to the user's gaze movement, and also according to the user's taste By displaying the screen with a sense of distance (aka display distance), the user's work view is secured and work efficiency is improved.

Description

Display method and apparatus of vehicle using head-up display {METHOD FOR DISPLAY IN VEHICLE USING HEAD UP DISPLAY AND APPARATUS THEREOF}

The present invention relates to a display of a construction machine vehicle, and more particularly to a display method and apparatus using a head-up display.

Construction machinery is used in civil engineering or building construction, and types of excavators, graders, cranes and transport machinery. In general, construction machinery is operated by operating a handle, pedal, stick or joystick. In addition, recently produced construction machinery is equipped with a display device, and displays the status and important data of the equipment to the user (that is, the worker) through the display device of the construction machine. Therefore, the user of the construction machine can operate the construction machine efficiently by using the status and information of the equipment displayed on the display. In particular, a head up display (HUD) has been in the spotlight as a medium for effectively transmitting vehicle driving information and surrounding situation information to the driver in order to secure driving safety while driving a vehicle driver (ie, a user or an operator). .

The head-up display (HUD) of a vehicle is a display device that provides vehicle driving information or other information within the driver's front face, that is, within the range of the driver's main line of sight while driving the vehicle. It has recently been installed in vehicles.

The vehicle's head-up display (HUD) displays the instrument panel information (speed, mileage, RPM, navigation information, warnings, etc.) on the driver's line of sight on the windshield of the vehicle, allowing the driver to easily identify driving information while driving. Make sure This allows the driver to keep safe driving by recognizing important driving information without taking his eyes off the road ahead.

However, the prior art of the HUD applied to the vehicle has a disadvantage that can not eliminate the effects of diffuse reflection and shading of the light because it projects a fixed screen in one direction on the front window of the operator. Such diffuse reflection and shadow of the light causes a problem in securing the worker's vision, which can be a factor that reduces the worker's work efficiency.

On the other hand, the vehicle driver (worker) can move along the working area between the projected surface of the HUD (ie, the front windshield in the cabin) and the working area (ie, the working area outside the cabin). Such movement of the driver's gaze may reduce the work efficiency of the driver in operation and may also cause a safety accident.

The technical problem to be solved by the present invention is to search (capture) the eyes of the driver of the vehicle, and to move and display work information and other instrument panel information in an optimal direction at the eye level of the driver according to the movement of the eyes. .

In addition, the technical problem to be solved by the present invention, by using the HUD to display the sense of distance desired by the driver, to improve the efficiency of the work and to prevent safety accidents.

In order to solve the above technical problem, a display method of a vehicle using a head-up display according to the present invention, the step of determining the user's face and eyes by processing the image captured by at least one or more cameras, Extracting location information of a user's eyes by processing the captured image, changing a reflection angle of a head up display (HUD) module using the extracted location information, and changing the HUD module Projecting the screen to the window of the vehicle at a reflection angle of the.

Preferably, the method further comprises: searching whether the user's gaze has moved, and if the user's gaze has moved, continuously focusing and tracking the pupil of the user, and according to the user's gaze movement, The method may further include extracting location information.

Preferably, searching for whether the display distance is set or changed, and if the display distance is set or changed, image processing to project the screen of the HUD module onto the vehicle window with the set or changed display distance, and the image processing. And projecting the screen of the HUD module onto the vehicle window.

Preferably, the location information includes distance information and direction information between the tracking camera and the pupil of the user.

Preferably, in the step of changing the reflection angle of the HUD module, the reflection angle of the HUD module is characterized in that the angle at which the screen of the HUD module can be projected on the vehicle window perpendicular to the eyes of the user.

In addition, in order to solve the technical problem of the invention as described above, the display device of the vehicle using the head-up display according to the present invention, to capture the image of the user with at least two or more small camera, and to track the eyes of the user Head-up display (HUD) module with adjustable tracking camera, reflection angle of the image projected on the window of the vehicle, and instrument panel for setting or changing the display distance of the screen of the HUD module to be projected on the vehicle window A user interface (UI) and an image of the user captured by the tracking camera are processed to discriminate the user's face and eyes, and the location information of the user's pupil is extracted using the processed and determined user's image, and the extracted It includes a control unit for adjusting the direction and the reflection angle of the screen to be projected on the vehicle window using the location information And a gong.

Preferably, the controller detects whether there is a setting or change of the display distance sense from the instrument panel UI, and if there is a setting or change of the display distance sense, image processing the screen of the HUD module corresponding to the setting or the change. And control the screen of the image-processed HUD module to be projected onto the vehicle window.

Preferably, the location information, characterized in that it comprises the distance information and direction information between the tracking camera and the pupil of the user.

Preferably, the controller is characterized by adjusting the direction and the reflection angle of the HUD module screen by adjusting the reflection lens of the HUD module to the direction of the screen to be projected on the vehicle window.

Preferably, the control unit controls the HUD module, characterized in that the screen of the HUD module alpha-branded to display on the vehicle window.

The present invention adaptively displays vehicle information and work information, etc., in a window of a vehicle in a direction that is optimal for the user's gaze according to the movement of the user's eyes, thereby preventing the effect of diffuse reflection and shadow of the light of the user's work. can do.

In addition, the present invention, even if the user moves three-dimensionally in the cabin, as the user's gaze movement is recognized and tracked, the screen in the optimal direction (for example, perpendicular to the direction of the user's gaze) of the user's gaze. While displaying, the user can set the sense of distance of the displayed screen, so that the user can work efficiently and conveniently.

1 is a view illustrating a display apparatus using a head up display (HUD) according to an embodiment of the present invention.
2 is a view illustrating a method of using a display device using a HUD according to an embodiment of the present invention.
3 illustrates an example in which the display device using the HUD according to the present invention projects the screen of the HUD module onto a vehicle window.
4 is a flowchart illustrating a display method of a display apparatus using a HUD according to an embodiment of the present invention.

The present invention is applied to vehicles and construction machinery using the HUD. However, the present invention is not limited thereto and may be applied to all technical fields to which the technical spirit of the present invention can be applied.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Hereinafter, in the description and the accompanying drawings, the substantially identical components are represented by the same reference numerals, and thus redundant description will be omitted. In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

On the other hand, the term described in the present invention, the term "and / or" includes any item of a plurality of related items described or a combination of a plurality of items described.

The technical term "display device" used in the present invention is a generic term for a device that provides an analog or digital display device, and may be abbreviated "display" or "monitor" or "board". The "display device" of the present invention may also be a device composed of one or more monitors or instrument clusters. In addition, the "display device" of the present invention may be a device integrated or independent of the control unit (or processor) for operating the "display".

The technical term "construction machine" used in the present invention is not only limited to the machine used literally for construction purposes, but also a generic meaning of all vehicles to which the joystick and the display device according to the present invention are applied.

The basic concept of the present invention is to display the screen of the HUD according to the user's eye level of the construction machine or the vehicle, and also to display the HUD screen by adaptively moving in the direction of the user's gaze according to the user's gaze movement. By displaying the screen with a desired distance sense (also called display distance sense) according to taste, it is possible to secure a working view of a user and further improve work efficiency. The concept of the present invention described above includes: 1) two or more cameras capable of capturing the face and gaze of a user; 2) a control unit (or image processor) capable of measuring position information (distance and direction) with a user from images captured by two or more cameras; 3) The reflection angle of the screen of the HUD displayed (projected) on the vehicle window may be implemented through the HUD module.

1 is an embodiment of the present invention, a display device using a HUD according to the present invention.

As shown in FIG. 1, the display apparatus 100 using the HUD according to the present invention includes two or more small cameras, and captures an image of a user, and focuses and tracks a user's pupils. And the HUD module 120 in which the reflection angle of the reflective lens can be adjusted or adjusted so that the direction and / or angle of the screen (image) to be projected onto the window of the vehicle (for example, the cabin window) can be changed. Processing the image (screen) of the user captured from the tracking camera 110 to recognize and determine the user's face and eyes, extract the location information of the user's pupil using the determined user's image, and extract the location By using the information, the controller 130 adjusts the direction of the image to be projected on the vehicle window to the reflection angle of the HUD module, and sets or changes the display distance of the image projected on the vehicle window. It is configured to include an instrument panel UI (user interface) (140).

The HUD module 120 has a general configuration and includes a backlight unit, an LCD panel, a lens, an LCD controller board, a lens essay, and a reflective lens. In addition, the HUD module 120 may further configure a device that can adjust the reflection angle to project the image in all directions (that is, up, down, left, right, front and back, three-dimensional moving direction). For example, the HUD module 120 may include a drive motor and a mount capable of moving and rotating the reflective lens in the up, down, left, and right directions. As another embodiment, the HUD module 120 may include the controller 130, or may be configured independently.

Hereinafter, the function and configuration of a display apparatus using a HUD according to the present invention will be described in more detail with reference to FIG. 1.

The tracking camera 110 is composed of at least two small cameras, and is mounted on the front window in the construction machine cabin. The two cameras track the user's face and eyes moving in three dimensions: up, down, left, right and front. In other words, sitting in a seat in the cabin captures the face and eyes of the user in operation, and transmits the captured image to the controller 130. In addition, when the controller 130 analyzing the transmitted captured image transmits a control signal (for example, information on eye position and a control signal related to focusing), the tracking camera 110 receives the tracking signal and focuses on the pupil of the user. focus and continuously track the user's eyes. For example, if the user watches the "A work area" outside the cabin and turns his head to move to the "B work area" or the "C area" in the cabin, the tracking camera 110 may control the controller 130. The control signal of the user focuses on the eyes of the user to track the movement of the user's eyes.

The controller 130 analyzes the captured image of the user from the tracking camera 110 (image processing) to determine the user's face and eyes. In addition, after determining the pupils of the user, the controller 130 extracts the position information (information about distance and direction, etc.) of the user pupils from the images captured by the two cameras, and processes the extracted position information to process the tracking camera. The tracking camera 110 is controlled to continuously track the eyes of the user. In this case, the controller 130 may control each of the tracking cameras 110 separately or integrally.

The controller 130 adjusts the reflection angle of the HUD module 120 by using the position information acquired by processing the captured image (ie, the position information of the user's eyes), so that the screen of the HUD module is adjusted to the eye level of the user. Control the HUD module 120 to be projected. That is, for example, suppose that the user looks at the "work area A" outside the cabin and turns his head to the work area "B" while working. At this time, the tracking camera 110 consisting of two cameras will continue to capture the image (video) of the user's face and eyes, the control unit processes the captured images to extract the position information about the user's eyes . The controller 130 uses the extracted location information to adjust the reflection angle of the HUD module so that the screen of the HUD module 120 is displayed in the cabin window on the copper line of the gaze of the "B work area" where the main user's main gaze stays. Adjust That is, the controller 130 projects the screen of the HUD module 120 on the cabin window on the copper line of the user's gaze as the location information changes. For example, the controller 130 may control the mount and / or the reflective lens in the HUD module 120 to adjust the reflection angle of the screen projected on the cabin window.

On the other hand, when the user selects (or sets) a sense of distance according to his / her preference with the instrument panel UI 140, the controller 130 projects the screen of the HUD module 120 suitable for the sense of distance selected by the user to the cabin window. The screen image of 120 is processed, and the HUD module 120 is controlled so that the processed screen image is projected by the HUD module 120 to the cabin window.

In addition, the controller 130 may alphablend the image of the screen projected by the HUD module 120 to the cabin window. Here, alpha branding is a technique of expressing a transmission image by adding a variable α representing transparency to general image data.

2 is a view illustrating a method of using a display device using a HUD according to an embodiment of the present invention.

As shown in FIG. 2, the tracking camera 110 composed of two small cameras according to the present invention may be mounted on the front window 21 of the cabin 20. The tracking camera 110 recognizes and tracks the eyes of the user whenever the user (ie, the driver) 22 moves his / her eyes, and focuses on the eyes of the user.

On the other hand, the HUD module 120 may be installed on the instrument cluster cluster (23). As the angle of the reflective lens of the HUD module 120 changes, the reflection angle (projection angle) of the screen projected on the front window 21 of the cabin 20 is converted. As described above, the reflection angle of the screen is converted by the control of the controller 130.

3 illustrates an example in which the display device using the HUD according to the present invention projects the screen of the HUD module onto a vehicle window.

As shown in FIG. 3, a tracking camera 110 consisting of two small cameras in a vehicle window (that is, a cabin window of a construction machine) focuses and tracks a user's eyes, and tracks the user's field of view on a moving line. The screen of the HUD module (ie, work information, or vehicle information, or work guide, etc.) is displayed in the vehicle window at.

4 is a flowchart illustrating a display method of a display apparatus using a HUD according to an embodiment of the present invention.

1 to 4, the image captured by two cameras (that is, the tracking camera) is processed to recognize and determine a user's face and eyes (S410). The captured image is processed to extract location information about the user's eyes (that is, information about the distance and direction of the user's eyes and each camera, for example) (S420).

The controller 130 loads information on the distance and direction between the user's pupil and the camera by using the extracted position information (S430), and adjusts the reflection angle of the HUD module 120 according to the loaded information. By changing the projection of the screen of the HUD module 120 on the cabin window (S440). In the step S440, the screen of the HUD module 120 projected on the cabin window is viewed at right angles to the user's line of sight, thereby eliminating diffuse reflection and shadow of light to provide an optimal view to the user.

 The controller 130 detects whether the user's gaze is moved (changed) during the work in the cabin (S450). If the user's gaze has moved, the camera 110 continues to focus and track the user's eyes, and the controller 130 executes step S410 in order to extract the location information of the user's eyes as the user moves. do.

The controller 130 searches whether the user sets or changes the display distance through the instrument panel UI 140 (S460). If the display distance is set or changed, the controller 130 processes the screen of the HUD module 120 so that the screen of the HUD module 120 is projected on the cabin window according to the set distance, and processes the processed HUD. The screen of the module 120 is controlled to be projected on the cabin window (S470).

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium may include a storage medium such as a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, etc.) and an optical reading medium (eg, a CD-ROM, a DVD, etc.).

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

Claims (9)

Capturing a user's face and eyes with at least one tracking camera 110 to obtain a captured image;
Extracting location information of the pupil of the user by analyzing the obtained captured image;
Changing a reflection angle of a head up display (HUD) module 120 using the extracted position information;
Projecting the screen onto the window of the vehicle at the reflection angle of the modified HUD module 120
Including,
In the step of changing the reflection angle,
Changing the reflection angle of the HUD module 120 so that the user's line of sight is perpendicular to the image plane projected on the vehicle window. The method of claim 1,
After projecting the screen onto a window of the vehicle,
Searching whether the user's gaze has moved;
If the user's gaze has moved, focusing and tracking the pupil of the user;
Extracting location information of the pupil of the user according to the movement of the user's eyes
Display method of the vehicle using the head-up display, characterized in that it further comprises. The method of claim 1,
Searching whether the display distance is set or changed;
If the display distance is set or changed, image processing is performed such that the screen of the HUD module 120 is projected onto the vehicle window with the set or changed display distance;
And projecting the screen of the image-processed HUD module (120) onto a vehicle window. The method of claim 1, wherein the location information
Includes distance information and pupil direction information between the user's pupil and the tracking camera 110,
The tracking camera 110 tracks the eyes of the user and captures the image of the user. delete A tracking camera 110 for tracking the eyes of the user with at least two small cameras and capturing an image of the user;
A head up display (HUD) module 120 in which a reflection angle of an image to be projected on a window of the vehicle can be adjusted;
An instrument panel UI (140) for setting or changing the display distance of the screen of the HUD module to be projected on the vehicle window;
The tracking camera 110 processes the captured image of the user to determine the user's face and eyes, extracts the location information of the user's pupil using the processed and determined user's image, and uses the extracted location information. And a controller 130 for adjusting a direction and a reflection angle of a screen to be projected onto the vehicle window.
The head up display (HUD) module 120 changes the reflection angle of the HUD module 120 so that the user's line of sight is perpendicular to the image plane projected on the vehicle window. Vehicle display device 100 using. The method of claim 6, wherein the controller 130
Search whether there is a setting or change in the display distance from the instrument panel UI 140,
If there is a setting or change of the display distance, image processing of the screen of the HUD module 120 corresponding to the setting or the change, and control of the screen of the image processed HUD module 120 to be projected on the vehicle window Vehicle display device 100 using a head-up display characterized in that. The method of claim 6, wherein the location information
Includes distance information and pupil direction information between the user's pupil and the tracking camera 110,
The tracking camera 110 tracks the eyes of the user to capture the image of the user, the display device of the vehicle (100) using a head-up display. The method of claim 6, wherein the controller 130
Display apparatus of the vehicle 100 using the head-up display, characterized in that for adjusting the direction of the screen to be projected on the vehicle window by adjusting the reflection lens of the HUD module 120, the direction and the reflection angle of the screen of the HUD module 120. ).

Images