KR20240008735A - System for controlling display of vehicle and method thereof - Google Patents

Abstract

A system and method for controlling a display device of a vehicle are disclosed. A display device control system for a vehicle according to an aspect of the present invention includes a sensor installed on the top of each seat in the vehicle to acquire an image of the user, a display device installed in front of each seat in the vehicle, and a sensor around each display device. The user's intention is determined based on the installed sound module, the lighting module installed around each display device, and the image acquired through the sensor. If the user's intention is program control of the display device, the sound installed around the display device Listening feedback is output through the module and lighting module, a specific area of the display device where the user's gaze gazes for more than a preset certain period of time is enlarged, an interaction guide for the icon of the specific area is output through the display device, and a sensor It includes a processor that recognizes the user's gesture based on the image acquired through and controls a program corresponding to the icon based on the recognized gesture.

Description

Vehicle display device control system and method {SYSTEM FOR CONTROLLING DISPLAY OF VEHICLE AND METHOD THEREOF}

The present invention relates to a system and method for controlling a display device of a vehicle. More specifically, when controlling the GUI (Graphical User Interface) of a display device provided in a vehicle using a touchless interaction method, the user's intention is analyzed and the user's intention is analyzed. The present invention relates to a system and method for controlling a display device of a vehicle that informs the user and, if the analyzed intention is correct, controls the GUI of the display device through gesture interaction.

Depending on the risk of infectious diseases, technologies to minimize contact are emerging. In addition to these reasons, touchless technology has been considered important to increase driving safety inside vehicles. This is because drivers can focus on driving without having to directly touch and control other devices. Touchless interaction methods commonly used in vehicles include gesture recognition, face recognition, and voice recognition.

However, in the case of gesture recognition among touchless interaction methods, despite being touchless, it has multiple steps and depths due to its dependence on the display screen in the vehicle. In other words, it is the same as the existing touch method except that touch is not used as a means of pressing various menu steps. In the case of facial recognition, it is used to recognize the passenger and provide a customized environment suitable for the passenger, but this also has problems in terms of interaction because the process of asking the passenger's opinion is omitted and the setting is automatic. In the case of voice recognition, feedback is also provided by voice, which can cause problems such as misunderstanding or not remembering what was momentarily delivered.

This touchless interaction method is used to control display devices installed in vehicles. For example, when controlling a program through the GUI of a display device using a specific gesture, there is a problem of misrecognition because the specific gesture can occur even in general situations. In other words, it is not possible to know whether a specific gesture is intentionally generated to control the GUI of the display device or is generated due to a general situation.

Accordingly, when controlling the GUI of a display device installed in a vehicle using a touchless interaction method, the user's intention is analyzed and notified to the user, and if the analyzed intention is correct, a technology is developed that allows the display device's GUI to be controlled through gesture interaction. This is being demanded.

The background technology of the present invention is disclosed in Korean Patent Publication No. 10-2022-0062165 (published on May 16, 2022) titled ‘User interface device, vehicle having the same, and method of controlling the same.’

The present invention was created to improve the above-mentioned problems. The purpose of the present invention is to analyze the user's intention and inform the user when controlling the GUI of the display device installed in the vehicle through a touchless interaction method, and to inform the user of the analyzed intention. If correct, the present invention provides a system and method for controlling a display device of a vehicle that allows the GUI of the display device to be controlled through gesture interaction.

The problem to be solved by the present invention is not limited to the problem(s) mentioned above, and other problem(s) not mentioned will be clearly understood by those skilled in the art from the description below.

A display device control system for a vehicle according to an aspect of the present invention includes a sensor installed on the upper surface of each seat in the vehicle to obtain an image of the user, a display device installed in front of each seat in the vehicle, and around the display device. A sound module installed in, a lighting module installed around each display device, and the user's intention are determined based on the image acquired through the sensor, and when the user's intention is program control of the display device, the display device It outputs listening feedback through sound modules and lighting modules installed around it, enlarges a specific area of the display device where the user's gaze gazes for more than a preset certain period of time, and provides an interaction guide for the icon in the specific area to the display device. and includes a processor that recognizes the user's gesture based on the image acquired through the sensor and controls a program corresponding to the icon based on the recognized gesture.

In the present invention, the sensor may include a first sensor that acquires a first image for tracking the user's gaze, and a second sensor that acquires a second image for detecting the user's gesture.

In the present invention, the display device may be implemented as a pillar type.

The present invention further includes a photographing device mounted on the exterior of the vehicle by applying the angle of the display device, wherein the display device outputs an image captured through the photographing device on a screen in real time, and outputs the image. At least one icon for program control can be displayed on the screen.

In the present invention, the processor acquires a face image of the user from the image, detects an eye image from the face image, obtains the position of the pupil from the detected eye image, and recognizes the position of the user's gaze. By tracking the position of the gaze, the user's intention can be determined.

In the present invention, when the user's gaze gazes toward the display device for a preset first time or more, the processor recognizes program control of the display device as the user's intention, and turns on the lighting module or causes a flicker. By controlling it to do so, listening feedback indicating that the user's gesture interaction is in a state to be listened to can be output.

In the present invention, the display device is divided into a plurality of areas, and the processor, after outputting the listening feedback, when the user's gaze gazes at a specific area of the display device for a preset second time or more, the specific area can be enlarged to the center of the display device, and an interaction guide for controlling icons displayed in the enlarged area can be displayed through the display device.

In the present invention, when the user's gaze moves to a direction other than the display device after outputting the listening feedback and before a preset second time has elapsed, the processor deactivates the sound module and the lighting module and switches to standby mode. You can do it.

In the present invention, if the recognized gesture is a gesture corresponding to the interaction guide, the processor notifies that a normal operation has been performed through the sound module and the lighting module, executes a program corresponding to the icon, and executes the recognized gesture. If the gesture does not correspond to the interaction guide, it can be notified that an abnormal operation has been performed through the sound module and lighting module.

A method of controlling a display device of a vehicle according to another aspect of the present invention includes, by a processor, identifying a user's intention based on a first image acquired through a first sensor, wherein the user's intention is program control of the display device. In this case, the processor outputs listening feedback through sound modules and lighting modules installed around the display device, and enlarging a specific area of the display device where the user's gaze gazes for more than a preset certain period of time, the processor Outputting an interaction guide for an icon in a specific area through the display device and activating a second sensor, and the processor recognizing the user's gesture based on a second image acquired through the second sensor , and controlling a program corresponding to the icon based on the recognized gesture.

In the present invention, in the step of determining the user's intention, the processor performs face tracking or gaze tracking of the user based on the first image, and the user's face angle or gaze is maintained for a preset first time or more. When looking toward the display device, program control of the display device can be determined as the user's intention.

In the present invention, in the step of enlarging a specific area of the display device, the processor controls lighting modules installed around the display device to turn on or flicker, thereby indicating that the user's gesture interaction is in a state of listening. Listening feedback can be output.

In the step of controlling the program corresponding to the icon based on the recognized gesture, the processor, if the recognized gesture is a gesture corresponding to the interaction guide, normalizes the program through the sound module and the lighting module. Notifies that an operation has been performed, executes a program corresponding to the corresponding icon, and, if the recognized gesture is a gesture that does not correspond to the interaction guide, may notify that an abnormal operation has been performed through the sound module and lighting module.

In addition to this, other methods for implementing the present invention, other systems, and computer programs for executing the methods may be further provided.

The system and method for controlling a display device of a vehicle according to an embodiment of the present invention analyzes the user's intention through face tracking or gaze tracking when controlling the GUI of the display device provided in the vehicle through a touchless interaction method and provides information to the user. By notifying and controlling the GUI of the display device through gesture interaction if the analyzed intention is correct, highly intuitive long-distance non-contact interaction can be implemented.

Meanwhile, the effects of the present invention are not limited to the effects mentioned above, and various effects may be included within the range apparent to those skilled in the art from the contents described below.

1 is a diagram for explaining a vehicle display device control system according to an embodiment of the present invention.
FIG. 2 is an example diagram for explaining the installation location of the display device control system shown in FIG. 1.
Figure 3 is an example screen diagram for explaining the operation of a display device control system according to an embodiment of the present invention.
Figure 4 is a flowchart illustrating a method of controlling a display device of a vehicle according to an embodiment of the present invention.

Hereinafter, a system and method for controlling a display device of a vehicle according to an embodiment of the present invention will be described with reference to the attached drawings. In this process, the thickness of lines or sizes of components shown in the drawing may be exaggerated for clarity and convenience of explanation.

Additionally, implementations described herein may be implemented as, for example, a method or process, device, software program, data stream, or signal. Although discussed only in the context of a single form of implementation (eg, only as a method), implementations of the features discussed may also be implemented in other forms (eg, devices or programs). The device may be implemented with appropriate hardware, software, firmware, etc. The method may be implemented in a device such as a processor, which generally refers to a processing device that includes a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices such as computers, cell phones, portable/personal digital assistants (“PDAs”) and other devices that facilitate communication of information between end-users.

Additionally, the terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof. Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by these terms. The above terms are used only for the purpose of distinguishing one component from another.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, identical or corresponding components are assigned the same drawing numbers and duplicate descriptions thereof are omitted. I decided to do it.

FIG. 1 is a diagram illustrating a display device control system for a vehicle according to an embodiment of the present invention, FIG. 2 is an exemplary diagram illustrating the installation position of the display device control system shown in FIG. 1, and FIG. 3 is a diagram illustrating the present invention. This is an example screen diagram for explaining the operation of a display device control system according to an embodiment of.

Referring to FIG. 1, the vehicle display device control system 100 according to an embodiment of the present invention includes a sensor 110, a sound module 120, a lighting module 130, a display device 140, and a processor 150. ) and memory 160.

The sensor 110 is installed on the upper surface of each seat in the vehicle, acquires an image of the user, and transmits the acquired image to the processor 150. The sensor 110 may be implemented as an RGB camera, depth camera, dual camera, 360-degree camera, 2D video capture camera, depth camera, vision camera, etc.

This sensor 110 includes a first sensor 112 that acquires a first image for tracking the user's face or gaze, and a second sensor 114 that acquires a second image for detecting the user's gesture. can do.

The first sensor 112 is installed above the display device 140 as shown in FIG. 2 and can capture a portion of the user's face. The first sensor 112 may be implemented as, for example, an RGB camera.

As shown in FIG. 2, the second sensor 114 is installed on the top of the vehicle, particularly near the top of the display device 140, and can capture the user's hand. The second sensor 114 may be implemented, for example, as a camera capable of obtaining 3D depth.

The sound module 120 is installed around each display device 140 and can output sound under the control of the processor 150. That is, the sound module 120 may output listening feedback indicating that it is in a state to listen to the user's gesture interaction as a sound. For example, the sound module 120 may output the sound of “hung~” as listening feedback.

This sound module 120 may be implemented, for example, as a speaker.

The lighting module 130 is installed around each display device 140 and can perform lighting, blinking, plugging, etc. under the control of the processor 150. The lighting module 130 is a light that indicates that it can be controlled by a user, and may be installed behind the display device 140. For example, the lighting module 130 may display listening feedback indicating that it is in a state to listen to the user's gesture interaction as a flicker. This lighting module 130 may be implemented as indirect lighting such as LED.

The display device 140 may be installed in front of each seat in the vehicle, as shown in FIG. 2 . For example, in the case of a vehicle equipped with four seats, the display device 140 may be installed in front of each of the four seats.

The display device 140 may be implemented as a pillar type.

The display device 140 outputs images captured through a photographing device (not shown) mounted outside the vehicle on a screen in real time, and displays at least one icon for program control of the display device 140 on the video output screen. can be displayed. At this time, the photographing device can capture an image by applying the angle of the display device 140, and the display device 140 outputs the image captured through the photographing device, as if the display device 140 and the window of the vehicle are connected. It can be seen.

The display device 140 can display a GUI (graphics) on a real-time screen that is actually being captured through a photographing device. Here, GUI may be a concept including icons, scrollbars, buttons, etc.

The display device 140 may be a variety of display panels capable of displaying images, such as a Liquid Crystal Display (LCD) panel or a Light Emitting Diode (LED) panel.

The processor 150 determines the user's intention based on the image acquired through the sensor 110, and when the user's intention is program control of the display device 140, the sound module installed around the display device 140 Listening feedback is output through 120 and lighting module 130, enlarges a specific area of the display device 140 where the user's gaze gazes for more than a preset certain period of time, and displays an interaction guide for the icon in the specific area. It is output through the device 140, recognizes the user's gesture based on the image acquired through the sensor 110, and controls a program corresponding to the icon based on the recognized gesture.

Hereinafter, the operation of the processor 150 will be described in detail.

The processor 150 may recognize the user's face from the first image acquired through the first sensor 112 and determine the user's intention through face tracking or gaze tracking.

That is, the processor 150 obtains the information obtained through the first sensor 112. The user's face image can be obtained by removing the background area from the first image, the face area can be extracted from the obtained face image using edge and color information, and then face tracking can be performed. The processor 150 can determine the face angle through face tracking and determine the user's intention based on the face angle.

In addition, the processor 150 acquires the user's face image from the first image acquired through the first sensor 112, detects the eye image from the face image, and obtains the position of the pupil from the detected eye image to obtain the user's face image from the first image acquired through the first sensor 112. By recognizing the position of the user's gaze and detecting the direction of the gaze, the user's intention can be determined.

In addition, the processor 150 tracks the position of the user's eyes by detecting the positions of the user's eyes (e.g., pupil positions) from the first image acquired through the first sensor 112. You can. The processor 150 continuously receives the user's image through the first sensor 112 and can track the eye position (or pupil position) based on feature points detected therefrom.

In this way, the processor 150 detects the pupil from the first image acquired through the first sensor 112 and can confirm the direction of the user's gaze according to the movement of the pupil. In addition, the processor 150 detects the eyes, nose, and overall facial outline of the user's face from the first image acquired through the first sensor 112, and detects the user's face according to the relative positions of the detected eyes, nose, and overall facial outline. The direction of the face can be estimated. The processor 150 can determine the user's intention by performing face tracking or gaze tracking.

As a result of the user's face tracking or gaze tracking, when the user's face angle or gaze is directed toward the display device 140 as shown in (a) of FIG. 3, the processor 150 controls the program of the display device 140 can be determined by the user's intention. At this time, when the user's face or gaze remains toward the display device 140 for more than a preset first time (e.g., 2-3 seconds), the processor 150 controls the program of the display device 140 according to the user's intention. It can be understood as

When the program control of the display device 140 is identified as the user's intention, the processor 150 outputs listening feedback through the sound module 120 and the lighting module 130 in response to the identified intention, and performs the user's gesture. The second sensor 114 for recognition may be activated. At this time, the processor 150 may output listening feedback through the lighting module 130 installed around the display device 140 where the user's face angle or gaze is directed.

That is, when the program control of the display device 140 is determined to be the user's intention, the processor 150 sets the lighting module 130 installed around the display device 140 to A as shown in (b) of FIG. By controlling it to light up or flicker as shown, listening feedback indicating that the user's gesture interaction is in a state of listening can be output. Additionally, if the program control of the display device 140 is determined to be the user's intention, the processor 150 may output listening feedback indicating that the user's gesture interaction is in a state to be listened to as a sound through the sound module 120. For example, the sound module 120 may output a “hum” sound as listening feedback.

In this way, when the program control of the display device 140 is determined to be the user's intention, the processor 150 sends a signal to the sound module 120 and the lighting module 130 along with a transition indicating that it is in a state to listen to the user's gesture interaction. The user can be informed of the listening status. The user can check the listening feedback output through the lighting module 130 and the sound module 120 and confirm whether the user's intention is correct.

After outputting the listening feedback, as shown in (c) of FIG. 3, when the user's gaze maintains a specific area of the display device 140 for more than a preset second time (e.g., 2-3 seconds), the processor ( 150) enlarges the specific area to the center of the display device 140 as shown in (d) of FIG. 3, and provides an interaction guide for controlling the icon displayed in the enlarged area as shown in (e) of FIG. 3 to the display device ( 140). Here, the interaction guide may be a guide that instructs a gesture (C) to select (click, touch) an icon displayed in the enlarged area, as shown in (e) of FIG. 3. The user can perform gestures according to the gesture guide displayed in the interaction guide. Then, the program corresponding to the icon can be executed.

If, after outputting the listening feedback, the user's face angle or gaze moves to a direction other than the display device 140 before the preset second time (e.g., 2-3 seconds) has elapsed, the processor 150 If it is determined that the display device 140 will not be controlled, the sound module 120 and the lighting module 130 may be deactivated and converted to standby mode (idle state).

When the interaction guide is displayed through the display device 140 and the second sensor 114 is activated, the processor 150 may recognize the user's gesture based on the image acquired through the second sensor 114. . At this time, the processor 150 obtains an image of the user's body by removing the background area from the image acquired through the second sensor 114, and uses edge and color information from the acquired body image of the user to determine the user's face. , the arm, hand, torso, leg, and foot regions are extracted, and feature points are extracted from the extracted face, arm, hand, torso, leg, and foot to recognize the gesture taken by the user. Additionally, the processor 150 may process the user's gesture detected in the gesture area according to a gesture recognition algorithm. For example, the gesture area may mean the entire area where a user's gesture can be detected. For example, the processor 150 may recognize the corresponding gesture by checking the user's skeleton and recognizing joint movements. Additionally, the processor 150 can detect the user's pointing motion, the motion of moving the hand forward/backward/left/right, and the motion of drawing a triangle or circle with the hand.

If the recognized gesture is a gesture corresponding to the interaction guide, the processor 150 notifies that a normal operation has been performed through the sound module 120 and the lighting module 130 installed around the display device 140 and displays the corresponding icon. You can select and run it.

If the recognized gesture is a gesture that does not correspond to the interaction guide, the processor 150 may notify that an abnormal operation has been performed through the sound module 120 and lighting module 130 installed around the display device 140. there is.

For example, when a gesture corresponding to an interaction guide is recognized, the processor 150 may turn on the lighting module 130 in blue and output through the sound module 120 that the operation has been performed normally. If the gesture corresponding to the interaction guide is not recognized, the processor 150 may light the lighting module 130 in red and output that an abnormal operation has been performed through the sound module 120.

The memory 160 is a component that stores data related to the operation of the control system of the display device 140. In particular, the memory 160 detects the user's intention based on the image, outputs listening feedback in response to the identified intention, and identifies a specific area of the display device 140 where the user's gaze is maintained for more than a preset certain period of time. An application (program or applet) that enlarges, outputs an interaction guide for icons in a specific area, recognizes user gestures based on images, and controls programs corresponding to icons based on recognized gestures will be stored. The stored information can be selected by the processor 150 as needed. That is, the memory 160 stores various types of data generated during the execution of an operating system for driving the display device 140 control system or a display device control application (program or applet). At this time, the memory 160 refers to a non-volatile storage device that continues to retain stored information even when power is not supplied, and a volatile storage device that requires power to maintain the stored information. Additionally, the memory 160 may perform a function of temporarily or permanently storing data processed by the processor 150. Here, the memory 160 may include magnetic storage media or flash storage media in addition to volatile storage devices that require power to maintain stored information, but the scope of the present invention is limited thereto. It doesn't work.

Figure 4 is a flowchart illustrating a method of controlling a display device of a vehicle according to an embodiment of the present invention.

Referring to FIG. 4, the processor 150 performs face tracking or gaze tracking of the user based on the first image acquired through the first sensor 112 (S402). That is, the processor 150 acquires the user's face image from the first image acquired through the first sensor 112, detects the eye image from the face image, and obtains the position of the pupil from the detected eye image to determine the user's face image. It is possible to recognize the position of the gaze and track the position of the recognized gaze. Additionally, the processor 150 may obtain a face image from the first image acquired through the first sensor 112, extract a face area from the face image, and perform face tracking.

When step S402 is performed, the processor 150 determines whether the user's face angle or gaze gazes toward the display device 140 for a preset first time (S404).

As a result of the determination in step S404, if the user's face angle or gaze gazes toward the display device 140 during the first time, the processor 150 determines the program control of the display device 140 as the user's intention (S406) . Listening feedback is output through the sound module 120 and the lighting module 130 installed around the display device 140 (S408).

That is, when the program control of the display device 140 is determined to be the user's intention, the processor 150 controls the lighting module 130 installed around the display device 140 to turn on or flicker, thereby Listening feedback indicating that the gesture interaction is in a state of listening can be output. Additionally, if the program control of the display device 140 is determined to be the user's intention, the processor 150 may output listening feedback indicating that the user's gesture interaction is in a state to be listened to as a sound through the sound module 120.

When step S408 is performed, the processor 150 determines whether the user's gaze gazes at a specific area of the display device 140 for more than a preset second time (eg, 2-3 seconds) (S410).

As a result of the determination in step S410, if the user's gaze gazes at a specific area of the display device 140 for a second time or more, the processor 150 enlarges the specific area to the center of the display device 140 (S412) and enlarges the specific area. An interaction guide for controlling the icon displayed in the area is displayed through the display device 140 (S414), and the second sensor 114 for recognizing the user's gesture is activated (S416).

When step S416 is performed, the processor 150 recognizes the user's gesture based on the second image acquired through the second sensor 114 (S418). At this time, the processor 150 can detect the user's pointing motion, the motion of moving the hand forward/backward/left/right, and the motion of drawing a triangle or circle with the hand.

When step S418 is performed, the processor 150 determines whether the recognized gesture is a gesture corresponding to the interaction guide (S420).

As a result of the determination in step S420, if the recognized gesture is a gesture corresponding to the interaction guide, the processor 150 performs a normal operation through the sound module 120 and the lighting module 130 installed around the display device 140. This is notified (S422), and the program corresponding to the icon is executed according to the gesture (S424).

If the recognized gesture is a gesture that does not correspond to the interaction guide, the processor 150 notifies that an abnormal operation has been performed through the sound module 120 and the lighting module 130 installed around the display device 140 ( S426).

As a result of the determination in step S410, if the user's face angle or gaze does not gaze at a specific area of the display device 140 for more than a second time, the processor 150 determines the sound module 120 installed around the display device 140. and deactivates the lighting module 130 and switches to standby mode (S428).

As described above, the system and method for controlling a display device of a vehicle according to an embodiment of the present invention detects the user's intention through face tracking or gaze tracking when controlling the GUI of a display device provided in a vehicle using a touchless interaction method. By analyzing and notifying the user, and controlling the GUI of the display device through gesture interaction if the analyzed intention is correct, highly intuitive long-distance non-contact interaction can be implemented.

The present invention has been described with reference to the embodiments shown in the drawings, but these are merely exemplary, and those skilled in the art will recognize that various modifications and other equivalent embodiments are possible therefrom. You will understand. Therefore, the true technical protection scope of the present invention should be determined by the scope of the patent claims below.

100: Display device control system
110: sensor
112: first sensor
114: second sensor
120: sound module
130: lighting module
140: display device
150: processor
160: memory

Claims (13)

A sensor installed on the upper surface of each seat in the vehicle to acquire an image of the user;
a display device installed in front of each seat in the vehicle;
A sound module installed around each display device;
Lighting modules installed around each display device; and
Determine the user's intention based on the image acquired through the sensor, and if the user's intention is program control of the display device, output listening feedback through a sound module and a lighting module installed around the display device, Enlarges a specific area of the display device where the user's gaze gazes for more than a predetermined period of time, outputs an interaction guide for the icon of the specific area through the display device, and outputs the display device based on the image acquired through the sensor. A processor that recognizes the user's gestures and controls a program corresponding to the icon based on the recognized gestures.
A display device control system for a vehicle including a.
According to paragraph 1,
The sensor is,
A first sensor that acquires a first image for tracking the user's gaze; and
A display device control system for a vehicle, comprising a second sensor that acquires a second image for detecting the user's gesture.
According to paragraph 1,
The display device is,
A display device control system for a vehicle, characterized in that it is implemented in a pillar type.
According to paragraph 1,
Further comprising a photographing device mounted on the outside of the vehicle by applying the angle of the display device,
A display device control system for a vehicle, wherein the display device outputs an image captured through the photographing device on a screen in real time, and displays at least one icon for program control on the image output screen.
According to paragraph 1,
The processor,
Obtain the user's face image from the image, detect the eye image from the face image, obtain the position of the pupil from the detected eye image, recognize the position of the user's gaze, and track the position of the gaze. ), a vehicle display device control system characterized by identifying the user's intention.
According to paragraph 1,
The processor,
When the user's gaze gazes toward the display device for a preset first time or more, the program control of the display device is recognized as the user's intention, and the lighting module is controlled to light or flicker, thereby A display device control system for a vehicle, characterized in that it outputs listening feedback indicating that it is in a state to listen to gesture interaction.
According to paragraph 1,
The display device is divided into a plurality of areas,
The processor,
After outputting the listening feedback, when the user's gaze gazes at a specific area of the display device for a preset second time or more, the specific area is enlarged to the center of the display device and an icon displayed in the enlarged area is displayed. A display device control system for a vehicle, characterized in that an interaction guide for control is displayed through the display device.
According to paragraph 1,
The processor,
After outputting the listening feedback, if the user's gaze moves to a direction other than the display device before a preset second time has elapsed, the sound module and the lighting module are deactivated and switched to standby mode. Display device control system.
According to paragraph 1,
The processor,
If the recognized gesture is a gesture corresponding to the interaction guide, notifying that a normal operation has been performed through the sound module and lighting module and executing a program corresponding to the icon,
A display device control system for a vehicle, characterized in that when the recognized gesture is a gesture that does not correspond to the interaction guide, it notifies that an abnormal operation has been performed through the sound module and the lighting module.
A step of the processor identifying the user's intention based on the first image acquired through the first sensor;
When the user's intention is to control the program of the display device, the processor outputs listening feedback through sound modules and lighting modules installed around the display device, and the display device on which the user's gaze gazes for more than a preset certain period of time enlarging a specific area;
The processor outputting an interaction guide for icons in the specific area through the display device and activating a second sensor; and
Recognizing the user's gesture based on a second image acquired through the second sensor by the processor, and controlling a program corresponding to the icon based on the recognized gesture
A method of controlling a display device of a vehicle comprising:
According to clause 10,
In the step of determining the user's intention,
The processor performs face tracking or gaze tracking of the user based on the first image, and when the user's face angle or gaze gazes toward the display device for a preset first time or more, program control of the display device A method of controlling a display device of a vehicle, characterized in that the user's intention is identified.
According to clause 10,
In the step of enlarging a specific area of the display device,
The processor controls lighting modules installed around the display device to turn on or flicker, thereby outputting listening feedback indicating that it is in a state to listen to the user's gesture interaction. .
According to clause 10,
In the step of controlling the program corresponding to the icon based on the recognized gesture,
When the recognized gesture is a gesture corresponding to the interaction guide, the processor notifies that a normal operation has been performed through the sound module and the lighting module and executes a program corresponding to the icon,
When the recognized gesture is a gesture that does not correspond to the interaction guide, a method of controlling a display device of a vehicle, characterized in that notifying that an abnormal operation has been performed through the sound module and the lighting module.