KR101603551B1 - Method for executing vehicle function using wearable device and vehicle for carrying out the same - Google Patents

Abstract

The present invention relates to, in selecting and performing various functions in a vehicle, a method of using a wearable mobile device which is worn by a driver and a vehicle which can carry out the same. To solve the above purpose, according to an embodiment of the present invention, a method for executing a vehicle function using smart glasses, comprises: a step of connecting with the smart glasses wirelessly; a step of transmitting a first image including a list of at least one executable function with the smart glasses; a step of receiving information indicating a specific location of the first image from the smart glasses; and a step of executing a function corresponding to the specific location among the at least one executable function when a selected signal inputted from a mechanical system is transmitted, after the information is received.

Description

TECHNICAL FIELD [0001] The present invention relates to a method of performing a function using a wearable device in a vehicle,

The present invention relates to a method of using a mobile device worn by a driver in selecting and executing various functions in a vehicle, and a vehicle capable of performing the method.

The frequency of use of the multimedia system in the vehicle is increasing, and the function of the multimedia system of the vehicle is becoming more complicated according to the needs of the user. However, in order to operate a multimedia system of a vehicle having various functions, the user's gaze may frequently be diverted from the front, and the risk of a vehicle accident increases as the frequency of departing from the front increases.

The current multimedia system operation method is divided into a screen touch method and an operation system such as a button or a jog dial. One example of such manipulation methods is shown in Fig.

1 shows an example of a multimedia system configuration in a vehicle.

Referring to FIG. 1, a visual point of a multimedia function during a driving operation of a vehicle, and a function of the user, may include a display of a head up display (HUD) 110 or an AVN (Audio, Video, Navigation) A key button in the vicinity thereof, an operating system 130 around the speed change gear, and an operating system 140 of the handle. Of these, the heads-up display 110 and the steering system 140 of the handle are likely to cause problems in safety because they are oriented in the same direction as the direction of the eyes to be watched by the user during operation. However, when the driver manipulates the display system of the AVN system 120, the key buttons around the AVN system 120, and the operating system 130 around the shift gear, the driver is difficult to prevent the driver from looking ahead.

The present invention is intended to provide a method by which a user can select and execute a desired function without departing from the front of the driver and a vehicle capable of performing the method.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

According to another aspect of the present invention, there is provided a method for an audio / video navigation (AVN) system of a vehicle, the method comprising: And wirelessly; Transmitting, through the wireless communication unit, a first image including the list of at least one function executable in the AVN system to the smart glasses; Receiving information indicating a specific position of the first image from the smart glasses through the wireless communication unit; And executing a function corresponding to the specific one of the at least one executable function in the controller of the AVN system when a selection signal inputted through the operating system is received after the information is received.

According to another aspect of the present invention, there is provided an audio-visual navigation system (AVN) of a vehicle, comprising: a wireless communication unit wirelessly connected to smart glasses; A wired communication unit provided on a handle of the vehicle, the wired communication unit being connected to an operating system for receiving a command from a driver; And a first image including a list of at least one function executable in the AVN system is transmitted through the wireless communication unit with the smart glasses, and information indicating a specific position of the first image from the smart glasses is received And a controller for executing a function corresponding to the specific position among the at least one executable function when the input signal inputted through the operating system is transmitted.
Further, a method of controlling smart glasses according to an exemplary embodiment of the present invention includes the steps of wirelessly connecting with an audio / video navigation system (AVN) of a vehicle through a wireless communication unit; Receiving from the AVN system a first image including a list of at least one function executable in the AVN system through the wireless communication unit; Displaying the received first image through a display unit in the form of a head mounted display (HMD); Determining a gaze direction of the user with a second image photographed through a camera arranged to photograph the user's eyes from the control unit; Acquiring a specific position in the first image using the determined viewing direction; And transmitting information on the obtained specific location to the AVN system through the wireless communication unit.

delete

In addition, the smart glasses according to an embodiment of the present invention can be connected to an audio / video navigation (AVN) system of a vehicle wirelessly and transmit a first image including a list of at least one function executable in the AVN system to the AVN system A wireless communication unit receiving the wireless communication signal; A display unit for displaying the received first image in the form of a head mounted display (HMD); A camera arranged to photograph a user's eyes; And acquiring a specific position in the image by using the determined gaze direction, and acquiring information on the obtained specific position from the AVN system To be transmitted to the mobile station.

According to at least one embodiment of the present invention, there are the following effects.

It simplifies the input system by unifying the distributed operating system of the multimedia system into one place and removing a large number of buttons by transmitting functions of the AVN screen using the smart glasses, selecting the function through the sight detection, and executing the function through the operating system located at the driver's front .

In addition, since the screen is displayed through the smart glasses so that the eye can watch ahead, and the desired function is selected through the eye information, it is possible to avoid departure from the front view.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

1 shows an example of a multimedia system configuration in a vehicle.
2 shows an example of a process of selecting and executing a vehicle function using a wearable device according to an embodiment of the present invention.
3 is a flowchart illustrating an example of a process of selecting and executing a vehicle function using a wearable device according to an embodiment of the present invention.
4 is a block diagram showing an example of the configuration of an AVN system according to an embodiment of the present invention.
5 shows an example of smart glasses that can be applied to embodiments of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals are used to designate identical or similar elements, and redundant description thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

In the embodiment of the present invention, when a wearer wears a glass-type wearable device (i.e., smart glasses) having a camera facing the driver's eyes, the screen displayed in the AVN system is transmitted to the smart glasses, And the user's eyesight direction watching the screen displayed on the smart glasses is read by the camera to determine the desired function to be executed by the driver. If the execution command is inputted through the vehicle or the smart glasses while the execution desired function is determined, the AVN system can execute it. For this purpose, the AVN system of the vehicle is preferably wirelessly connected to the driver's smart glasses. In addition, the function may be performed through operation of a key button or a touch sensor provided in the smart glasses, or may be performed through operation of any one of the operation systems provided in the handle have.

Hereinafter, the above-described embodiment will be described in more detail with reference to FIG.

2 shows an example of a process of selecting and executing a vehicle function using a wearable device according to an embodiment of the present invention.

In FIG. 2, the smart glasses worn by a driver assume a state in which a connection is completed through a wireless communication system such as Bluetooth or Wi-Fi with a wireless communication system connected to an AVN system or AVN system of a vehicle supporting a wireless connection function.

Referring to FIG. 2 (a), the smart glasses 200 worn by the driver are provided with a camera in the direction toward the eyes of the driver, so that the direction of the driver's eyes can be sensed.

When the driver desires to execute a specific function during operation, a specific key button can be operated in the operating system or smart glasses provided on the handle. Accordingly, the AVN system 120 can transmit its own screen image 120 'including executable functions to the smart glasses through wireless connection as shown in FIG. 2 (b). The screen image 120 'may display a list of at least one executable function. Here, the list may be a form in which icons corresponding to respective functions are listed, or text, an image, or a combination thereof. In the smart glasses, the received screen image 120 'can be displayed through its own display. Here, when the driver watches the area 121 'corresponding to the position of the desired execution function in the image 120' displayed on the display of the smart glasses, the smart glasses determine the direction of the driver's gaze through the camera , And may transmit information on the determined direction to the AVN system 120. [

Accordingly, the AVN system can display a predetermined type of visual effect (for example, highlight, coloring, icon size / shape change, etc.) in the selection area corresponding to the driver's gaze direction. The screen image of the AVN system having such a visual effect can be transmitted to the smart glasses again, and the smart glasses can display the selected area to the driver through the display thereof through the display thereof.

When a visual effect is given to a position at which an execution desired function is displayed, the driver confirms that his / her line of sight is correctly recognized and operates a specific key button of the operating system 140 provided in the handle as shown in Fig. 2 (c) When the operation signal of the key button is transmitted to the AVN system 120, the AVN system 120 can execute the corresponding function.

The above-described process will be described in the form of a flowchart with reference to FIG.

3 is a flowchart illustrating an example of a process of selecting and executing a vehicle function using a wearable device according to an embodiment of the present invention.

Referring to FIG. 3, as the smart glasses of the driver are connected to the AVN system, the screen of the AVN system is transmitted to the display of the smart glasses, and the received screen is displayed through the display (S310).

The driver looks at the position where the desired function is displayed on the screen displayed on the smart glasses (S320), and acquires the pupil position information of the driver through the image photographed by the camera facing the driver's eyes (S330) . The smart glasses can grasp information on the driver's gaze (direction) based on the acquired pupil position (S340), and can determine the position indicated by the driver's gaze on the display of smart glasses through dozens of gaze information (S350 ). The determined position is transmitted to the AVN system, and a predetermined visual effect may be displayed in the selected area corresponding to the determined position in the AVN system and the display of the smart glasses, in operation S360.

Then, the user confirms the selected area through the visual effect (S370), and inputs an execution command for the corresponding function by selecting icons of the selected area through the smart glasses or the operating system of the vehicle (S380).

Accordingly, the function corresponding to the selected icon can be executed in the AVN system (S390).

Hereinafter, an apparatus configuration for implementing the above-described embodiments will be described.

An example of the configuration of the AVN system for performing the above functions is shown in Fig.

4 is a block diagram showing an example of the configuration of an AVN system according to an embodiment of the present invention.

Referring to FIG. 4, the AVN system of the vehicle includes a wireless communication unit 410 for exchanging various control signals and AVN screen images by a wireless communication method such as Bluetooth or WiFi in connection with smart glasses, A wired communication unit 420 for exchanging signals with the controller, a display unit 430 for displaying a list of various functions or execution screens, a command input unit 440 for receiving commands from a driver such as a touch pad or a key button, A controller 450 for controlling the components, performing a judgment required for performing the embodiment, and performing an operation required for executing various functions (navigation, multimedia reproduction, etc.). For example, the control unit 450 may control the display of the screen to be transmitted to the smart glasses through the wireless communication unit, or may display the displayed images on the display unit such that a predetermined visual effect is provided at a position corresponding to the gaze direction information of the user, The control unit 430 may control the function or the function determined to be the function desired by the driver according to the signal of the command input unit 440. [

Of course, it will be apparent to those skilled in the art that the configuration of FIG. 4 is illustrative and can include more or fewer components as needed. For example, the wireless communication unit may be provided in a controller existing outside the AVN system, and may further include an audio output unit for outputting multimedia or navigation announcement audio.

Next, the configuration of the smart glasses will be described with reference to Fig.

5 shows an example of smart glasses that can be applied to embodiments of the present invention.

The smart glasses 200 are configured to be worn on the head of a human body, and a frame part (case, housing, etc.) for the smart glasses 200 may be provided. The frame portion may be formed of a flexible material to facilitate wearing. In this figure, it is illustrated that the frame portion includes a first frame 201 and a second frame 202 of different materials.

The frame portion is supported on the head portion, and a space for mounting various components is provided. As shown in the figure, electronic parts such as the control module 280, the sound output module 252, and the like may be mounted on the frame part. In addition, a lens 203 covering at least one of the left and right eyes may be detachably mounted on the frame portion.

The control module 280 is configured to control various electronic components included in the smart glasses 200. For example, the control module 280 may establish a wireless data path with the AVN system of the vehicle through a wireless communication unit (not shown) of the smart glasses and allow the AVN image of the vehicle to be received through the established data path. In addition, the control module 280 can display the received AVN image on the display 251, and can determine the direction of the user's gaze to the image photographed through the camera disposed facing the user's eyes. In addition, the control module 280 may display a predetermined visual effect at a position corresponding to the visual line direction determined in the displayed image. When the specific command inputting section 223 is displayed while the visual effect is being displayed, A signal requesting execution of a function corresponding to the AVN module is transmitted to the AVN module.

In this figure, the control module 280 is provided on the frame portion on one side of the head. However, the position of the control module 280 is not limited thereto.

The display unit 251 may be implemented as a head mounted display (HMD). The HMD type refers to a display method that is mounted on a head and displays an image directly in front of the user's eyes. When the user wears the smart glasses 200, the display unit 251 may be disposed to correspond to at least one of the left eye and the right eye so that the user can directly provide an image in front of the user's eyes. In this figure, the display unit 251 is located at a portion corresponding to the right eye so that an image can be output toward the user's right eye.

The display unit 251 can project an image with the user's eyes using a prism. Further, the prism may be formed to be transmissive so that the user can view the projected image and the general view of the front (the range that the user views through the eyes) together.

As described above, the image output through the display unit 251 can be overlapped with the general view. The smart glasses 200 can utilize the characteristics of such a display to assist the driver in selecting functions without disturbing the forward gaze during operation.

The camera 221 is disposed adjacent to at least one of the left eye and the right eye, and is configured to photograph a forward image. Since the camera 221 is located adjacent to the eye, the camera 221 can acquire a scene viewed by the user as an image.

Although the camera 221 is provided in the control module 280 in this figure, it is not limited thereto. The camera 221 may be installed in the frame part, or may be provided in a plurality of ways to acquire a stereoscopic image. In addition, a camera (not shown) facing the user's eyes may be additionally arranged in a direction opposite to the camera 221 facing the front. The camera facing the user's eyes acquires an image for determining the gaze direction through pupil tracking of the user.

The smart glasses 200 may include command input units 223a and 223b operated to receive a control command. The command input units 223a and 223b can be operated by a touch, a push, or the like.

The present invention described above can be implemented as computer readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, , And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet).

Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

Claims (18)

A method for an audio video navigation (AVN) system of a vehicle to perform functions of a vehicle using smart glasses,
Connecting wirelessly with smart glasses through a wireless communication unit of the AVN system;
Transmitting, through the wireless communication unit, a first image including the list of at least one function executable in the AVN system to the smart glasses;
Receiving information indicating a specific position of the first image from the smart glasses through the wireless communication unit; And
And executing a function corresponding to the specific one of the at least one executable function in the control unit of the AVN system when a selection signal input through the manipulation system is received after the information is received, How to implement the vehicle function. The method according to claim 1,
Wherein the first image is a video displayed through a display of an audio-video navigation (AVN) system. 3. The method of claim 2,
Further comprising displaying a predetermined visual effect at a specific location of the first image after the information is received. The method according to claim 1,
In the smart glasses,
And displaying the transmitted first image in the form of a head mounted display (HMD). 5. The method of claim 4,
In the smart glasses,
And a camera for photographing the user's eyes,
And determining a gaze direction of the user with a second image photographed through the camera to acquire the specific position. An audio-video navigation (AVN) system for a vehicle,
A wireless communication unit wirelessly connected to the smart glasses;
A wired communication unit provided on a handle of the vehicle, the wired communication unit being connected to an operating system for receiving a command from a driver; And
A first image including a list of at least one function executable in the AVN system is transmitted through the wireless communication unit with the smart glasses and information indicating a specific position of the first image is received from the smart glasses And a control unit for executing a function corresponding to the specific position among the at least one executable function when a selection signal inputted through the operating system is transmitted. The method according to claim 6,
Further comprising a display for displaying the first image. ≪ Desc / Clms Page number 19 > 8. The method of claim 7,
Wherein,
And controls the predetermined visual effect to be added to a specific position of the first image after the information is received. The method according to claim 6,
In the smart glasses,
And displays the transmitted first image in the form of a head mounted display (HMD). 10. The method of claim 9,
In the smart glasses,
And a camera for photographing the user's eyes,
(AVN) system of a vehicle, wherein the AVN system acquires the specific position by determining a direction of the user's gaze to a second image photographed through the camera. A method of controlling smart glasses,
Wirelessly connecting to an audio-video navigation (AVN) system of the vehicle through a wireless communication unit;
Receiving from the AVN system a first image including a list of at least one function executable in the AVN system through the wireless communication unit;
Displaying the received first image through a display unit in the form of a head mounted display (HMD);
Determining a gaze direction of the user with a second image photographed through a camera arranged to photograph the user's eyes from the control unit;
Acquiring a specific position in the first image using the determined viewing direction; And
And transmitting information on the obtained specific location to the AVN system through the wireless communication unit. 12. The method of claim 11,
Further comprising the step of displaying, in the displayed received image, a predetermined visual effect at the obtained specific position. 13. The method of claim 12,
Further comprising transmitting to the AVN system a signal requesting execution of a function corresponding to the specific position when the command input unit is operated when the preset visual effect is displayed. 12. The method of claim 11,
The received image may be,
And displaying the image on the AVN system. A wireless communication unit wirelessly connected to an audio / video navigation (AVN) system of a vehicle and receiving a first image from the AVN system, the first image including a list of at least one function executable in the AVN system;
A display unit for displaying the received first image in the form of a head mounted display (HMD);
A camera arranged to photograph a user's eyes; And
A second image obtained through the camera, determines a direction of the user's gaze, acquires a specific position in the image using the determined gaze direction, and transmits the information on the obtained specific position to the AVN system And a control unit for controlling the light source to be transmitted. 16. The method of claim 15,
Wherein,
And controls to display a predetermined visual effect at the acquired specific position in the displayed received image. 17. The method of claim 16,
And a command input unit for receiving a command from a user,
Wherein,
And controls to transmit a signal requesting execution of a function corresponding to the specific position to the AVN system when the command input unit is operated when the preset visual effect is displayed. 16. The method of claim 15,
The received image may be,
The smart glasses are images displayed through the AVN system.

Images