KR20140145332A - HMD system of vehicle and method for operating of the said system - Google Patents

Abstract

The present invention relates to a vehicle HMD system and an operating method thereof. The vehicle HMD system includes a head-mounted projection HMD which is linked to an AVM providing the vehicle operation information and navigation information. The system includes: the AVN which collects the sensing information acquired from the sensors mounted on a vehicle and vehicle front image information to generate the vehicle operation information and navigation information; and the head-mounted projection HMD which is in the form of eyeglasses including a display window and a camera arranged on a side to input the image information obtained through the camera to the AVN and displays the operation information and the navigation information on the display window.

Description

[0001] The present invention relates to a vehicle HMD system and a method of operating the HMD system,

The present invention relates to a head mounted display (HMD) system for a vehicle and a method of operating the same. More particularly, the present invention relates to a vehicle HMD system including a spectacled projection HMD and an operation method thereof.

The HUD (Head Up Display) system is called as a forward upward vision system. It is a type of display that reflects information directly to human vision. It was first developed as military aviation technology, but it is widely applied to the private sector. In order to reduce the burden of watching the instrument while flying the clock, the pilot's vision was designed to facilitate visual flight while projecting the instrument's information.

HUD for aviation uses hologram-based HUD for CRT. In the case of aviation HUD, the system is complex, requires precision, and is expensive.

In recent years, HUD systems for vehicles have been widely applied to vehicles using the HUD principle for aviation. As with aircraft, if you are driving at high speeds on highways, it is difficult to quickly identify situations on the road ahead or on the road between your eyes turning to the dashboard to view speed, oil, or navigation information. In addition, since an accident may occur while moving the field of view, the HUD system is introduced in the vehicle in order to eliminate such a risk.

The HUD for the vehicle displays information on the instrument panel such as speed, mileage, and RPM on the main sight line of the windshield operator and information on the driving direction of the navigation system so that the driver can recognize important driving information without taking his eyes off the road. have.

However, in the case of the conventional HUD for vehicles, it is possible to limit the area of information display by fixing the sight line of the driver to the driver's line of sight ahead of the HUD. However, the information displayed by the size limitation of the display is limited, , The image may be distorted when the vehicle is tinted on the front surface of the vehicle.

In addition, when the driver is looking at the left or right side or rear side depending on the situation during driving, the information displayed on the HUD becomes meaningless.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a vehicle HMD system including an eyeglass-type projection HMD interlocked with an AVN that provides vehicle information and navigation information, and an operation method thereof.

However, the objects of the present invention are not limited to those mentioned above, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the present invention provides an AVN that collects image information about a vehicle front and sensing information obtained from sensors mounted on a vehicle to generate driving information and navigation information related to driving of the vehicle; And a spectacles-shaped projection HMD for inputting the image information obtained through the camera into the AVN and displaying the driving information and the navigation information through the display window, the spectacles having a display window and a camera on one side And a vehicle HMD system.

Preferably, the vehicle HMD system further includes a driver terminal for connecting the CAN network in the vehicle with an external network, and sharing the navigation information and the navigation information stored in the AVN with people other than the driver.

Preferably, the display-type window, the camera, the microphone, and the speaker are integrally mounted on one side of the spectacle-type projection HMD.

Preferably, the eyeglass-type projection HMD displays information obtained by the in-vehicle Lane Keeping Assistance System (LKAS) and a blind spot warning on the display window, Audio, sound, and vibration.

Preferably, the AVN analyzes the driver's voice inputted from the glasses-type projection HMD to generate the navigation information again, and the glasses-type projection HMD displays the generated navigation information on the display window.

According to another aspect of the present invention, there is provided a method for acquiring image information about a vehicle ahead of a vehicle using a camera mounted on a projection type HMD; A sensing information acquiring step of acquiring sensing information using sensors mounted on the vehicle; A driving and navigation information generating step of collecting the image information and the sensing information to generate driving information and navigation information related to the driving of the vehicle; And a navigation and information display step of displaying the navigation information and the navigation information through a display window mounted on one side of the eyeglass-type projection HMD, and a navigation information display step of displaying the navigation information.

Preferably, the navigation and information sharing step may include a step of connecting the CAN network in the vehicle with the external network to share the navigation information and the navigation information with other persons other than the driver, after the navigation and navigation information display step.

Preferably, the image information acquisition step and the navigation and navigation information display step use an HMD in which the display window, the camera, the microphone, and the speaker are integrally mounted on one side with the spectacle-type projection HMD.

Preferably, the driving and navigation information display step displays information obtained by the Lane Keeping Assistance System (LKAS) and a Blind Spot Warning in the vehicle to the display window, And outputs the information by one of voice, sound, and vibration.

Preferably, the driving and navigation information generating step generates the navigation information by analyzing the voice of the driver input from the glasses-type projection HMD, and the driving and navigation information display step displays the generated navigation information on the display window .

The present invention can improve a driver's forward vision through a vehicle HMD system including a glasses-type projection HMD interlocked with an AVN providing driving information and navigation information of a vehicle, and can provide various information to a driver through a smart phone, Driving can be pursued.

1 is a conceptual diagram schematically showing a vehicle HMD system according to a preferred embodiment of the present invention.
Fig. 2 is a perspective view of a projection type HMD according to the present invention.
3 is a conceptual diagram showing interlocking between various systems installed in a vehicle and a vehicle HMD system.
FIG. 4 is a flow chart for explaining interlocking between the LKAS system and the HMD system as an example of FIG. 3. FIG.
5 is a flow chart for explaining the interlocking between the BSW system and the HMD system as an example of FIG.
6 is a reference diagram for supplementing explanation of the interlocking operation of FIG.
7 is a flowchart for explaining a navigation function of the HMD system.
FIG. 8 is a reference diagram for supplementing the function of FIG. 7. FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.

The HMD system is called a head mounted vision system, which is a helmet type head mounted or mounted with glasses. It is a system which is mainly applied to the multimedia field, and a large screen is realized by using a small display device with a display in front of the eyes. It is a next-generation video display device that can be used on a large-sized screen while being carried, can be used for surgery or diagnosis, and can be applied to a virtual reality.

The present invention proposes a vehicle HMD system equipped with a projection type HMD for glasses. Unlike existing HMDs, HMDs for vehicles do not affect the driver's ability to operate, and provide various meter information and driving information. In addition, since the use of a mobile phone during operation greatly affects safety, a user interface is provided that displays information on the vehicle HMD and allows the driver to easily control the vehicle while driving. HMD, navigation (AVN) and mobile phone to implement additional functions through interworking.

1 is a conceptual diagram schematically showing a vehicle HMD system according to a preferred embodiment of the present invention.

1, a vehicle HMD system 100 includes a spectacular projection HMD 110, an AVN 120, and a driver terminal 130. As shown in FIG.

The AVN 120 collects the image information about the front of the vehicle and the sensing information obtained from the sensors mounted on the vehicle, and generates driving information and navigation information related to the driving of the vehicle.

In the present embodiment, the AVN 120 includes a navigation function, an Internet information search function through voice recognition, a hands-free function, a transmission / reception of multimedia data received from the driver terminal 130, A vehicle information collection function via the CAN network, a network sharing function using the driver terminal 130 (connecting the vehicle network to the external Internet), and a black box image storage function.

The eyeglass-type projection HMD 110 is a type of eyeglasses having a display window and a camera on one side, and inputs image information obtained through the camera to the AVN, and displays driving information and navigation information through a display window.

In the present embodiment, the eyeglass-type projection HMD 110 can perform an information search through a voice recognition, a result screen display function, a vehicle driving information display function, an HMD camera image information transfer function, and the like.

The projection type HMD 110 may be an out-of-sight projection display, a camera for external photography, a Wifi for communication inside a vehicle, a microphone for voice recognition and a handsfree function, and a speaker for providing voice information. A more detailed description of the projection type HMD 110 will be described later with reference to FIG.

The driver terminal 130 connects the CAN network in the vehicle with the external network and performs a function of sharing driving information and navigation information stored in the AVN 120 with people other than the driver.

In this embodiment, the driver terminal 130 has a function of transmitting the 3G / LTE network sharing function (connecting the vehicle network to the external network), telephone, text, App Data, etc. to the AVN 120, internal storage data and streaming data And a function of transmitting various multimedia data to the AVN 120, for example.

The vehicle HMD system 100 described above with reference to FIG. 1 is a system in which a communication module for providing a wireless network is installed in the AVN 120 and the vehicle is directly connected to an external network. However, in such a case, a continuous communication cost is generated for each individual vehicle. Therefore, a method of sharing the LTE and the 3G network of the driver terminal 130 is effective in cost reduction and may have a higher degree of freedom depending on each individual driver.

On the other hand, the spectacle-type projection HMD 110 is characterized in that a display window, a camera, a microphone, and a speaker are integrally mounted on one side. This will be described below. Fig. 2 is a perspective view of a projection type HMD according to the present invention.

The eyeglass-type projection HMD 110 displays data processed and calculated by the AVN 120.

The projection display window 112 has a position and size that does not disturb the driver ' s view. It also has the minimum brightness value to display the necessary information, especially automatically adjusted according to the external brightness. It also applies to only one eye.

The external photographing camera 111 has a camera direction coinciding with the sight line of the driver. Also, basic information for recognizing the same information as the driver's view and outputting the augmented reality information is stored. It also stores real-time vehicle information such as black box images.

The vehicle internal communication Wifi is connected to the AVN 120.

The microphone 113 has a display and HMD system control function through voice commands and also provides a hands-free function.

The speaker 114 provides navigation and vehicle information by voice, and provides a hands-free function.

Although the above-described projection type HMD 110 is a HMD having a shape of glasses, it can be designed to have only an HMD portion without glasses eyeglasses according to circumstances, and it is designed to be able to change the number of glasses according to the driver's vision.

It is a glasses type that can be a little inconvenient to wear, but it has a big advantage over the existing HUD.

First, since it coincides with the direction of the field of view of the driver, it is possible to utilize the image data photographed by using the external photographing camera 111 variously, and the information can always be displayed in front of the field of view of the driver.

The image information collected by the external photographing camera 111 basically records all the work occurring in the driver's field of view like a black box image, so that it can be used as a basis data in case of an accident.

Since the image is stored according to the driver's will rather than the existing fixed image, more efficient storage is possible.

Also, the augmented reality based information output which recognizes the photographed image and displays various information by recognizing the forward vehicle number, the sign, etc., is possible.

The navigation information and the vehicle information are inputted from the AVN 120 and displayed on the front side to fix the front view of the driver in front and the related information is also displayed on the screen in cooperation with the system such as the lane change alarm device of the vehicle and the blind spot warning.

Also, information such as telephone, text, etc., transmitted from the driver terminal 130 is briefly displayed, and hands-free and character information are output through the speaker 114 so that only the driver can recognize it.

HMD display information and an interface for controlling the AVN 120 through voice recognition, and by reducing the operation of the AVN 120 as much as possible, it is possible to concentrate on driving the vehicle.

On the other hand, the eyeglass-type projection HMD 110 displays information obtained by a Lane Keeping Assistance System (LKAS) and a Blind Spot Warning (LKAS) on a display window and outputs the information to a sound, And the vibration can be output by any one of the methods. This will be described below. 3 is a conceptual diagram showing interlocking between various systems installed in a vehicle and a vehicle HMD system.

Functions performed by the central processing AVN 120 will be described below.

1. Vehicle and driving information collection

- Gather information on speed / RPM / vehicle status using CAN network

- Acquisition of traffic situation / direction through GPS-based navigation information

- LKAS, BSW and other operation support system information collection

2. Network sharing and data transmission / reception with the driver terminal 130

- LTE / 3G tethering to connect wireless networks to vehicle networks

- Transmits phone and text sending and receiving information

- Application data including KakaoTalk / SNS / search information

- Streaming smartphone multimedia data through in-vehicle network

- Streaming multimedia information mirroring provided to the driver terminal 130 over a wireless network, such as a melon / Youtube

3. HMD external camera shooting image processing

- Save the received video (black box function)

- Reception image recognition processing: Vehicle number, object, person, sign, lane, etc.

4. HMD speech recognition and voice command processing, voice information provision

- Interface control and system control by recognizing HMD microphone input voice command

- Multimedia and system related information played through AVN speakers

- Operator privacy information and hands-free information are sent to the HMD and provided through the HMD speaker / earphone

5. Provide HMD display information

- collected vehicle and driving information, alarm display

- augmented reality data generated through external camera image recognition Display

- Phone party and text message information display

Interworking between the vehicle system and the HMD system is possible in various ways. Basically, since the AVN 120 itself is connected to the CAN network of the vehicle, it is possible to share data with the various systems 200 and 300 using the vehicle CAN.

3 shows the interlocking of a lane keeping assistance system 200, a blind spot warning system 300 and an HMD system 100. As shown in FIG. Instead of displaying warning and information on the existing cluster and side mirrors, an alarm signal is displayed on the HMD screen immediately displayed in front of the driver's eyes, and a voice alarm is issued. This makes it possible to increase the safety of driving the vehicle by directly conveying information that is important for vehicle safety to the user.

The operation flow according to the interlocking between the LKAS system 200 and the BSW system 300 and the HMD system 100 can be seen in FIG.

In FIG. 3, reference numerals 210, 220, 310, and 320 denote an assembly in which a camera and an ECU are combined, an MDPS (Motor Driven Power Steering), a first side rearward radar, and a second side rearward radar.

FIG. 4 is a flow chart for explaining interlocking between the LKAS system and the HMD system as an example of FIG. 3. FIG.

The LKAS camera detects lane departure of the vehicle through lane photographing (S405). If the lane departure of the vehicle is detected (S410), the driver is warned of the departure situation before the active steering intervention to control the MDPS (S415 to S425).

In the case of the conventional LKAS, a warning signal is displayed on the cluster of the vehicle. Thus, the driver had to change the field of view to identify the cluster, and it was difficult to recognize easily.

As in the present invention, the HMD system receives the lane departure information of the LKAS from the AVN through CAN communication (S415). The AVN transmits lane departure warning information to the HMD through the wireless network in the vehicle (S420). The HMD displays a warning signal on the projection type screen, and warns the driver of lane departure through vibration and voice (S425 ).

When the driver returns to the normal state through the steering operation, the existing driving is performed (S430), and the lane-based shooting through the LKAS is continued (S405).

However, if the lane departure of the vehicle continues without the driver's vehicle control after the warning (S410), the lane departure information is received in the MDPS (S435). Accordingly, the vehicle is actively intervened in the vehicle operation through the steering motor control of the MDPS (S440) and maintains the driving direction of the vehicle (S445).

FIG. 5 is a flowchart for explaining the interlocking between the BSW system and the HMD system as an example of FIG. 3, and FIG. 6 is a reference diagram for supplementing explanation of the interlocking operation of FIG.

In the conventional BSW, when there is a vehicle on the blind spot, the left and right side mirrors are displayed. In the present invention, this is displayed on the screen of the HMD, thereby helping the driver to recognize the left and right situations even in a situation where the driver is looking forward.

FIG. 5 includes a portion for controlling the BSW display through a voice command. As shown in Fig. 6 (a), when the driver instructs to "sound BSW display" by voice (S510), voice is recorded through the microphone of the HMD, and contents are transmitted to the AVN through the wireless network.

Recognizes the voice in the AVN (S520), and recognizes that it is an instruction to display the BSW warning display on the HMD. Then, BSW information is received from the BSW system via CAN communication and prepared for display on the HMD (S530).

When the user operates the blinking for the lane change (S540), the left and right BSW radar detects whether or not the vehicle exists in the blind spot (S550).

If the vehicle exists, the BSW warning information is transmitted to the AVN via CAN communication (S560), and the AVN transmits the received information to the HMD via the vehicle wireless network (S570).

The HMD receives the alarm, displays warning signs on the left and right sides of the projection-type HMD screen, and warns the driver through vibration and sound as shown in FIG. 6 (b) (S580).

On the other hand, the AVN 120 analyzes the driver's voice inputted from the glasses-type projection HMD 110 to generate navigation information again, and the glasses-type projection HMD 110 can display the generated navigation information on the display window. This will be described below.

FIG. 7 is a flowchart for explaining the navigation function of the HMD system, and FIG. 8 is a reference diagram for supplementing explanation of the functions of FIG.

7 shows a flow of operation of the navigation system operating with the combination of the projection HMD 110, the AVN 120 and the driver terminal 130. FIG.

The driver instructs the desired destination via the HMD microphone as shown in FIG. 8 (a), such as "Tell Gwanghwamun fast way" (S710).

The voice recorded in the HMD microphone is transmitted to the AVN through the wireless network, the AVN recognizes the voice command (S720), and performs an operation of searching for a fast route to the user in step S730.

And connects to the traffic information server through the external network to receive the optimized route to the real time traffic information and the destination (S740).

Based on the received contents, the AVN creates an optimal route list, and displays the optimal route list on the HMD screen as shown in FIG. 8B (S750).

The driver selects a desired route from the displayed list by voice command as shown in FIG. 8C (S760), and the voice command is recognized again by the AVN.

The selected route is displayed on a map embedded in the AVN, and navigation is started to guide the AVN in the same manner as the existing navigation (S770).

In addition, as shown in FIG. 8 (d), the HMD screen also displays simplified route guidance for the driver to recognize and transmits information to the driver by voice (S780).

The HMD system proposed in the present invention is based on an AVN and an external network connection through a driver terminal as an HDM system designed for a vehicle rather than suggesting a simple HMD display.

HMD performs video display and voice related functions, and the overall system operation and control is performed by AVN.

The driver terminal plays a role of linking the internal network of the vehicle with the external network through the wireless network.

AVN is composed of a system using a high-performance processor, since parts such as video signal recognition and multimedia processing require a lot of computing power. CAN information can be exchanged with various systems of the vehicle through CAN communication. Therefore, it is possible to utilize such information to provide various functions by merging with various systems.

The present invention can provide various information for driving easily while allowing the driver to concentrate more on driving.

Next, an operation method of the vehicle HMD system will be described.

First, the image information about the front of the vehicle is acquired using a camera mounted on the projection type HMD (image information acquisition step).

Thereafter, sensing information is acquired using the sensors mounted on the vehicle (sensing information acquiring step).

The AVN then collects the image information and the sensing information to generate driving information and navigation information related to the driving of the vehicle (driving and navigation information generating step).

Then, the driving information and the navigation information are displayed through a display window mounted on one side of the eyeglass type projection HMD (display of navigation and navigation information).

Meanwhile, after the operation and navigation information display step, the driver terminal connects the CAN network in the vehicle with the external network to share driving information and navigation information with the driver and other people (operation and navigation information sharing step).

It is to be understood that the present invention is not limited to these embodiments, and all elements constituting the embodiment of the present invention described above are described as being combined or operated in one operation. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. In addition, such a computer program may be stored in a computer readable medium such as a USB memory, a CD disk, a flash memory, etc., and read and executed by a computer to implement an embodiment of the present invention. As the recording medium of the computer program, a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like can be included.

Furthermore, all terms including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined in the Detailed Description. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

Claims (10)

An AVN which collects image information about the front of the vehicle and sensing information obtained from sensors mounted on the vehicle to generate driving information and navigation information related to the driving of the vehicle; And
Type projecting HMD for inputting the image information obtained through the camera into the AVN and displaying the driving information and the navigation information through the display window in the form of glasses having a display window and a camera on one side,
Wherein the HMD system comprises: The method according to claim 1,
A driver terminal for connecting the CAN network in the vehicle with an external network and sharing the navigation information and the navigation information stored in the AVN with people other than the driver
Further comprising: a light emitting diode (LED). The method according to claim 1,
Wherein the display window, the camera, the microphone, and the speaker are integrally mounted on one side of the spectacle-type projection HMD. The method of claim 3,
The eyeglass-type projection HMD displays information obtained by the in-vehicle Lane Keeping Assistance System (LKAS) and a blind spot warning on the display window and outputs the information to the sound, And the vibration is outputted by any one of the following methods. The method according to claim 1,
The AVN analyzes the driver's voice inputted from the projector-type HMD to generate the navigation information again,
Wherein the eyeglass-type projection HMD displays the generated navigation information on the display window. An image information acquiring step of acquiring image information about the front of the vehicle using a camera mounted on a spectacled projection HMD;
A sensing information acquiring step of acquiring sensing information using sensors mounted on the vehicle;
A driving and navigation information generating step of collecting the image information and the sensing information to generate driving information and navigation information related to the driving of the vehicle; And
A driving and navigation information display step of displaying the driving information and the navigation information through a display window mounted on one side of the spectacled projection HMD
Wherein the HMD system is configured to operate the vehicle. The method according to claim 6,
A navigation and information sharing step of connecting the CAN network in the vehicle with an external network to share the driving information and the navigation information with other persons other than the driver
Further comprising the steps < RTI ID = 0.0 > of: < / RTI > The method according to claim 6,
Wherein the image information acquisition step and the navigation and navigation information display step use an HMD having the display window, the camera, the microphone and the speaker integrally mounted on one side with the spectacles-like projection HMD. 9. The method of claim 8,
The navigation and navigation information display step displays the information obtained by the in-vehicle Lane Keeping Assistance System (LKAS) and the blind spot warning on the display window, , And outputting the sound signal by one of a sound and a vibration. The method according to claim 6,
The driving and navigation information generating step generates the navigation information by analyzing the driver's voice inputted from the spectacled projection HMD,
Wherein the navigation and navigation information display step displays the generated navigation information to the display window.

Images