KR20210088487A - System for providing 3D head-up display and control method thereof - Google Patents

Abstract

Disclosed are a 3D head-up display system, and a control method thereof. Thus, the present invention provides an image located at a desired distance to a user by using a 3D display which can provide different information to both eyes of the user, thereby providing the more realistic HUD image to the user and displaying the information at a state with reduced sense of heterogeneity so as to increase user satisfaction.

Description

3D head-up display system and its control method {System for providing 3D head-up display and control method thereof}

The present invention relates to a 3D head-up display system and a method for controlling the same, and in particular, a 3D display that provides a user with an image located at a desired distance using a 3D display that can provide different information to both eyes of the user It relates to a head-up display system and a method for controlling the same.

2. Description of the Related Art A head-up display (or windshield display) is a device for displaying a vehicle speed or a driving direction on a windshield of a vehicle.

Among these heads-up displays, the refractive head-up display has a distance difference between the image of the head-up display and an actual external object due to the limitation of the distance between the image and the user's eye, and such a head-up display Due to the difference in distance between the image of the display and the actual external object, a sense of difference is felt during use.

In addition, the head-up display has to create a length at which an image is formed with a projection unit and a plurality of reflectors in order to create a difference in the length of the path, and the longer the position of the image, the larger the size of the module inevitably. In addition to having a , the position of the image thus created has no choice but to be located in a single position, so there is a limit to the image to be expressed.

Korean Patent Registration No. 10-1610275 [Title: Variable Focus Hud System]

It is an object of the present invention to provide a 3D head-up display system and a control method thereof for providing a user with an image located at a desired distance using a 3D display capable of providing different information to both eyes of the user. have.

Another object of the present invention is to generate a 3D image using information obtained through a communication unit, a sensor unit, a photographing unit, etc., and reflect a 3D image displayed through a 3D display through a reflector to wind a virtual 3D HUD image. An object of the present invention is to provide a 3D head-up display system and a method for controlling the same.

A 3D head-up display system according to an embodiment of the present invention includes: a sensor unit for measuring information on an object located in front of a vehicle, which is a traveling direction of the vehicle, and collecting information related to the vehicle; a photographing unit for obtaining image information including the object; a communication unit for receiving additional information related to an object in the image information and surrounding situation information according to the location of the vehicle provided from a server; a gaze tracking unit detecting a gaze direction and a gaze position of the user in the vehicle; Information on the measured object located in front of the vehicle, information related to the collected vehicle, the acquired image information, the received surrounding situation information according to the location of the vehicle, and addition related to the object in the received image information a controller for generating 3D content to be provided to the user based on at least one of information, the sensed gaze direction and gaze position of the user, and destination information according to a user input; a 3D display for outputting the 3D content generated by the control unit; a reflector for reflecting the 3D content output by the 3D display; and a windshield displaying the image reflected by the reflector.

As an example related to the present invention, the 3D content includes at least one of route guidance information according to the destination information, additional information on an object located in front of the vehicle, and detailed information related to the vehicle, wherein the additional information includes: It includes at least one of a distance between the vehicle and the object, a speed, and additional information, and the detailed information may include surrounding situation information according to the location of the vehicle and information related to the collected vehicle.

As an example related to the present invention, when the 3D content includes additional information about an object located in front of the vehicle and detailed information related to the vehicle, the windshield is adjacent to the real object located in front of the vehicle. The additional information included in the reflected image may be displayed on one side of the screen, and the detailed information included in the reflected image may be displayed on the other side of the windshield.

A method of controlling a 3D head-up display system according to an embodiment of the present invention includes measuring, by a sensor unit, information on an object located in front of a vehicle in a traveling direction of the vehicle; collecting, by the sensor unit, information related to the vehicle; obtaining, by the photographing unit, image information including the object; receiving, by a communication unit, additional information related to an object in the image information and surrounding situation information according to the location of the vehicle provided from a server; detecting, by a gaze tracking unit, a gaze direction and a gaze position of a user in the vehicle; By the control unit, the measured information on the object located in front of the vehicle, the collected vehicle-related information, the obtained image information, the received surrounding situation information according to the location of the vehicle, and the received image information. generating 3D content to be provided to the user based on at least one of additional information related to an object, the detected gaze direction and gaze position of the user, and destination information according to a user input; outputting the 3D content generated by the control unit through a 3D display; and displaying, by the windshield, an image of the 3D content output by the 3D display reflected by the reflector.

As an example related to the present invention, in the displaying of the reflected image, the 3D content may include route guidance information according to the destination information, additional information about an object located in front of the vehicle, and detailed information related to the vehicle. At this time, based on the actual lane located in front of the vehicle, information on the traveling direction of the vehicle according to the road guidance information included in the reflected image is displayed on one side of the windshield based on the actual lane located in front of the vehicle, and information on the actual object located in front of the vehicle is displayed. Additional information included in the reflected image may be displayed on the other side of the adjacent windshield, and the detailed information included in the reflected image may be displayed on another side of the windshield.

The present invention provides the user with an image located at a desired distance using a 3D display that can provide different information to both eyes of the user, thereby providing a more realistic HUD image to the user and reducing the sense of heterogeneity There is an effect of increasing user satisfaction by displaying the information.

In addition, the present invention generates a 3D image by using information obtained through a communication unit, a sensor unit, a photographing unit, etc., and reflects a 3D image displayed through a 3D display through a reflector to apply a virtual 3D HUD image to the windshield. By displaying the entire system in a compact size, the cost is reduced, the distance between the display and the eye is closer, providing a wider usage environment even with a smaller size 3D display, expressing information about distant objects, and using multiple distances. It has the effect of expressing information about an existing object.

1 and 2 are block diagrams showing the configuration of a 3D head-up display system according to an embodiment of the present invention.
3 is a flowchart illustrating a control method of a 3D head-up display system according to an embodiment of the present invention.
4 is a diagram illustrating an example of image information according to an embodiment of the present invention.
5 and 6 are diagrams illustrating examples of virtual images displayed on a windshield according to an embodiment of the present invention.

It should be noted that the technical terms used in the present invention are only used to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be interpreted as meanings generally understood by those of ordinary skill in the art to which the present invention belongs, unless otherwise defined in particular in the present invention, and excessively comprehensive It should not be construed in the meaning of a human being or in an excessively reduced meaning. In addition, when the technical term used in the present invention is an incorrect technical term that does not accurately express the spirit of the present invention, it should be understood by being replaced with a technical term that can be correctly understood by those skilled in the art. In addition, general terms used in the present invention should be interpreted as defined in advance or according to the context before and after, and should not be interpreted in an excessively reduced meaning.

Also, as used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present invention, terms such as "consisting of" or "comprising" should not be construed as necessarily including all of the various elements or several steps described in the invention, and some of the elements or some steps may not be included. It should be construed that it may further include additional components or steps.

In addition, terms including ordinal numbers such as first, second, etc. used in the present invention may be used to describe the components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted.

In addition, in the description of the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easy understanding of the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings.

1 and 2 are block diagrams showing the configuration of a 3D head-up display system 100 according to an embodiment of the present invention.

As shown in FIG. 1 , the 3D head-up display system 100 includes a communication unit 110 , a sensor unit 120 , a photographing unit 130 , an eye tracking unit 140 , a storage unit 150 , and a 3D display ( 160 ), a reflecting plate 170 , a windshield 180 , and a control unit 190 . Not all of the components of the 3D head-up display system 100 shown in FIGS. 1 and 2 are essential components, and the 3D head-up display system ( 100) may be implemented, and the 3D head-up display system 100 may be implemented with fewer components.

The 3D head-up display system (or articulated 3D head-up display system) 100 may be disposed (or configured/formed) at a specific position in the vehicle and configured to be detachable/attachable.

The communication unit 110 communicates with any internal component or at least one external terminal through a wired/wireless communication network. In this case, the external arbitrary terminal may include a server (not shown), another terminal (not shown) provided in another adjacent vehicle, and the like. Here, as wireless Internet technologies, wireless LAN (WLAN), DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), HSDPA (High Speed Downlink Packet Access) ), High Speed Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), etc. In this case, the communication unit 110 transmits and receives data according to at least one wireless Internet technology within a range including Internet technologies not listed above. In addition, short-range communication technologies include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near Field Communication (NFC). , Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi (Wi-Fi), Wi-Fi Direct (Wi-Fi Direct), etc. may be included. In addition, the wired communication technology may include power line communication (PLC), USB communication, Ethernet, serial communication, optical/coaxial cable, and the like.

Also, the communication unit 110 may mutually transmit information with an arbitrary terminal through a Universal Serial Bus (USB).

In addition, the communication unit 110 is a technology standard or communication method for mobile communication (eg, GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV -DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced, etc.) transmits and receives radio signals to and from the base station, the server, and the other terminal on a mobile communication network constructed according to the same.

In addition, the communication unit 110, under the control of the control unit 190, the location information of the vehicle that is checked in real time with the server, image information obtained through the photographing unit 130 (or an object in the image information) information (for example, including a vehicle number for an object such as a vehicle)) and the like.

In addition, the communication unit 110, under the control of the control unit 190, information (or surrounding situation information) related to the surrounding situation according to the location of the vehicle transmitted from the server in response to the information transmission (or surrounding situation information) (for example, Receives speed limit information on the corresponding road, construction information on surrounding roads, etc.), additional information related to the object in the image information (including information such as whether the vehicle is stolen, whether or not a fine for negligence is arrears, etc.).

The sensor unit 120 is disposed (or configured/formed) on one side of the vehicle in which the 3D head-up display system 100 is configured.

In addition, the sensor unit 120 includes various sensors (eg, including a lidar sensor, an ultrasonic sensor, a 3D scanner, etc.) for measuring a distance, speed, etc. related to an object located in the traveling direction of the vehicle (or in front of the vehicle) do. Here, the object includes another vehicle, a building, a road sign, a traffic sign, and the like located in front of the vehicle.

In addition, the sensor unit 120 measures (or senses) information on an object including a distance, speed, etc. related to the object located in front of the vehicle.

In addition, the sensor unit 120 measures information on an object located in a direction opposite to the traveling direction of the vehicle (or at the rear of the vehicle).

In addition, the sensor unit 120 collects information related to the vehicle (including, for example, driving speed, engine RPM, instantaneous fuel efficiency, real-time voltage, water temperature, mileage, etc.). In this case, the sensor unit 120 may collect information related to the vehicle measured through various other sensors (not shown) provided in the vehicle.

The photographing unit (or camera unit) 130 is disposed (or configured/formed) on one side of the vehicle in which the 3D head-up display system 100 is configured.

In addition, the photographing unit 130 includes one or more image sensors (a camera module or a camera) to photograph a front surface including the front of the vehicle, the rear of the vehicle, and the side of the vehicle. In this case, the photographing unit 130 may be configured as a stereo camera capable of acquiring image information for a 360 degree omnidirectional view.

In addition, the photographing unit 130 acquires (or captures) image information including one or more objects located in the front of the vehicle.

That is, the photographing unit 130 processes an image frame such as a still image or a moving picture obtained by an image sensor (a camera module or a camera) in a video call mode, a photographing mode, a video conference mode, and the like. That is, the corresponding image data obtained by the image sensor according to the CODEC is encoded/decoded to meet each standard. For example, the photographing unit 130 photographs an object (or subject), and outputs a video signal corresponding to the photographed image (subject image).

Also, the photographing unit 130 acquires other image information including one or more other objects located at the rear of the vehicle.

In addition, the image frame (or image information) processed by the photographing unit 130 is stored in a digital video recorder (DVR), stored in the storage unit 150, or through the communication unit 110 . It may be transmitted to an external server or the like.

Also, the 3D head-up display system 100 may receive destination information according to a user input through an input unit (not shown).

In this way, the communication unit 110, the sensor unit 120, the photographing unit 130, the input unit, etc. included in the 3D head-up display system 100 provide information related to the vehicle, information related to the vehicle in which the vehicle is driving, and the like. Various information about one or more objects located in the front of the vehicle and/or at the rear of the vehicle corresponding to the driving direction may be collected (or measured/detected).

The eye tracking unit 140 is disposed (or configured/formed) on one side of the vehicle in which the 3D head-up display system 100 is configured.

In addition, the gaze tracking unit 140 detects (or tracks) the gaze direction and gaze position of the user (or the driver) located in the driver's seat in the vehicle (or the direction and position in which the user looks out of the vehicle).

That is, the eye tracking unit 140 detects the direction and position in which the user located in the driver's seat looks at the front of the vehicle or a side mirror (not shown) configured on the side of the vehicle.

The storage unit 150 stores various user interfaces (UIs), graphic user interfaces (GUIs), and the like.

In addition, the storage unit 150 stores data and programs necessary for the 3D head-up display system 100 to operate.

That is, the storage unit 150 is a plurality of application programs (application programs or applications) driven in the 3D head-up display system 100, data for the operation of the 3D head-up display system 100, Commands can be stored. At least some of these applications may be downloaded from an external server via wireless communication. In addition, at least some of these application programs may exist on the 3D head-up display system 100 from the time of shipment for the basic function of the 3D heads-up display system 100 . On the other hand, the application program is stored in the storage unit 150, installed in the 3D head-up display system 100, the control unit 190 performs the operation (or function) of the 3D head-up display system 100 can be driven to

In addition, the storage unit 150 is a flash memory type (Flash Memory Type), a hard disk type (Hard Disk Type), a multimedia card micro type (Multimedia Card Micro Type), a card type memory (for example, SD or XD) memory, etc.), magnetic memory, magnetic disk, optical disk, RAM (Random Access Memory: RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory: ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), It may include at least one storage medium among Programmable Read-Only Memory (PROM). In addition, the 3D head-up display system 100 may operate a web storage that performs a storage function of the storage unit 150 on the Internet, or may operate in connection with the web storage.

In addition, the storage unit 150 includes location information of the corresponding vehicle transmitted to the server under the control of the control unit 190 , image information acquired through the photographing unit 130 (or information about an object in the image information). information (eg, including a vehicle number for an object such as a vehicle)), and the like.

In addition, the storage unit 150 is information (or surrounding situation information) related to the surrounding situation according to the location of the vehicle transmitted from the server in response to the information transmission under the control of the control unit 190 (for example, It stores speed limit information on the corresponding road, construction information on surrounding roads, etc.) and additional information related to the object in the image information (including, for example, whether the vehicle is stolen, whether or not the fine is arrears), and the like.

The 3D display (or 3D monitor/3D display unit) 160 is disposed (or configured/formed) on one side of the vehicle in which the 3D head-up display system 100 is configured.

In addition, the 3D display 160 may display various contents such as various menu screens using a user interface and/or a graphic user interface stored in the storage 150 under the control of the controller 190 . Here, the content displayed on the 3D display 160 includes various text or image data (including various information data) and a menu screen including data such as icons, list menus, and combo boxes. Also, the 3D display 160 may be a touch screen.

In addition, the 3D display 160 is a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), an organic light-emitting diode (Organic Light-Emitting Diode: OLED), flexible The display may include at least one of a flexible display, an e-ink display, and a light emitting diode (LED).

In addition, the 3D display 160 outputs (or displays) 3D content generated based on the collected various information under the control of the controller 190 . In this case, the 3D content includes route guidance information (or route guidance information) according to the destination information, etc., additional information on an object located in front of the vehicle (or an object included in the image information), detailed information related to the vehicle, and the like. Including, the route guidance information, the additional information, the detailed information, etc. are displayed adjacent to the real object located in front of the vehicle in consideration of the location of the object in the image information, or the user's viewing angle or the user and the object A position in the corresponding 3D content may be set to be displayed at a specific position in consideration of the focus of the liver. Here, the additional information (or additional information on the object located on the front/rear side of the vehicle) includes information such as the distance between the vehicle and the object, the speed, and additional information (for example, whether the vehicle is stolen, or not in arrears of a fine) ), etc. In addition, the detailed information (or detailed information related to the vehicle) includes various information related to the surrounding situation according to the location of the vehicle (eg, including speed limit information on the corresponding road, construction information on the surrounding road, etc.), the vehicle It includes related driving speed, engine rotation value, instantaneous fuel efficiency, real-time voltage, water temperature, mileage, alarm switch, overspeed alarm, water temperature alarm, etc.

In addition, the 3D heads-up display system 100 may further include a voice output unit (not shown) for outputting voice information included in a signal processed by a predetermined signal by the controller 190 . Here, the audio output unit may include a receiver, a speaker, a buzzer, and the like.

In addition, the voice output unit outputs a guide voice generated by the control unit 190 .

In addition, the audio output unit outputs audio information (or sound effects) included in the 3D content output through the 3D display 160 under the control of the controller 190 .

The reflector (or mirror unit/half mirror) 170 is disposed (or configured/formed) on one side of the vehicle in which the 3D head-up display system 100 is configured.

In addition, the reflector 170 reflects the 3D content output from the 3D display 160 to the windshield 180 .

The windshield 180 is disposed (or configured/formed) on one side of the windshield of the vehicle on which the 3D head-up display system 100 is configured. In this case, the windshield 180 may have a structure in which a reflective film (not shown) is formed between an outer plate glass (not shown) and an inner plate glass (not shown), or a transparent or translucent structure in which the reflective film is formed.

In addition, the windshield 180 displays an image (or virtual image/virtual image) in which 3D content output (or displayed) by the 3D display 160 is reflected by the reflector 170 . . Here, the reflected image is a state in which a focal length of the reflected image is adjusted in real space, and the reflected image may be displayed at an optimal focal position in real space.

In this case, when the 3D content includes additional information on an object located in front of the vehicle, detailed information related to the vehicle, etc., the windshield 180 is reflected on one side adjacent to the real object located in front of the vehicle. The corresponding additional information included in the image is displayed, and detailed information related to the corresponding vehicle included in the reflected image is displayed on the other side of the windshield 180 .

In addition, when the 3D content includes the route guidance information, the additional information, the detailed information, etc. according to the destination information, the windshield 180 is reflected on one side based on the actual lane located in front of the corresponding vehicle. Information on the vehicle's traveling direction according to the corresponding road guidance information included in the image (for example, information on going straight, left turn, right turn, lane use information, and information such as left turn/right turn/U-turn from a few meters ahead) is displayed, and the vehicle Additional information related to the corresponding object included in the reflected image is displayed on the other side of the windshield 180 adjacent to the real object located in front, and the reflected image is displayed on the other side of the windshield 180. Displays detailed information related to the included vehicle.

As such, with respect to the 3D content generated by the control unit 190 , the windshield 180 provides a single user (eg, a user located in a driver's seat or a passenger seat) or a plurality of users (eg, a plurality of 3D contents located in the driver's seat or the passenger seat). of the user) can provide different images to both eyes.

Also, as described above, the reflected image displayed on the windshield 180 provides an image corresponding to the information of the real object.

For example, by locating the name tag of an object (or object) 30 m away from the vehicle equipped with the 3D head-up display system 100 (or including additional information related to the object) as a virtual image, the user can It accepts the real and virtual images together and accepts them as one real image, or expresses the real object 30m away and the path to it as a 0-30m virtual image.

In this way, the 3D head-up display system 100 tracks the direction and location of the user's eye (or gaze) and expresses an appropriate virtual image through the 3D display 160, so that the real driver feels that there is an image in the distance. do.

The 3D heads-up display system 100 may further include a GPS receiver (not shown) for calculating (or acquiring) GPS location information of the 3D heads-up display system 100 .

That is, the GPS receiver receives a GPS signal transmitted from a satellite, and receives location data (or location information) of the 3D head-up display system 100 based on the longitude and latitude coordinates included in the received GPS signal. generate

In addition, the signal received through the GPS receiver includes 802.11, which is a standard standard for wireless networks for wireless LANs including wireless LANs and some infrared communications, proposed by the Institute of Electrical and Electronics Engineers (IEEE), Bluetooth, UWB, and Zigbee. 802.15, which is a standard specification for wireless PAN (Personal Area Network), including, etc., a wireless Metropolitan Area Network (MAN), including a fixed wireless access (FWA), and broadband wireless access (BWA) ) using wireless communication methods such as 802.16, which is a standard standard for , and 802.20, which is a standard for mobile Internet for wireless MAN (Mobile Broadband Wireless Access: MBWA) including Wibro and WiMAX. It may be configured to provide precise position information of the 3D head-up display system 100 to the 3D head-up display system 100 .

In this way, the GPS receiver receives location information of the corresponding 3D head-up display system 100 under the control of the controller 190 (or, when the 3D heads-up display system 100 is mounted on the vehicle, the location of the corresponding vehicle). information) is created.

The controller (or microcontroller unit (MCU) 190 ) is disposed (or configured/formed) on one side of the vehicle in which the 3D head-up display system 100 is configured.

In addition, the control unit 190 executes an overall control function of the 3D head-up display system 100 .

In addition, the control unit 190 executes the overall control function of the 3D head-up display system 100 using the program and data stored in the storage unit 150 . The controller 190 may include RAM, ROM, CPU, GPU, and a bus, and the RAM, ROM, CPU, GPU, etc. may be connected to each other through a bus. The CPU may access the storage unit 150 to perform booting using the O/S stored in the storage unit 150 , and use various programs, contents, data, etc. stored in the storage unit 150 . Thus, various operations can be performed.

In addition, the control unit 190 through the sensor unit 120 through the information (eg, including distance, speed, etc.) on the object located in the front of the vehicle measured (or sensed) through the sensor unit 120 . Collected information related to the corresponding vehicle, image information including one or more objects acquired through the photographing unit 130 , various information related to the surrounding situation of the corresponding vehicle received through the communication unit 110 , and the received destination 3D content to be provided to the user is generated based on the information, the sensed user's gaze direction and information on the gaze position. In this case, the 3D content includes route guidance information (or route guidance information) according to the destination information, etc., additional information on an object located in front of the vehicle (or an object included in the image information), detailed information related to the vehicle, and the like. Including, the route guidance information, the additional information, the detailed information, etc. are displayed adjacent to the real object located in front of the vehicle in consideration of the location of the object in the image information, or the user's viewing angle or the user and the object A position in the corresponding 3D content may be set to be displayed at a specific position in consideration of the focus of the liver. Here, the additional information includes a distance between the vehicle and a corresponding object, a speed, and additional information (eg, including information such as whether a stolen vehicle is in arrears, etc.). In addition, the detailed information includes various information related to the surrounding situation according to the location of the vehicle (for example, including speed limit information on the corresponding road, construction information on the surrounding road, etc.), information related to the collected vehicle (eg, For example, the vehicle-related driving speed, engine rotation value, instantaneous fuel efficiency, real-time voltage, water temperature, mileage, alarm switch, speed warning, water temperature alarm, etc.) are included.

In this way, the generated 3D content displays additional information related to the corresponding object at a location corresponding to the real object, displays detailed information related to the vehicle at a preset specific location, and displays the vehicle-related detailed information at a location corresponding to the actual lane. It may be information in consideration of the distance between the user's eyes and the HUD image (or virtual image) so as to display information on the traveling direction of the corresponding vehicle according to the generated road guide information.

In addition, when the user's gaze moves to one side based on the sensed information on the user's gaze direction and gaze position, the control unit 190 reflects the moving user's gaze direction and gaze position to the 3D content. , In response to a change in the user's gaze direction and movement of the gaze position, various information (including, for example, road guidance information, additional information, detailed information, etc.) included in the 3D content is moved in the opposite direction corresponding to the movement change can do it

For example, when moving to the left by about 5 cm while facing the front, the control unit 190 changes the corresponding movement (for example, 5 cm to the left) with direction guide information, additional information, and detailed information included in the 3D content. ), the corrected 3D content is output through the 3D display 160 in a state of moving about 5 cm in the opposite direction, and the various information is moved in the direction opposite to the change in the user's gaze movement compared to the previous 3D content. The corrected 3D content may be displayed through the windshield 180 .

Also, as described above, an image located at a desired distance may be provided to the user by using a 3D display capable of providing different information to both eyes of the user.

In addition, as described above, a 3D image is generated using the information obtained through the communication unit, the sensor unit, the photographing unit, etc., and the 3D image displayed through the 3D display is reflected through the reflector to apply the virtual 3D HUD image to the windshield. can be displayed

Hereinafter, a method of controlling a 3D head-up display system according to the present invention will be described in detail with reference to FIGS. 1 to 6 .

3 is a flowchart illustrating a control method of a 3D head-up display system according to an embodiment of the present invention.

First, the sensor unit 120 measures (or detects) information (eg, including distance, speed, etc.) on an object located in the traveling direction of the vehicle (or the front of the vehicle). Here, the object includes another vehicle, a building, a road sign, a traffic sign, and the like located in front of the vehicle. In this case, the sensor unit 120 may measure information on an object located in a direction opposite to the traveling direction of the vehicle (or at the rear of the vehicle).

In addition, the sensor unit 120 collects information related to the vehicle (including, for example, driving speed, engine RPM, instantaneous fuel efficiency, real-time voltage, water temperature, mileage, etc.). In this case, the sensor unit 120 may collect information related to the vehicle measured through various other sensors (not shown) provided in the vehicle.

Also, the photographing unit 130 acquires (or captures) image information including one or more objects located in front of the vehicle. In this case, the photographing unit 130 may acquire other image information including one or more other objects located at the rear of the vehicle.

In addition, the communication unit 110 transmits the real-time location of the vehicle provided from the server (not shown), information related to the surrounding situation according to the object in the image information obtained through the photographing unit 130, information related to the object, and the like. receive

That is, the communication unit 110 is controlled by the control unit 190, the location information of the vehicle checked in real time, the image information obtained through the photographing unit 130 (or information about the object in the image information ( For example, information related to the surrounding situation according to the location of the vehicle (or surrounding situation information) transmitted from the server in response to the transmission and the like)) of an object such as a vehicle) (For example, including speed limit information on the corresponding road, construction information on surrounding roads, etc.), additional information related to the object in the image information (including information such as whether the vehicle is stolen, whether or not a fine for negligence is not paid), etc. .

In addition, the 3D head-up display system 100 receives destination information according to a user input through an input unit (not shown).

In this way, the communication unit 110, the sensor unit 120, the photographing unit 130, the input unit, etc. included in the 3D head-up display system 100 provide information related to the vehicle, information related to the vehicle in which the vehicle is driving, and the like. Various information about one or more objects located in the front of the vehicle and/or at the rear of the vehicle corresponding to the driving direction may be collected (or measured/detected).

For example, the sensor unit 120 may include a plurality of vehicles, a plurality of buildings (or buildings), etc. located in the traveling direction of the first vehicle in which the 3D head-up display system 100 provided with the sensor unit 120 is configured. Measure information.

In addition, the sensor unit 120 collects information related to the first vehicle (eg, driving speed, engine rotation value, instantaneous fuel efficiency, real-time voltage, water temperature, mileage, etc.).

In addition, as shown in FIG. 4 , the photographing unit 130 acquires first image information 400 including a plurality of vehicles, a plurality of buildings (or buildings), etc. located in the traveling direction of the first vehicle. do.

In addition, the communication unit 110 transmits, under the control of the control unit 190 , the location information of the first vehicle checked in real time, the first image information obtained through the photographing unit 130 , and the like to the server. and first surrounding situation information related to the surrounding situation in which the first vehicle is currently located, transmitted from the server in response to the information transmission (eg, first speed limit information on the driving road (eg, 80 km/h)) , second additional information about the second vehicle included in the first image information (for example, there is no state of arrears in a fine for negligence), and the like.

As another example, the input unit receives first destination information (eg, Sinsa Station intersection) according to the user input (S310).

In addition, the gaze tracking unit 140 detects (or tracks) the gaze direction and gaze position of the user (or the driver) located in the driver's seat in the vehicle (or the direction and position in which the user looks out of the vehicle).

That is, the eye tracking unit 140 detects the direction and position in which the user located in the driver's seat looks at the front of the vehicle or a side mirror (not shown) configured on the side of the vehicle.

For example, the gaze tracking unit 140 detects a first gaze direction and gaze position of the user located in the driver's seat in the vehicle in real time (S320).

Thereafter, the control unit 190 receives information (eg, including distance, speed, etc.) on an object located in the front of the vehicle measured (or sensed) through the sensor unit 120 , through the sensor unit 120 . Collected information related to the corresponding vehicle, image information including one or more objects acquired through the photographing unit 130 , various information related to the surrounding situation of the corresponding vehicle received through the communication unit 110 , and the received destination 3D content to be provided to the user is generated based on the information, the sensed user's gaze direction and information on the gaze position. In this case, the 3D content includes route guidance information (or route guidance information) according to the destination information, etc., additional information on an object located in front of the vehicle (or an object included in the image information), detailed information related to the vehicle, and the like. Including, the route guidance information, the additional information, the detailed information, etc. are displayed adjacent to the real object located in front of the vehicle in consideration of the location of the object in the image information, or the user's viewing angle or the user and the object A position in the corresponding 3D content may be set to be displayed at a specific position in consideration of the focus of the liver. Here, the additional information includes a distance between the vehicle and a corresponding object, a speed, and additional information (eg, including information such as whether the vehicle is stolen or not in arrears with a fine). In addition, the detailed information includes various information related to the surrounding situation according to the location of the vehicle (for example, including speed limit information on the corresponding road, construction information on the surrounding road, etc.), information related to the collected vehicle (eg, For example, the vehicle-related driving speed, engine rotation value, instantaneous fuel efficiency, real-time voltage, water temperature, mileage, alarm switch, speed warning, water temperature alarm, etc.) are included.

For example, the control unit 190 may include information about a plurality of vehicles, a plurality of buildings, etc. located in front of the vehicle measured through the sensor unit 120 , and related to the first vehicle collected through the sensor unit 120 . information (including, for example, driving speed, engine rotation value, instantaneous fuel efficiency, real-time voltage, water temperature, mileage, etc.), a plurality of vehicles located in the traveling direction of the first vehicle obtained through the photographing unit 130, First image information including a plurality of buildings (or buildings), and first surrounding situation information (for example, first on the driving road) related to the surrounding situation in which the first vehicle is currently located, received through the communication unit 110 . 1 speed limit information (for example, 80 km/h)), second additional information about the second vehicle included in the first image information (for example, no arrears in fines), detected through the eye tracking unit 140 The first 3D content to be provided to the user is generated based on the user's first gaze direction and information on the gaze position. In this case, the first 3D content includes second additional information about the second vehicle (eg, a first distance (eg, 50 m) between the first vehicle and the second vehicle), and a second speed of the second vehicle. (including, for example, 80 km/h), no non-payment of fines, etc.), first detailed information about the first vehicle (eg, first speed limit information on the driving road (eg, 80 km/h), the first It includes the running speed of the vehicle (for example, 70 km/h), the rotational value of the engine (for example, 2,000 RPM), etc.).

As another example, the control unit 190 generates first route guidance information according to the real-time current location (eg, Yeoksam Station intersection) of the first vehicle according to the received first destination information (eg, Sinsa Station intersection) do.

In addition, the control unit 190 includes information on a plurality of vehicles and a plurality of buildings located in front of the vehicle measured through the sensor unit 120 , and information related to the first vehicle collected through the sensor unit 120 . (including, for example, driving speed, engine rotation speed, instantaneous fuel efficiency, real-time voltage, water temperature, mileage, etc.), a plurality of vehicles located in the traveling direction of the first vehicle acquired through the photographing unit 130 , a plurality of first image information including a building (or building) of Speed limit information (for example, 80 km/h)), second additional information about the second vehicle included in the first image information (for example, no fine arrears), the generated first route guidance information, the gaze The second 3D content to be provided to the user is generated based on the information on the first gaze direction and the gaze position of the user detected through the tracking unit 140 . In this case, the second 3D content includes second additional information about the second vehicle (eg, a first distance (eg, 50m) between the first vehicle and the second vehicle), and a second speed of the second vehicle. (including, for example, 80 km/h), no non-payment of fines, etc.), second detailed information about the first vehicle (eg, first speed limit information on the driving road (eg, 80 km/h), the first Information on the driving speed of the vehicle (eg, 70 km/h), the engine rotation value (eg, 2,000 RPM), and the direction of the vehicle according to the first route guidance information (eg, turn right before 659 m) ) and the like (S330).

Thereafter, the 3D display 160 outputs (or displays) the 3D content generated by the controller 190 .

That is, the 3D display 160 outputs the generated 3D content under the control of the controller 190 .

For example, the 3D display 160 configured (or disposed) at a specific location in the vehicle outputs the generated first 3D content under the control of the controller 190 .

As another example, the 3D display 160 outputs the generated second 3D content under the control of the controller 190 (S340).

Thereafter, the windshield 180 configured on the front of the vehicle is an image (or virtual image/virtual image) in which 3D content output (or displayed) by the 3D display 160 is reflected by the reflector 170 . to display Here, the reflected image is a state in which a focal length of the reflected image is adjusted in real space, and the reflected image may be displayed at an optimal focal position in real space.

In this case, when the 3D content includes additional information on an object located in front of the vehicle, detailed information related to the vehicle, etc., the windshield 180 is reflected on one side adjacent to the real object located in front of the vehicle. The corresponding additional information included in the image is displayed, and detailed information related to the corresponding vehicle included in the reflected image is displayed on the other side of the windshield 180 .

In addition, when the 3D content includes the route guidance information, the additional information, the detailed information, etc. according to the destination information, the windshield 180 is reflected on one side based on the actual lane located in front of the corresponding vehicle. Information on the vehicle's traveling direction according to the corresponding road guide information included in the image (for example, information on going straight, left turn, right turn, lane use information, and information such as left turn/right turn/U-turn after a few meters) is displayed, and the vehicle Additional information related to the corresponding object included in the reflected image is displayed on the other side of the windshield 180 adjacent to the real object located in front, and the reflected image is displayed on the other side of the windshield 180. Displays detailed information related to the included vehicle.

For example, as shown in FIG. 5 , the windshield 180 displays a first virtual image 500 in which the first 3D content output from the 3D display 160 is reflected by the reflector 170 . do. In this case, the second additional information 510 on the first vehicle included in the first 3D content is displayed adjacent to the second vehicle 520, and the first detailed information 530 on the first vehicle. is displayed at a preset first specific position.

As another example, as shown in FIG. 6 , the windshield 180 displays a second virtual image 600 in which the second 3D content output from the 3D display 160 is reflected by the reflector 170 . indicate In this case, the second additional information 610 on the second vehicle included in the second 3D content is displayed adjacent to the second vehicle 620, and the second detailed information 630 on the first vehicle. is displayed at a preset second specific location, and information on the traveling direction of the vehicle according to the first route guidance information (for example, turn right in front of 650m) 640 is displayed at a preset third specific location (S350) ).

As described above, the embodiment of the present invention provides the user with an image located at a desired distance using a 3D display that can provide different information to both eyes of the user, thereby providing a more realistic HUD image By providing users with information and displaying information in a state with less heterogeneity, user satisfaction can be increased.

In addition, as described above, the embodiment of the present invention generates a 3D image using information obtained through a communication unit, a sensor unit, a photographing unit, etc., and reflects the 3D image displayed through the 3D display through a reflector. By displaying the 3D HUD image of the virtual image on the windshield, it reduces the cost by composing the entire system in a compact size, and provides a wider use environment even with a smaller 3D display as the distance between the display and the eyes is closer, It can express information about objects and can express information about objects at multiple distances.

Those of ordinary skill in the art to which the present invention pertains may modify and modify the above-described contents without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: 3D head-up display system 110: communication unit
120: sensor unit 130: photographing unit
140: eye tracking unit 150: storage unit
160: 3D display 170: reflector
180: windshield 190: control unit

Claims (3)

a sensor unit that measures information on an object located in the front or rear of the vehicle, which is a traveling direction of the vehicle, and collects information related to the vehicle;
a photographing unit including a stereo camera capable of acquiring image information for a 360-degree omnidirectional view of the vehicle including the object;
a communication unit configured to receive additional information related to an object in the image information and surrounding situation information according to the location of the vehicle provided from a server;
a gaze tracking unit detecting a gaze direction and a gaze position of the user in the vehicle;
Information on the measured object located on the front or rear of the vehicle, the collected vehicle-related information, the acquired image information, the received surrounding situation information according to the location of the vehicle, the object in the received image information a control unit configured to generate 3D content to be provided to the user based on at least one of related additional information, the sensed gaze direction and gaze position of the user, and destination information according to a user input;
a 3D display for outputting the 3D content generated by the control unit;
a voice output unit for outputting voice information included in the 3D content generated by the control unit;
a reflector for reflecting the 3D content output by the 3D display; and
A reflective film is formed between the outer plate glass and the inner plate glass, comprising a windshield displaying an image reflected by the reflective plate,
The 3D content is
It includes at least one of route guidance information according to the destination information, additional information on an object located on the front or rear of the vehicle, and detailed information related to the vehicle,
The additional information is
It includes at least one of distance, speed, and additional information between the vehicle and the object,
The detailed information is
Including information related to the surrounding situation information and the collected vehicle according to the location of the vehicle,
The windshield is
When the 3D content includes additional information about an object located at the front or rear of the vehicle and detailed information related to the vehicle, the reflected image on one side of the windshield adjacent to a real object located at the front or rear of the vehicle Display the included additional information, and display the detailed information included in the reflected image on the other side of the windshield,
In the 3D display, the location in the 3D content is set so that the route guidance information, the additional information, and the detailed information are displayed in consideration of a user's viewing angle or a focus between a user and an object,
The control unit is
When the user's gaze moves to one side based on the detected information on the user's gaze direction and gaze position, the moving user's gaze direction and gaze position are reflected in the 3D content to reflect the user's gaze direction and gaze position At least one of route guidance information according to the destination information included in the 3D content, additional information on an object located on the front or rear of the vehicle, and detailed information related to the vehicle in response to a change in the movement of the vehicle corresponding to the change in movement moving in the opposite direction
3D head-up display system. According to claim 1,
a GPS receiver that receives a GPS signal transmitted from a satellite and calculates location information of the vehicle based on longitude and latitude coordinates included in the received GPS signal;
3D head-up display system, characterized in that it further comprises. 3. The method of claim 2,
a storage unit for storing the vehicle location information, the acquired image information, surrounding situation information according to the location of the vehicle, and additional information related to an object in the image information;
further comprising,
The image information acquired by the photographing unit is stored in a digital video recorder (DVR), stored in the storage unit, or transmitted to the server through the communication unit.

Images