KR102480704B1 - Apparatus for user-interface for a vehicle - Google Patents

Abstract

The present invention relates to a first display, a second display disposed at a distance from the first display, an input unit for obtaining a user input, and a user input for the first display obtained through the input unit according to a preset control mode. Based on the above, it relates to a vehicle user interface device including a processor for controlling a second display.

Description

Vehicle user interface device {Apparatus for user-interface for a vehicle}

The present invention relates to a vehicle user interface device.

A vehicle is a device that allows a user to move in a desired direction. A typical example is a car.

On the other hand, for the convenience of users using vehicles, various types of sensors and electronic devices are being provided. In particular, research on an Advanced Driver Assistance System (ADAS) is being actively conducted for user's driving convenience. Furthermore, development of an autonomous vehicle is being actively conducted.

A vehicle may include a user interface device to communicate with a user. In the user interface device, a function of receiving an input from a user and a function of providing information generated in a vehicle to a user may be performed by a single device, or a plurality of devices may separately perform a receiving function and a providing function, respectively. there is.

A user interface device may include an input unit for sensing a user's input. Also, the user interface device may include several display units. Different contents may be displayed on each display unit. In order to manipulate each of these display units, each separate input device is required, or when using one input device, there is a difficult manipulation method.

In addition, when a driver tries to manipulate content displayed on each display unit while driving, there is a problem in that driving safety is lowered because concentration is disturbed.

Therefore, it may be necessary to provide an improved user interface device so that a driver can safely and conveniently manipulate a plurality of displays even while driving.

An object of the present invention is to enable a driver to safely operate a user interface device having a plurality of displays even while driving in order to solve the above problems.

In addition, an object of an embodiment of the present invention is to provide a user interface device that allows a user to conveniently change an object to be controlled among a plurality of displays.

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, a vehicle user interface device according to an embodiment of the present invention includes a first display; a second display disposed at a position spaced apart from the first display; an input unit that obtains a user input; and a processor controlling the second display based on a user input for the first display acquired through the input unit according to a preset control mode.

In order to achieve the above object, a vehicle user interface device according to another embodiment of the present invention includes an interface unit; display; and a processor that obtains user input information from the mobile terminal through the interface unit and controls the display based on the user input information obtained from the mobile terminal.

Details of other embodiments are included in the detailed description and drawings.

According to an embodiment of the present invention, one or more of the following effects are provided.

First, by selectively controlling the first display or the second display in the same way, the user can conveniently operate a user interface device having a plurality of displays.

Second, by providing a user interface device that can be conveniently operated by a user, the driver can safely operate the user interface device while driving.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a view showing the exterior of a vehicle according to an embodiment of the present invention.
2 is a view of a vehicle according to an embodiment of the present invention viewed from various external angles.
3 and 4 are views illustrating the interior of a vehicle according to an embodiment of the present invention.
5 and 6 are referenced diagrams for explaining objects according to an embodiment of the present invention.
7 is a block diagram referred to describe a vehicle according to an embodiment of the present invention.
8 is a block diagram of a user interface device according to an embodiment of the present invention.
9A and 9B are control flowcharts of a user interface device according to an embodiment of the present invention.
10A and 10B are diagrams for explaining a user interface device according to an embodiment of the present invention.
11A and 11B are diagrams for explaining a user interface device according to an embodiment of the present invention.
12A and 12B are diagrams for explaining a user interface device according to an embodiment of the present invention.
13A and 13B are diagrams for explaining a user interface device according to an embodiment of the present invention.
14A and 14B are diagrams for explaining a user interface device according to an embodiment of the present invention.
15A and 15B are diagrams for explaining a user interface device according to an embodiment of the present invention.
16A and 16B are diagrams for explaining a user interface device according to an embodiment of the present invention.
17A and 17B are diagrams for explaining a user interface device according to an embodiment of the present invention.
18 is a diagram for explaining a user interface device according to another embodiment of the present invention.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, the terms "comprise" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that it does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

A vehicle described in this specification may be a concept including a car and a motorcycle. Hereinafter, vehicles will be mainly described with respect to vehicles.

The vehicle described in this specification may be a concept including all of an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source.

In the following description, the left side of the vehicle means the left side of the driving direction of the vehicle, and the right side of the vehicle means the right side of the driving direction of the vehicle.

1 is a view showing the exterior of a vehicle according to an embodiment of the present invention.

2 is a view of a vehicle according to an embodiment of the present invention viewed from various external angles.

3 and 4 are views illustrating the interior of a vehicle according to an embodiment of the present invention.

5 and 6 are referenced diagrams for explaining objects according to an embodiment of the present invention.

7 is a block diagram referred to describe a vehicle according to an embodiment of the present invention.

Referring to FIGS. 1 to 7 , the vehicle 100 may include wheels rotated by a power source and a steering input device 510 for adjusting the driving direction of the vehicle 100 .

Vehicle 100 may be an autonomous vehicle.

The vehicle 100 may switch to an autonomous driving mode or a manual mode based on a user input.

For example, the vehicle 100 may switch from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on a user input received through the user interface device 200 .

The vehicle 100 may switch to an autonomous driving mode or a manual mode based on driving situation information.

The driving condition information may include at least one of object information outside the vehicle, navigation information, and vehicle condition information.

For example, the vehicle 100 may switch from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on driving situation information generated by the object detection device 300 .

For example, the vehicle 100 may switch from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on driving situation information received through the communication device 400 .

The vehicle 100 may switch from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on information, data, and signals provided from an external device.

When the vehicle 100 is operated in the autonomous driving mode, the autonomous vehicle 100 may be operated based on the driving system 700 .

For example, the autonomous vehicle 100 may operate based on information, data, or signals generated by the driving system 710 , the vehicle exit system 740 , and the parking system 750 .

When the vehicle 100 is operated in the manual mode, the autonomous vehicle 100 may receive a user input for driving through the driving control device 500 . Based on the user input received through the driving control device 500, the vehicle 100 may be driven.

The overall length means the length from the front part to the rear part of the vehicle 100, the width means the width of the vehicle 100, and the height means the length from the lower part of the wheel to the roof. In the following description, the overall length direction (L) is the standard direction for measuring the overall length of the vehicle 100, the overall width direction (W) is the standard direction for measuring the overall width of the vehicle 100, and the overall height direction (H) is the vehicle It may mean a direction that is a standard for measuring the total height of (100).

As illustrated in FIG. 7 , the vehicle 100 includes a user interface device 200, an object detection device 300, a communication device 400, a driving manipulation device 500, a vehicle driving device 600, and a driving system. 700 , a navigation system 770 , a sensing unit 120 , an interface unit 130 , a memory 140 , a control unit 170 and a power supply unit 190 .

Depending on embodiments, the vehicle 100 may further include components other than the components described herein, or may not include some of the components described herein.

The user interface device 200 is a device for communication between the vehicle 100 and a user. The user interface device 200 may receive a user input and provide information generated in the vehicle 100 to the user. The vehicle 100 may implement UI (User Interfaces) or UX (User Experience) through the user interface device 200 .

The user interface device 200 may include an input unit 210, an internal camera 220, a biological sensor 230, an output unit 250, and a processor 270.

Depending on embodiments, the user interface device 200 may further include components other than the described components or may not include some of the described components.

The input unit 210 is for receiving information from a user, and the data collected by the input unit 210 may be analyzed by the processor 270 and processed as a user's control command.

The input unit 210 may be disposed inside a vehicle. For example, the input unit 210 may include one area of a steering wheel, one area of an instrument panel, one area of a seat, one area of each pillar, and a door One area of the door, one area of the center console, one area of the head lining, one area of the sun visor, one area of the windshield or window It may be placed in one area or the like.

The input unit 210 may include a voice input unit 211 , a gesture input unit 212 , a touch input unit 213 , and a mechanical input unit 214 .

The voice input unit 211 may convert a user's voice input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170 .

The voice input unit 211 may include one or more microphones.

The gesture input unit 212 may convert a user's gesture input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170 .

The gesture input unit 212 may include at least one of an infrared sensor and an image sensor for detecting a user's gesture input.

Depending on the embodiment, the gesture input unit 212 may detect a user's 3D gesture input. To this end, the gesture input unit 212 may include a light output unit outputting a plurality of infrared lights or a plurality of image sensors.

The gesture input unit 212 may detect a user's 3D gesture input through a time of flight (TOF) method, a structured light method, or a disparity method.

The touch input unit 213 may convert a user's touch input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170 .

The touch input unit 213 may include a touch sensor for sensing a user's touch input.

Depending on the embodiment, the touch input unit 213 may be integrally formed with the display unit 251 to implement a touch screen. Such a touch screen may provide both an input interface and an output interface between the vehicle 100 and the user.

The mechanical input unit 214 may include at least one of a button, a dome switch, a jog wheel, and a jog switch. An electrical signal generated by the mechanical input unit 214 may be provided to the processor 270 or the controller 170 .

The mechanical input unit 214 may be disposed on a steering wheel, a center fascia, a center console, a cockpit module, or a door.

The internal camera 220 may acquire an image inside the vehicle. The processor 270 may detect the user's state based on the vehicle interior image. The processor 270 may obtain gaze information of the user from the vehicle interior image. The processor 270 may detect a user's gesture from the vehicle interior image.

The biometric sensor 230 may obtain user's biometric information. The biometric sensor 230 may include a sensor capable of obtaining user's biometric information, and may obtain user's fingerprint information, heart rate information, and the like, using the sensor. Biometric information may be used for user authentication.

The output unit 250 is for generating an output related to sight, hearing or touch.

The output unit 250 may include at least one of a display unit 251 , an audio output unit 252 , and a haptic output unit 253 .

The display unit 251 may display graphic objects corresponding to various pieces of information.

The display unit 251 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. display), a 3D display, and an e-ink display.

The display unit 251 and the touch input unit 213 may form a mutual layer structure or be integrally formed to implement a touch screen.

The display unit 251 may be implemented as a Head Up Display (HUD). When the display unit 251 is implemented as a HUD, the display unit 251 may include a projection module to output information through an image projected on a windshield or window.

The display unit 251 may include a transparent display. The transparent display may be attached to a windshield or window.

The transparent display may display a predetermined screen while having a predetermined transparency. In order to have transparency, transparent displays are transparent TFEL (Thin Film Electroluminescent), transparent OLED (Organic Light-Emitting Diode), transparent LCD (Liquid Crystal Display), transmissive transparent display, transparent LED (Light Emitting Diode) display may include at least one of them. Transparency of the transparent display can be adjusted.

Meanwhile, the user interface device 200 may include a plurality of display units 251a to 251g.

The display unit 251 includes one area of the steering wheel, one area 251a, 251b, and 251e of the instrument panel, one area 251d of the seat, one area 251f of each pillar, and one area of the door ( 251g), one area of the center console, one area of the headlining, one area of the sun visor, or one area 251c of the windshield and one area 251h of the window.

The audio output unit 252 converts an electrical signal provided from the processor 270 or the controller 170 into an audio signal and outputs the converted audio signal. To this end, the sound output unit 252 may include one or more speakers.

The haptic output unit 253 generates a tactile output. For example, the haptic output unit 253 may vibrate the steering wheel, seat belt, and seats 110FL, 110FR, 110RL, and 110RR so that the user can recognize the output.

The processor 270 may control overall operations of each unit of the user interface device 200 .

Depending on embodiments, the user interface device 200 may include a plurality of processors 270 or may not include a plurality of processors 270 .

When the processor 270 is not included in the user interface device 200, the user interface device 200 may be operated under the control of a processor of another device in the vehicle 100 or the control unit 170.

Meanwhile, the user interface device 200 may be referred to as a vehicle display device.

The user interface device 200 may be operated under the control of the controller 170 .

The object detection device 300 is a device for detecting an object located outside the vehicle 100 . The object detection device 300 may generate object information based on the sensing data.

The object information may include information about the presence or absence of an object, location information of the object, distance information between the vehicle 100 and the object, and relative speed information between the vehicle 100 and the object.

Objects may be various objects related to driving of the vehicle 100 .

Referring to FIGS. 5 and 6 , the object O includes a lane OB10, another vehicle OB11, a pedestrian OB12, a two-wheeled vehicle OB13, traffic signals OB14 and OB15, lights, roads, structures, May include speed bumps, terrain objects, animals, and the like.

The lane OB10 may be a driving lane, a lane adjacent to the driving lane, or a lane in which an opposing vehicle drives. The lane OB10 may be a concept including left and right lines forming the lane. A lane may be a concept including an intersection.

The other vehicle OB11 may be a vehicle running around the vehicle 100 . The other vehicle may be a vehicle located within a predetermined distance from the vehicle 100 . For example, the other vehicle OB11 may be a vehicle preceding or following the vehicle 100 .

The pedestrian OB12 may be a person located around the vehicle 100 . The pedestrian OB12 may be a person located within a predetermined distance from the vehicle 100 . For example, the pedestrian OB12 may be a person located on a sidewalk or road.

The two-wheeled vehicle OB13 may refer to a vehicle that is located around the vehicle 100 and moves using two wheels. The two-wheeled vehicle OB13 may be a vehicle having two wheels located within a predetermined distance from the vehicle 100 . For example, the two-wheeled vehicle OB13 may be a motorcycle or bicycle located on a sidewalk or road.

The traffic signal may include a traffic light OB15, a traffic sign OB14, and a pattern or text drawn on a road surface.

The light may be light generated from a lamp provided in another vehicle. The light may be light generated from a street lamp. The light may be sunlight.

The road may include a road surface, a curve, an incline such as an uphill or a downhill.

The structure may be an object located near a road and fixed to the ground. For example, structures may include streetlights, roadside trees, buildings, telephone poles, traffic lights, bridges, curbs, and walls.

A feature may include a mountain, a hill, and the like.

Meanwhile, objects may be classified into moving objects and stationary objects. For example, the moving object may be a concept including another moving vehicle and a moving pedestrian. For example, the stopped object may be a concept including a traffic signal, a road, a structure, another stopped vehicle, and a stopped pedestrian.

The object detection device 300 may include a camera 310, a radar 320, a lidar 330, an ultrasonic sensor 340, an infrared sensor 350, and a processor 370.

Depending on embodiments, the object detection apparatus 300 may further include components other than the described components or may not include some of the described components.

The camera 310 may be located at an appropriate location outside the vehicle in order to obtain an external image of the vehicle. The camera 310 may be a mono camera, a stereo camera 310a, an around view monitoring (AVM) camera 310b, or a 360-degree camera.

The camera 310 may obtain position information of an object, distance information with respect to the object, or relative speed information with respect to the object by using various image processing algorithms.

For example, the camera 310 may obtain distance information and relative speed information with respect to the object based on a change in the size of the object over time in the obtained image.

For example, the camera 310 may obtain distance information and relative speed information with respect to an object through a pinhole model, road profiling, and the like.

For example, the camera 310 may obtain distance information and relative speed information with respect to an object based on disparity information in a stereo image acquired by the stereo camera 310a.

For example, the camera 310 may be disposed in the interior of the vehicle and proximate to the front windshield to obtain an image of the front of the vehicle. Alternatively, the camera 310 may be disposed around a front bumper or a radiator grill.

For example, the camera 310 may be disposed in the interior of the vehicle and proximate to the rear glass to obtain an image behind the vehicle. Alternatively, the camera 310 may be disposed around a rear bumper, a trunk, or a tailgate.

For example, the camera 310 may be disposed close to at least one of the side windows in the interior of the vehicle to obtain an image of the side of the vehicle. Alternatively, the camera 310 may be disposed around side mirrors, fenders, or doors.

The camera 310 may provide the acquired image to the processor 370 .

The radar 320 may include an electromagnetic wave transmitter and a receiver. The radar 320 may be implemented in a pulse radar method or a continuous wave radar method in terms of radio wave emission principles. The radar 320 may be implemented in a frequency modulated continuous wave (FMCW) method or a frequency shift keyong (FSK) method according to a signal waveform among continuous wave radar methods.

The radar 320 detects an object based on a Time of Flight (TOF) method or a phase-shift method through electromagnetic waves, and detects the position of the detected object, the distance to the detected object, and the relative speed. can be detected.

The radar 320 may be disposed at an appropriate location outside the vehicle to detect an object located in front, rear or side of the vehicle.

LiDAR 330 may include a laser transmitter and a receiver. The LIDAR 330 may be implemented in a Time of Flight (TOF) method or a phase-shift method.

LiDAR 330 may be implemented as a driven or non-driven type.

When implemented in a driving manner, the lidar 330 is rotated by a motor and may detect an object around the vehicle 100 .

When implemented as a non-driving type, the lidar 330 may detect an object located within a predetermined range with respect to the vehicle 100 by light steering. The vehicle 100 may include a plurality of non-powered lidars 330 .

The LIDAR 330 detects an object based on a Time of Flight (TOF) method or a phase-shift method through a laser light medium, and determines the location of the detected object, the distance to the detected object, and the Relative speed can be detected.

The lidar 330 may be disposed at an appropriate location outside the vehicle in order to detect an object located in front, rear or side of the vehicle.

The ultrasonic sensor 340 may include an ultrasonic transmitter and receiver. The ultrasonic sensor 340 may detect an object based on ultrasonic waves, and detect a position of the detected object, a distance to the detected object, and a relative speed.

The ultrasonic sensor 340 may be disposed at an appropriate location outside the vehicle to detect an object located in front, rear or side of the vehicle.

The infrared sensor 350 may include an infrared transmitter and a receiver. The infrared sensor 340 may detect an object based on infrared light and detect a position of the detected object, a distance to the detected object, and a relative speed.

The infrared sensor 350 may be disposed at an appropriate location outside the vehicle in order to detect an object located in front, rear or side of the vehicle.

The processor 370 may control overall operations of each unit of the object detection device 300 .

The processor 370 compares the data sensed by the camera 310, the radar 320, the lidar 330, the ultrasonic sensor 340, and the infrared sensor 350 with previously stored data to detect an object or can be classified.

The processor 370 may detect and track an object based on the obtained image. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with an object through an image processing algorithm.

For example, the processor 370 may obtain distance information and relative speed information with respect to the object based on a change in the size of the object over time in the obtained image.

For example, the processor 370 may obtain distance information and relative speed information with respect to an object through a pinhole model, road profiling, and the like.

For example, the processor 370 may obtain distance information and relative speed information with respect to an object based on disparity information in a stereo image obtained from the stereo camera 310a.

The processor 370 may detect and track the object based on the reflected electromagnetic wave that is transmitted and reflected by the object and returned. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with an object based on electromagnetic waves.

The processor 370 may detect and track the object based on reflected laser light that is transmitted and reflected by the object. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed to an object based on laser light.

The processor 370 may detect and track the object based on the reflected ultrasonic wave after the transmitted ultrasonic wave is reflected on the object. The processor 370 may perform operations such as calculating a distance with an object and calculating a relative speed with an object based on ultrasonic waves.

The processor 370 may detect and track the object based on the reflected infrared light that is transmitted and reflected by the object and returned. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with an object based on infrared light.

Depending on embodiments, the object detection device 300 may include a plurality of processors 370 or may not include the processor 370 . For example, each of the camera 310, the radar 320, the lidar 330, the ultrasonic sensor 340, and the infrared sensor 350 may individually include a processor.

When the processor 370 is not included in the object detection device 300, the object detection device 300 may be operated according to the control of the processor or the control unit 170 of the device in the vehicle 100.

The object detection device 300 may be operated under the control of the controller 170 .

The communication device 400 is a device for communicating with an external device. Here, the external device may be another vehicle, a mobile terminal, or a server.

The communication device 400 may include at least one of a transmission antenna, a reception antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication.

The communication device 400 includes a short-distance communication unit 410, a location information unit 420, a V2X communication unit 430, an optical communication unit 440, a broadcast transceiver 450, an Intelligent Transport Systems (ITS) communication unit 460, and a processor. (470).

Depending on embodiments, the communication device 400 may further include components other than the described components, or may not include some of the described components.

The short-range communication unit 410 is a unit for short-range communication. The short-range communication unit 410 includes Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), and Wireless (Wi-Fi). -Fidelity), Wi-Fi Direct, and wireless USB (Wireless Universal Serial Bus) technologies may be used to support short-distance communication.

The short-range communication unit 410 may perform short-range communication between the vehicle 100 and at least one external device by forming wireless area networks.

The location information unit 420 is a unit for acquiring location information of the vehicle 100 . For example, the location information unit 420 may include a Global Positioning System (GPS) module or a Differential Global Positioning System (DGPS) module.

The V2X communication unit 430 is a unit for performing wireless communication with a server (V2I: Vehicle to Infrastructure), another vehicle (V2V: Vehicle to Vehicle), or a pedestrian (V2P: Vehicle to Pedestrian). The V2X communication unit 430 may include an RF circuit capable of implementing communication with infrastructure (V2I), vehicle-to-vehicle communication (V2V), and pedestrian communication (V2P) protocols.

The optical communication unit 440 is a unit for communicating with an external device via light. The optical communication unit 440 may include an optical transmitter that converts an electrical signal into an optical signal and transmits the optical signal to the outside and an optical receiver that converts the received optical signal into an electrical signal.

Depending on the embodiment, the light emitting unit may be integrally formed with a lamp included in the vehicle 100 .

The broadcast transceiver 450 is a unit for receiving a broadcast signal from an external broadcast management server or transmitting a broadcast signal to the broadcast management server through a broadcast channel. Broadcast channels may include satellite channels and terrestrial channels. The broadcasting signal may include a TV broadcasting signal, a radio broadcasting signal, and a data broadcasting signal.

The ITS communication unit 460 may exchange information, data or signals with the traffic system. The ITS communication unit 460 may provide acquired information and data to the traffic system. The ITS communication unit 460 may receive information, data or signals from the traffic system. For example, the ITS communication unit 460 may receive road traffic information from the traffic system and provide the received road traffic information to the controller 170 . For example, the ITS communication unit 460 may receive a control signal from the traffic system and provide the received control signal to the controller 170 or a processor included in the vehicle 100 .

The processor 470 may control the overall operation of each unit of the communication device 400.

Depending on embodiments, the communication device 400 may include a plurality of processors 470 or may not include a plurality of processors 470 .

When the processor 470 is not included in the communication device 400, the communication device 400 may be operated under the control of a processor of another device in the vehicle 100 or the control unit 170.

Meanwhile, the communication device 400 may implement a vehicle display device together with the user interface device 200 . In this case, the vehicle display device may be referred to as a telematics device or an audio video navigation (AVN) device.

The communication device 400 may operate under the control of the control unit 170 .

The driving control device 500 is a device that receives a user input for driving.

In the case of the manual mode, the vehicle 100 may be operated based on a signal provided by the driving control device 500 .

The driving control device 500 may include a steering input device 510 , an acceleration input device 530 and a brake input device 570 .

The steering input device 510 may receive an input of a driving direction of the vehicle 100 from a user. The steering input device 510 is preferably formed in a wheel shape so that steering input is possible by rotation. Depending on the embodiment, the steering input device may be formed in the form of a touch screen, touch pad, or button.

The acceleration input device 530 may receive an input for acceleration of the vehicle 100 from a user. The brake input device 570 may receive an input for decelerating the vehicle 100 from a user. The acceleration input device 530 and the brake input device 570 are preferably formed in the form of pedals. Depending on the embodiment, the acceleration input device or brake input device may be formed in the form of a touch screen, touch pad, or button.

The driving control device 500 may be operated according to the control of the control unit 170 .

The vehicle driving device 600 is a device that electrically controls driving of various devices in the vehicle 100 .

The vehicle driving device 600 may include a power train driving unit 610, a chassis driving unit 620, a door/window driving unit 630, a safety device driving unit 640, a lamp driving unit 650, and an air conditioning driving unit 660. can

Depending on embodiments, the vehicle driving apparatus 600 may further include components other than the described components or may not include some of the described components.

Meanwhile, the vehicle driving device 600 may include a processor. Each unit of the vehicle driving device 600 may individually include a processor.

The power train driver 610 may control the operation of the power train device.

The power train driving unit 610 may include a power source driving unit 611 and a transmission driving unit 612 .

The power source driver 611 may control the power source of the vehicle 100 .

For example, when a fossil fuel-based engine is a power source, the power source driver 611 may perform electronic control of the engine. This makes it possible to control the output torque of the engine and the like. The power source driver 611 may adjust the engine output torque according to the control of the control unit 170 .

For example, when a motor based on electric energy is a power source, the power source driver 611 may control the motor. The power source driver 611 may adjust the rotational speed and torque of the motor according to the control of the control unit 170 .

The transmission driving unit 612 may control the transmission.

The transmission driving unit 612 can adjust the state of the transmission. The transmission driving unit 612 may adjust the state of the transmission to forward (D), reverse (R), neutral (N), or parking (P).

Meanwhile, when the engine is the power source, the transmission driving unit 612 may adjust the engagement state of the gear in the forward (D) state.

The chassis driving unit 620 may control the operation of the chassis device.

The chassis driving unit 620 may include a steering driving unit 621 , a brake driving unit 622 and a suspension driving unit 623 .

The steering driver 621 may perform electronic control of a steering apparatus in the vehicle 100 . The steering drive unit 621 can change the traveling direction of the vehicle.

The brake driver 622 may perform electronic control of a brake apparatus in the vehicle 100 . For example, the speed of the vehicle 100 may be reduced by controlling the operation of brakes disposed on wheels.

Meanwhile, the brake driving unit 622 may individually control each of a plurality of brakes. The brake driving unit 622 may differently control braking force applied to a plurality of wheels.

The suspension driver 623 may perform electronic control of a suspension apparatus in the vehicle 100 . For example, the suspension driving unit 623 may control the vibration of the vehicle 100 to be reduced by controlling the suspension device when there is a curve on the road surface.

Meanwhile, the suspension driving unit 623 may individually control each of the plurality of suspensions.

The door/window driver 630 may perform electronic control of a door apparatus or a window apparatus in the vehicle 100 .

The door/window driver 630 may include a door driver 631 and a window driver 632 .

The door driver 631 may control the door device. The door driving unit 631 may control opening and closing of a plurality of doors included in the vehicle 100 . The door driver 631 may control opening or closing of a trunk or a tail gate. The door driver 631 may control opening or closing of the sunroof.

The window driving unit 632 may perform electronic control of a window apparatus. Opening or closing of a plurality of windows included in the vehicle 100 may be controlled.

The safety apparatus driver 640 may perform electronic control of various safety apparatuses in the vehicle 100 .

The safety device driving unit 640 may include an airbag driving unit 641 , a seat belt driving unit 642 , and a pedestrian protection device driving unit 643 .

The airbag driver 641 may perform electronic control of an airbag apparatus in the vehicle 100 . For example, the airbag driver 641 may control the airbag to be deployed when danger is detected.

The seat belt driver 642 may perform electronic control of a seat belt appartus in the vehicle 100 . For example, the seat belt driver 642 may control the occupants to be fixed to the seats 110FL, 110FR, 110RL, and 110RR using the seat belt when danger is detected.

The pedestrian protection device driving unit 643 may perform electronic control of the hood lift and the pedestrian airbag. For example, when detecting a collision with a pedestrian, the pedestrian protection device driver 643 may control the hood to be lifted up and the pedestrian airbag to be deployed.

The lamp driver 650 may perform electronic control of various lamp apparatuses in the vehicle 100 .

The air conditioning driving unit 660 may perform electronic control of an air cinditioner in the vehicle 100 . For example, when the temperature inside the vehicle is high, the air conditioning driving unit 660 may operate the air conditioner and control cool air to be supplied to the inside of the vehicle.

The vehicle driving apparatus 600 may include a processor. Each unit of the vehicle driving device 600 may individually include a processor.

The vehicle driving device 600 may be operated according to the control of the controller 170 .

The operating system 700 is a system that controls various operations of the vehicle 100 . The driving system 700 may be operated in an autonomous driving mode.

The driving system 700 may include a driving system 710 , a vehicle exit system 740 and a parking system 750 .

Depending on embodiments, the driving system 700 may further include components other than the described components or may not include some of the described components.

Meanwhile, the driving system 700 may include a processor. Each unit of the driving system 700 may individually include a processor.

Meanwhile, according to embodiments, when the driving system 700 is implemented in software, it may be a sub-concept of the control unit 170.

Meanwhile, according to an embodiment, the driving system 700 includes a user interface device 200, an object detection device 300 and a communication device 400, a driving manipulation device 500, a vehicle driving device 600, a navigation system 770 , the sensing unit 120 and the control unit 170 may be a concept including at least one of them.

The driving system 710 may perform driving of the vehicle 100 .

The driving system 710 may perform driving of the vehicle 100 by receiving navigation information from the navigation system 770 and providing a control signal to the vehicle driving device 600 .

The driving system 710 may receive object information from the object detection device 300 and provide a control signal to the vehicle driving device 600 to drive the vehicle 100 .

The driving system 710 may receive a signal from an external device through the communication device 400 and provide a control signal to the vehicle driving device 600 to drive the vehicle 100 .

The driving system 710 includes a user interface device 200, an object detection device 300 and a communication device 400, a driving manipulation device 500, a vehicle driving device 600, a navigation system 770, a sensing unit ( 120) and the control unit 170, it may be a concept of a system that performs driving of the vehicle 100.

Such a driving system 710 may be referred to as a vehicle driving control device.

The vehicle extraction system 740 may perform vehicle 100 extraction.

The vehicle exit system 740 may receive navigation information from the navigation system 770 and provide a control signal to the vehicle driving device 600 to exit the vehicle 100 .

The vehicle extraction system 740 may receive object information from the object detection device 300 and provide a control signal to the vehicle driving device 600 to extract the vehicle 100 .

The vehicle extraction system 740 may receive a signal from an external device through the communication device 400 and provide a control signal to the vehicle driving apparatus 600 to extract the vehicle 100 .

The vehicle exit system 740 includes a user interface device 200, an object detection device 300 and a communication device 400, a driving manipulation device 500, a vehicle driving device 600, a navigation system 770, a sensing unit ( 120) and the control unit 170, it may be a concept of a system that performs taking out of the vehicle 100.

Such a vehicle extraction system 740 may be referred to as a vehicle extraction control device.

The parking system 750 may perform parking of the vehicle 100 .

The parking system 750 may receive navigation information from the navigation system 770 and provide a control signal to the vehicle driving device 600 to park the vehicle 100 .

The parking system 750 may perform parking of the vehicle 100 by receiving object information from the object detection device 300 and providing a control signal to the vehicle driving device 600 .

The parking system 750 may receive a signal from an external device through the communication device 400 and provide a control signal to the vehicle driving device 600 to park the vehicle 100 .

The parking system 750 includes a user interface device 200, an object detection device 300 and a communication device 400, a driving manipulation device 500, a vehicle driving device 600, a navigation system 770, a sensing unit ( 120) and the control unit 170, it may be a concept of a system that performs parking of the vehicle 100.

Such a parking system 750 may be referred to as a vehicle parking control device.

The navigation system 770 may provide navigation information. The navigation information may include at least one of map information, set destination information, route information according to the destination setting, information on various objects on the route, lane information, and current location information of the vehicle.

The navigation system 770 may include a memory and a processor. The memory may store navigation information. The processor may control the operation of the navigation system 770 .

According to an embodiment, the navigation system 770 may receive information from an external device through the communication device 400 and update pre-stored information.

According to embodiments, the navigation system 770 may be classified as a sub-component of the user interface device 200 .

The sensing unit 120 may sense the state of the vehicle. The sensing unit 120 may include an inertial navigation unit (IMU) sensor, a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight detection sensor, a heading sensor, a position module, a vehicle Includes forward/reverse sensor, battery sensor, fuel sensor, tire sensor, steering sensor by steering wheel rotation, vehicle interior temperature sensor, vehicle interior humidity sensor, ultrasonic sensor, light sensor, accelerator pedal position sensor, brake pedal position sensor, etc. can do.

Meanwhile, an inertial navigation unit (IMU) sensor may include one or more of an acceleration sensor, a gyro sensor, and a magnetic sensor.

The sensing unit 120 includes vehicle attitude information, vehicle motion information, vehicle yaw information, vehicle roll information, vehicle pitch information, vehicle collision information, vehicle direction information, and vehicle location information (GPS information). ), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward/reverse information, battery information, fuel information, tire information, vehicle lamp information, vehicle internal temperature information, vehicle internal humidity information, steering wheel rotation Sensing signals for angle, external illuminance of the vehicle, pressure applied to the accelerator pedal, pressure applied to the brake pedal, and the like may be acquired.

In addition, the sensing unit 120 includes an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake temperature sensor (ATS), a water temperature sensor (WTS), and a throttle position sensor. (TPS), a TDC sensor, a crank angle sensor (CAS), and the like may be further included.

The sensing unit 120 may generate vehicle state information based on the sensing data. The vehicle state information may be information generated based on data sensed by various sensors provided inside the vehicle.

For example, vehicle state information includes vehicle attitude information, vehicle speed information, vehicle tilt information, vehicle weight information, vehicle direction information, vehicle battery information, vehicle fuel information, vehicle tire pressure information, Vehicle steering information, vehicle interior temperature information, vehicle interior humidity information, pedal position information, vehicle engine temperature information, and the like may be included.

The interface unit 130 may serve as a passage for various types of external devices connected to the vehicle 100 . For example, the interface unit 130 may have a port connectable to a mobile terminal, and may be connected to the mobile terminal through the port. In this case, the interface unit 130 can exchange data with the mobile terminal.

Meanwhile, the interface unit 130 may serve as a passage for supplying electric energy to a connected mobile terminal. When the mobile terminal is electrically connected to the interface unit 130, under the control of the controller 170, the interface unit 130 may provide electrical energy supplied from the power supply unit 190 to the mobile terminal.

The memory 140 is electrically connected to the controller 170 . The memory 140 may store basic data for the unit, control data for controlling the operation of the unit, and input/output data. The memory 140 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware. The memory 140 may store various data for overall operation of the vehicle 100, such as a program for processing or control by the control unit 170.

Depending on the embodiment, the memory 140 may be integrally formed with the controller 170 or may be implemented as a sub-component of the controller 170 .

The controller 170 may control overall operations of each unit in the vehicle 100 . The controller 170 may be referred to as an Electronic Control Unit (ECU).

The power supply unit 190 may supply power required for operation of each component according to the control of the control unit 170 . In particular, the power supply unit 190 may receive power from a battery inside the vehicle.

One or more processors and controllers 170 included in the vehicle 100 include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs ( It may be implemented using at least one of field programmable gate arrays, processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions.

<First Embodiment>

8 is a block diagram of a user interface device according to an embodiment of the present invention.

The user interface device 200 may include an input unit 210, a display unit 251, an interface unit 830, a memory 840, a processor 870, and a power supply unit 890. The display unit 251 may include a first display 810 and a second display 820 .

The first display 810 may display graphic objects corresponding to various pieces of information.

The first display 810 may include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display ( It may include at least one of a flexible display), a 3D display, and an e-ink display.

The first display 810 may form a mutual layer structure with the touch input unit 213 or be formed integrally with the touch input unit 213 to realize a touch screen.

The first display 810 may be disposed in one area inside the vehicle 100 .

For example, the first display 810 may be implemented in one area (251a and 251b of FIG. 3 and 251e of FIG. 4) of the instrument panel.

The description of the display unit 251 described above may be applied to the first display 810 .

Hereinafter, a case in which the first display 810 is a center information display (CID) of a vehicle will be described as an example.

The second display 820 may display graphic objects corresponding to various pieces of information.

The second display 820 may include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display ( It may include at least one of a flexible display), a 3D display, and an e-ink display.

The second display 820 may be disposed at a position spaced apart from the first display.

The second display 820 may be implemented as a Head Up Display (HUD).

When the second display 820 is implemented as a HUD, the second display 820 may include a projection module to output information through an image projected on a windshield or window.

When the second display 820 is implemented as a HUD, the second display 820 may project an image onto all or part of the windshield or window.

For example, the second display 820 may implement augmented reality (AR) by projecting an image onto a windshield or window and displaying the image superimposed on the environment outside the vehicle visible through the actual windshield. there is.

Alternatively, when the second display 820 is implemented as a HUD, information may be output by projecting an image onto the combiner.

The description of the display 251 described above may be applied to the second display 820 .

Hereinafter, a case where the second display 820 is a head-up display (HUD) displaying an image on a windshield of a vehicle will be described as an example.

The interface unit 830 may serve as a passage for various types of external devices connected to the vehicle 100 .

For example, the interface unit 830 may obtain driving situation information from at least one of the object detection device 300, the communication device 400, and other devices.

For example, the interface unit 830 may be electrically connected to the mobile terminal by wire or wirelessly.

The interface unit 830 may serve as a passage through which electrical energy is supplied to a connected mobile terminal.

The interface unit 830 may be structurally and functionally separated from or integrated with the aforementioned interface unit 130 .

The above description of the interface unit 130 may be applied to the interface unit 830 .

The memory 840 may store various data for overall operation of the user interface device 200, such as a program for processing or controlling the processor 870.

The memory 840 may be structurally and functionally separated from or integrated with the memory 140 described above.

According to embodiments, the memory 840 may be integrally formed with the processor 870 or may be implemented as a sub-component of the processor 870 .

The processor 870 may control the overall operation of each unit in the user interface device 200.

The processor 870 includes application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, and controllers. It may be implemented using at least one of controllers, micro-controllers, microprocessors, and electrical units for performing other functions.

Also, the interface unit 830, the memory 840, the first display 810, the second display 820, and the power supply unit 890 may have individual processors or may be integrated into the processor 870.

The processor 870 may control the second display 820 based on a user input for the first display 810 acquired through the input unit 210 according to the selected control mode.

The control mode may include a first mode for controlling the first display 810 based on a user input to the first display 810 . The control mode may include a second mode for controlling the second display 820 based on a user input to the first display 810 .

The control mode may be selected based on a user's input or may be selected by the processor 870 .

The processor 870 may determine the control mode according to the operating state of the first display 810 .

For example, the processor 870 determines that the control mode is the second mode when a user input to the first display 810 is detected while the screen of the first display 810 is turned off. can be judged and controlled.

For example, the processor 870 determines that the control mode is the first mode when a user input to the first display 810 is detected while the screen of the first display 810 is turned on. can be judged and controlled.

When the user's gaze toward the second display 820 is detected by the user's gaze detection unit 880, the processor 870 determines the second display 820 based on the user's input to the first display 810. can control.

The processor 870 may set the control mode to the second mode when the user's gaze toward the second display 820 is detected by the user's gaze detection unit 880 .

The processor 870 may set the control mode to the second mode when it is determined by the user gaze detection unit 880 that the user's gaze is directed toward the front of the vehicle in which the second display 820 is implemented.

The processor 870 may switch the control mode based on a user input.

The processor 870 may switch the control mode from the first mode to the second mode or from the second mode to the first mode when a pre-stored mode switching user input is sensed.

When the control mode before the user input is detected is the first mode, the processor 870 may set the control mode to the second mode when the mode switching user input pre-stored in the memory 840 is detected. When the control mode before the user input is detected is the second mode, the processor 870 may set the control mode to the first mode when the mode switching user input pre-stored in the memory 840 is detected.

The processor 870 may switch the control mode when a user's voice input that meets the conditions stored in the memory 840 is received through the voice input unit 211 . In this case, the mode switching user input may be a preset user voice input.

The processor 870 may switch the control mode when a user's gesture input that meets the conditions stored in the memory 840 is received through the gesture input unit 212 . In this case, the mode switching user input may be a preset user gesture input.

For example, the user's gesture input for switching the control mode may be a gesture input (called a swipe) moving from one side of the first display 810 to the other side. The processor 870 may switch the control mode when a swipe gesture input for the first display 810 is received.

For example, when a gesture input of swiping from left to right with respect to the first display 810 is received, the processor 870 may switch the control mode from the first mode to the second mode. When a gesture input of swiping from right to left with respect to the first display 810 is received, the processor 870 may switch the control mode from the second mode to the first mode.

The processor 870 may switch the control mode when a user's touch input satisfying a condition stored in the memory 840 is received through the touch input unit 213 . In this case, the mode switching user input may be a preset user touch input.

For example, the user touch input for switching the control mode may be an input of touching the first display 810 two or more times within a preset period of time. The processor 870 may switch the control mode when a user input of touching the first display 810 two or more times is received within a preset time period.

For example, the user touch input for switching the control mode may be an input of touching the first display 810 with a pressure equal to or higher than a preset value. The processor 870 may switch the control mode when a user input of touching the first display 810 with a pressure equal to or higher than a preset value is received.

For example, the user touch input for switching the control mode may be an input (called a swipe) for moving from one point on the first display 810 to another point while touching. The processor 870 may switch the control mode when a swipe touch input to the first display 810 is received.

For example, when a touch input of swiping from left to right with respect to the first display 810 is received, the processor 870 may switch the control mode from the first mode to the second mode. When a touch input of swiping from right to left with respect to the first display 810 is received, the processor 870 may switch the control mode from the second mode to the first mode.

The processor 870 may switch the control mode when a user's button input is received through the mechanical input unit 214 . In this case, the mode switching user input may be an input in which the user presses the control mode button.

The processor 870 may obtain from at least one of the object detection device 300, the communication device 400, and other devices through the interface unit 830. The processor 870 may determine a control mode based on driving situation information.

For example, the processor 870 may determine the control mode based on the speed of the vehicle.

For example, the processor 870 may determine that the control mode is the second mode when the speed of the vehicle is greater than or equal to a preset value.

The processor 870 may determine the degree of attention required of the user based on the driving situation information. The processor 870 may determine that the control mode is the second mode when the determined degree of attention is greater than or equal to a preset value, and determine that the control mode is the first mode when the determined degree of attention is less than the preset value. there is.

For example, the processor 870 may determine that the degree of attention required of the user is 'low' when the vehicle is driving on a straight road and no object is located on the movement path of the vehicle. . The processor 870 controls the first display 810 based on the user input when a user input for the first display 810 is obtained in a state in which the level of attention required of the user is 'low'. can do. That is, the processor 870 may control the first display 810 and/or the second display 820 in the first mode.

For example, the processor 870 may determine that the degree of attention required of the user is 'high' when an object is located on the moving path of the vehicle or when the speed of the vehicle is greater than or equal to a preset value. there is. The processor 870 controls the second display 820 based on the user input when a user input for the first display 810 is obtained in a state in which the level of attention required of the user is 'high'. can That is, the processor 870 may control the first display 810 and/or the second display 820 in the second mode.

Through this, in a situation where the driver's attention is required, it is possible to prevent an accident due to the driver's distraction by allowing the driver to manipulate only the content displayed on the head-up display.

The processor 870 may determine a control mode based on image information displayed on the second display 820 .

The processor 870 may determine that the control mode is the second mode when a user input for the first display 810 is received while a preset type of image is displayed on the second display 820. . That is, when a user input for the first display 810 is received while a preset type of image is displayed on the second display 820, the processor 870 displays the second display 820 based on the user input. ) can be controlled.

In the memory 840 , types of images displayed on the second display 820 may be stored as a first group and a second group.

When a user input for the first display 810 is received while an image corresponding to the first group is displayed on the second display 820, the processor 870 determines the first display 810 based on the user input. ) can be controlled.

When a user input for the first display 810 is received while an image corresponding to the second group is displayed on the second display 820, the processor 870 displays the second display 820 based on the user input. ) can be controlled.

When a user input to the first display 810 is sensed and the control mode is the first mode, the processor 870 determines at least one of the images displayed on the first display 810 on the second display 820 . The second display 820 may be controlled to display a portion of the second display 820 .

When a user input to the first display 810 is sensed and the control mode is the first mode, the processor 870 displays at least a part of the image displayed on the first display 810 for a predetermined time period, The second display 820 may be controlled.

When a user input to the first display 810 is sensed and the control mode is the first mode, the processor 870 controls at least some of the images displayed on the first display 810 while the user input is continuously sensed. The second display 820 may be controlled so that is displayed.

For example, while the user touches the first display 810, the processor 870 may control the second display 820 to display at least a part of an image displayed on the first display 810. there is.

When an image to be displayed on the first display 810 is displayed on the second display 820, the processor 870 controls the first display 810 so that the image is displayed with less than a preset brightness. can control.

Alternatively, the processor 870 may control the screen of the first display 810 to be turned off when an image to be displayed on the first display 810 is displayed on the second display 820 . That is, when an image that can be displayed on the first display 810 is displayed on the second display 820, the processor 870 may control not to display it on the first display 810.

When an additional user input is not received, the processor 870 may stop displaying at least some of the images displayed on the first display 810 on the second display 820 after a predetermined time has elapsed.

The processor 870 detects a user input to the first display 810 after at least a portion of the image displayed on the first display 810 is displayed on the second display 820 for a certain period of time and then disappears. The second display 820 may be controlled based on a user input.

The processor 870 may control the second display 820 to display a graphic object corresponding to a user input to the first display 810 .

The graphic object may include at least one of a circular shape, a polygonal shape, and a graphic image.

The graphic object may be a finger-shaped image. Through this, it is possible to increase awareness by allowing the user to recognize intuitively.

A method of controlling the processor 870 to display a graphic object will be described in more detail below.

The user interface device 200 may further include a user gaze sensor 880 .

The user gaze detector 880 may be structurally and functionally separated from or integrated with the input unit 210 .

The user gaze detection unit 880 may include at least one of an infrared sensor and an image sensor for detecting the user's gaze.

The user's line of sight detector 880 is composed of an internal camera and may obtain an image inside the vehicle.

Depending on the embodiment, the internal camera may be an RGB camera and/or a black/white camera capable of capturing black and white images.

The user gaze detection unit 880 may detect a user's gesture input spaced from the first display by image processing an image inside the vehicle acquired from an internal camera.

For example, the user gaze detection unit 880 may be provided to detect a user's gesture input provided within 20 cm from the first display.

The user gaze detector 880 may determine whether the user is looking ahead of the vehicle based on the driving situation information.

The user gaze detector 880 may obtain driving situation information from the object detection device 300, the communication device 400, or an external device.

For example, when the user manipulates the driving control device 500 in response to an object in front of the vehicle 100, the user's gaze detector 880 may determine that the user is looking in front of the vehicle.

For example, the user's line of sight detector 880 may determine that the user is looking in front of the vehicle when a user input that changes beyond a preset range is detected through the driving control device 500 of the vehicle 100. can

For example, when a user input for steering at a predetermined angle or more is received through the steering input device 510, the user line of sight sensor 880 may determine that the user is looking ahead of the vehicle.

For example, the user gaze sensor 880 may determine that the user is looking ahead of the vehicle when a user input accelerating at least a preset value is received through the acceleration input device 530 .

For example, the user's line of sight sensor 880 may determine that the user is looking ahead of the vehicle when a user input decelerating beyond a preset value is received through the brake input device 570 .

Through this, the user's gaze sensor 880 may indirectly determine whether the user's gaze is directed forward without directly detecting the user's gaze.

Meanwhile, when the user's gaze is indirectly determined based on the driving situation information, the user's gaze detection unit 880 may be structurally and functionally integrated with the processor 870 and implemented. That is, the processor 870 may determine whether the user is looking ahead of the vehicle based on the driving condition information.

The power supply 890 may supply power required for operation of each component under the control of the processor 870 . The power supply 890 may receive power from a battery inside the vehicle.

9A and 9B are control flowcharts of a user interface device according to an embodiment of the present invention.

The processor 870 may obtain a user input for the first display 810 through the input unit 210 (S100).

The input unit 210 may include a voice input unit 211 , a gesture input unit 212 , a touch input unit 213 , and a mechanical input unit 214 .

The processor 870 may obtain a user input converted into an electrical signal from at least one of the voice input unit 211 , the gesture input unit 212 , the touch input unit 213 , and the mechanical input unit 214 .

The processor 870 may obtain a user input for the first display 810 through the internal camera 220 . Specifically, the processor 870 may process an image of the inside of the vehicle acquired from the inside camera 220 . The processor 870 may generate information about user input through image processing.

The processor 870 may control the second display 820 based on a user input to the first display 810 according to the control mode (S200).

When a user input for the first display 810 is received, the processor 870 determines whether to control the first display 810 (first mode) or the second display 820 based on the user input. It is possible to determine whether to control (second mode) (S210).

For example, the processor 870 may determine the control mode based on driving condition information obtained from at least one device of the vehicle through the interface unit 830 .

For example, the processor 870 may determine that the control mode is the first mode when a first user input is received based on user input information stored in the memory 840 .

For example, the processor 870 may determine that the control mode is the second mode when a second user input is received based on user input information stored in the memory 840 .

The processor 870 may control the second display 820 based on a user input to the first display 810 according to the control mode (S220).

For example, when a first user input is received based on information about the user input stored in the memory 840, the processor 870 displays the first display 810 and/or the second display in the first mode. (820) can be controlled.

For example, when a second user input is received based on information about the user input stored in the memory 840, the processor 870 displays the first display 810 and/or the second display in the second mode. (820) can be controlled.

A method of controlling the second display 820 by the processor 870 will be described in more detail below.

10A and 10B are diagrams for explaining a user interface device according to an embodiment of the present invention.

The processor 870 may obtain a user input (G) for the first display 810 through the input unit 210 .

Referring to FIGS. 10A and 10B , the user interface device 200 may include a touch screen integrally equipped with a first display 810 and a touch input unit 210 .

The processor 870 may obtain the user's touch input (G) converted into an electrical signal in the touch input unit 210 of the touch screen.

The processor 870 may control the second display 820 based on the user input G to the first display 810 according to the control mode.

Referring to FIG. 10A , the second display 820 may be a head-up display that projects an image onto a windshield of a vehicle.

The processor 870 may control the second display 820 to display the graphic objects 1011 and 1012 .

When the user input G for the first display 810 is received in a state in which the screen of the first display 810 is turned off, the processor 870 performs a processing based on the user input G. 2 The display 820 can be controlled.

For example, in a state in which the screen of the first display 810 is turned off, the processor 870 displays the graphic object 1010 at a location corresponding to the user input G, so that the second display ( 820) can be controlled.

For example, the processor 870, in a state where the screen of the first display 810 is turned off, the second display ( 820) can be controlled.

For example, the processor 870 may turn off the first display 810 if there is no user input to the first display 810 for a preset period of time.

For example, the processor 870 may display the graphic object 1020 moving corresponding to the movement of the touched finger when the user touches the touch screen and moves without taking the hand off the second display 820. can control.

Referring to FIG. 10B , the processor 870, when a user input G to the first display 810 is received while the screen of the first display 810 is turned on, the user input ( Based on G), the first display 810 may be controlled.

For example, the processor 870 may turn on the first display 810 when the user touches the touch screen and then releases the hand.

For example, when the user touches the touch screen while the first display 810 is turned on, the processor 870 displays an image of the first display 810 on the second display 820. You can control what is displayed. The processor 870 may control the second display 820 to display the graphic object 1020 moving corresponding to the movement of the touched finger when the user touches the touch screen and moves without releasing the hand. there is.

The user interface device configured as described above can control the first display 810 or the second display 820 with the same user input according to the on/off state of the first display 810, so that the user can conveniently It has the advantage of being able to manipulate interface devices.

In addition, content displayed on the head-up display can be manipulated using a touch screen (eg, CID) generally provided in vehicles without a separate control device for manipulating the second display implemented as the head-up display. There are also advantages to having it.

11A and 11B are diagrams for explaining a user interface device according to an embodiment of the present invention.

The processor 870 may obtain a user input (G) for the first display 810 through the input unit 210 .

Referring to FIGS. 11A and 11B , the user interface device 200 may include a touch screen integrally equipped with a first display 810 and a touch input unit 210 . The user interface device 200 may include a second display 820 implemented as a head-up display that projects an image onto a windshield of a vehicle.

The processor 870 may obtain an electrical signal according to a user's touch input (G) from the touch input unit 210 of the touch screen.

The processor 870 may control the first mode when a user input G to the first area of the first display 810 is sensed.

The first area may be a predetermined partial area of the first display 810 .

Alternatively, the first area may be an area determined by the processor 870 based on the image displayed on the first display 810 .

Referring to FIG. 11A , the processor 870 displays a menu image on the second display 820 when a user input G to the first area on the first display 810 on which the menu 1110 is displayed is detected. The second display 820 may be controlled so that 1120 is displayed.

The processor 870 operates the second display 820 so that the graphic object 1130 moving in correspondence with the user input G for the first area on the first display 810 on which the menu 1110 is displayed is displayed. can control.

Referring to FIG. 11B , the processor 870 may control the second mode when a user input to an area other than the first area of the first display 810 is detected.

The processor 870 may control the second display 820 to display one or more contents 1140 .

When a user input (G) to the first region on the first display 810 is detected, the processor 870 generates a graphic object 1130 to correspond to the user input G to the first display 810. The second display 820 may be controlled so that is displayed.

The user interface device 200 configured as described above may control content of the head-up display according to a user input detected by the first display 810 .

Through this, there is an advantage in that the user can easily manipulate the contents of the head-up display with the same manipulation method by using a device that the user is familiar with, such as a CID of a vehicle.

12A and 12B are diagrams for explaining a user interface device according to an embodiment of the present invention.

The processor 870 may obtain user input information on the first display 810 through the input unit 210 .

Referring to FIGS. 12A and 12B , the user interface device 200 may include a touch screen integrally equipped with a first display 810 and a touch input unit 210 . The user interface device 200 may include a second display 820 implemented as a head-up display that projects an image onto a windshield of a vehicle.

The processor 870 may obtain the user's touch input (G) converted into an electrical signal in the touch input unit 210 of the touch screen.

When a user input (G) to the first display 810 is sensed, the processor 870 may determine the control mode based on the content displayed on the second display 820 .

The processor 870, when the user input (G) to the first display 810 is detected in a state where information requiring user input (G) is displayed on the second display 820, the control mode is set to the second mode. can be judged to be

Information requiring user input (G) may be defined as content that requires a certain input by the user in any way, rather than simply providing an alarm to the user.

For example, information requiring user input (G) may be a menu requesting user selection.

For example, the information requiring user input (G) is alarm information provided to the user, and the user may need to make a certain input in order to release the alarm.

For example, the information for which user input (G) is required may be a menu asking whether to execute at least one function of the vehicle.

Referring to FIG. 12A , while a menu 1210 asking whether to execute automatic cruise control (AAC) is displayed on the second display 820, a user input (G) to the first display 810 is displayed. ) is received, the processor 870 may control the second display 820 to display the graphic object 1220 corresponding to the user input G.

Meanwhile, in a state in which information requiring user input (G) is displayed on the second display 820 or in a state in which content is not displayed, the processor 870 may determine that the control mode is the second mode. .

Referring to FIG. 12B , when a user input (G) to the first display 810 is detected in a state in which content requiring user input (G) is not displayed on the second display 820, the processor ( 870 may control the first display 810 and the second display 820 so that the mirroring image 1230 of the image of the first display 810 is displayed on the second display 820 .

In this case, the processor 870 controls the first display 810 and the second display 820 so that the graphic object 1220 is displayed corresponding to the user input G to the first display 810. can

Using the user interface device 200 configured as described above, the user can more intuitively operate the head-up display. In addition, there is an advantage that a user-friendly interface can be provided.

13A and 13B are diagrams for explaining a user interface device according to an embodiment of the present invention.

The processor 870 may obtain a user input (G) for the first display 810 through the input unit 210 .

Referring to FIGS. 13A and 13B , the user interface device 200 may include a touch screen integrally equipped with a first display 810 and a touch input unit 210 . The user interface device 200 may include a second display 820 implemented as a head-up display that projects an image onto a windshield of a vehicle.

The processor 870 may obtain the user's touch input (G) converted into an electrical signal in the touch input unit 210 of the touch screen.

The processor 870 may identify different first user inputs and second user inputs based on the user input obtained through the input unit 210 .

For example, the first user input may be an input in which the user touches the touch screen with one finger. The second user input may be an input of simultaneously touching a plurality of points of the first display 810 . The second user input may be an input in which the user touches the touch screen with two fingers.

For example, the first user input may be an input in which the user touches the touch screen with a pressure less than a preset intensity. The second user input may be an input in which the user touches the touch screen with a pressure equal to or greater than a predetermined intensity.

When the first user input is sensed, the processor 870 may determine that the control mode is the first mode. When the second user input is sensed, the processor 870 may determine that the control mode is the second mode.

Referring to FIG. 13A , when a user's input to the first display 810 is detected as a touch input using two fingers, the processor 870 controls the touch screen and the second display 820 in the second mode. can

The processor 870 may control the second display 820 to display the graphic object 1320 corresponding to the second user input (touch input using two fingers) to the first display 810 .

Referring to FIG. 13B , when a user touches the touch screen with one finger, the processor 870 may control the touch screen and the second display 820 in the first mode.

When a first user input (a touch input using one finger) is detected on the first display 810, the processor 870 displays a mirroring image 1330 of the image of the first display 810 on the second display 820. ) may be displayed, the second display 820 may be controlled.

The processor 870 may control the second display 820 to display the graphic object 1320 at a location corresponding to the first user input on the first display 810 .

The user interface device 200 configured as described above has an advantage in that the user can conveniently manipulate the user interface device by manipulating the content of the touch screen or the content of the head-up display according to user inputs that are distinguished from each other.

That is, there is an advantage in that a user can conveniently manipulate content on a head-up display displayed at a location that is difficult to directly manipulate in a manner similar to a method of manipulating a relatively easily manipulable touch screen.

14A and 14B are diagrams for explaining a user interface device according to an embodiment of the present invention.

The processor 870 may obtain a user input (G) for the first display 810 through the input unit 210 .

Referring to FIGS. 14A and 14B , the first display 810 may be a touch screen integrally equipped with the touch input unit 210 . The second display 820 may be a head-up display that projects an image onto a windshield of a vehicle.

The processor 870 may obtain the user's touch input (G) converted into an electrical signal from the touch input unit 210 of the touch screen.

Referring to FIG. 14A , the first display 810 may be divided into a plurality of virtual areas.

For example, the first display 810 may be divided into nine virtual areas, from an upper left virtual area A1 to a lower right virtual area A9. Meanwhile, the first display 810 may be divided into fewer or more virtual regions than nine.

For example, the first display 810 may be divided into a plurality of virtual regions each having a rectangular shape. Meanwhile, the first display 810 may be divided into a plurality of virtual areas having polygons other than rectangles or different shapes.

Referring to FIG. 14B , the second display 820 may be divided into a number of virtual areas corresponding to those of the first display 810 .

For example, when the first display 810 is divided into 9 virtual regions each having a rectangular shape as shown in FIG. 14A , the second display 820 may also be divided into 9 virtual regions each having a rectangular shape.

For example, the second display 820 may include a first virtual area B1 of the second display 820 corresponding to the first virtual area A1 of the first display 810 . The second display 820 may include a second virtual area B2 of the second display 820 corresponding to the second virtual area A2 of the first display 810 . As such, the second display 820 may include a plurality of virtual areas B1 to B9 of the second display 820 corresponding to the plurality of virtual areas A1 to A9 of the first display 810. can

When a user input for one area of the first display 810 is detected, the processor 870 displays a graphic object on one area of the second display 820 corresponding to one area of the first display 810. As much as possible, the second display 820 can be controlled.

14B shows a state in which the outside of the vehicle including objects 1411 and 1412 outside the vehicle are seen through the vehicle windshield.

To display information related to the first object 1411, the processor 870 may display the first graphic object 1421 at a position adjacent to the first object 1411 outside the vehicle based on the user's gaze. can In this case, the processor 870 may display the first graphic object 1421 related to the first object 1411 located in the B4 virtual area on the B1 virtual area based on the screen of the second display 820 .

The processor 870 may display the second graphic object 1422 at a position adjacent to the second object 1412 outside the vehicle based on the user's gaze to display information related to the second object 1412. can At this time, the processor 870 may display the second graphic object 1422 related to the second object 1412 located in the B6 virtual area on the B3 virtual area based on the screen of the second display 820 .

When a user input (G) for the A3 virtual area of the first display 810 is sensed, the processor 870 performs B3 of the second display 820 corresponding to the A3 virtual area of the first display 810. The second graphic object 1422 located in the virtual area may be highlighted. For example, the processor 870 may control the second display 820 to change the brightness, color, or size of the second graphic object.

Meanwhile, although not shown, the second display 820 may not necessarily be divided into virtual areas corresponding to the number of virtual areas of the first display 810 . The second display 820 may have a number equal to or smaller than the number of virtual areas of the first display 810 . In this case, each virtual area of the first display 810 is set to correspond to any one virtual area of the second display 820, and when one virtual area of the first display 810 is touched, the second display 810 is touched. A graphic object may be displayed at one point in 820 .

15A and 15B are diagrams for explaining a user interface device according to an embodiment of the present invention.

The processor 870 may obtain user input information on the first display 810 through the input unit 210 .

Referring to FIGS. 15A and 15B , the user interface device 200 may include a touch screen integrally equipped with a first display 810 and a touch input unit 210 . The user interface device 200 may include a second display 820 implemented as a head-up display that projects an image onto a windshield of a vehicle.

The processor 870 may obtain the user's touch input (G) converted into an electrical signal in the touch input unit 210 of the touch screen.

Referring to FIG. 15A , the first display 810 may be divided into a plurality of virtual areas.

For example, the first display 810 may be divided into nine virtual areas, from an upper left virtual area A1 to a lower right virtual area A9.

Referring to FIG. 15B , the second display 820 may be divided into a number of virtual areas corresponding to those of the first display 810 .

For example, as shown in FIG. 15A , when the first display 810 is divided into 9 virtual regions each having a rectangular shape, the second display 820 may also be divided into 9 virtual regions each having a rectangular shape.

When a user input to the first display 810 is sensed to move from a first point to a second point, the processor 870 displays a graphic object moving from one point to another point on the second display ( 820) can be controlled.

Referring to FIG. 15A , the user may apply a touch input (G) moving from virtual area A8 to virtual area A6 of the first display 810 to the touch screen.

Referring to FIG. 15B , when a user input G to the first display 810 is sensed, the processor 870 moves to a preset location (eg, the center of the second display 820, B5 virtual area). The second display 820 may be controlled so that the graphic object 1520 is displayed on the screen.

The processor 870 provides graphics moving from virtual area B5 to virtual area B3 of the second display 820 in response to a user input G moving from virtual area A8 to virtual area A6 of the first display 810 . The object 1520 may be displayed.

The processor 870 may control the graphic object 1520 of the second display 820 to move to correspond to the vector value of the user input to the first display 810 .

For example, when the ratio between the left and right lengths of the first display 810 and the left and right lengths of the second display 820 is 1:1.5, the processor 870 receives a user input (G) for the first display 810 When moving 5 cm from left to right, the graphic object 1520 of the second display 820 may be controlled to move 7.5 cm from left to right. Similarly, the graphic object 1520 may be controlled by the processor 870 to move in proportion to the vertical ratio of the first display 810 and the second display 820 .

The processor 870 may change at least one of brightness, color, or size of the image 1510 displayed in the virtual area B3 to which the graphic object 1520 has moved. For example, when the graphic object 1520 moves to a position overlapping the image 1510, the processor 870 may control the image 1510 to blink.

The user interface device 200 configured as described above has an advantage in that the user can easily manipulate the contents of the head-up display without looking at the first display.

16A and 16B are diagrams for explaining a user interface device according to an embodiment of the present invention.

The processor 870 may obtain a user input (G) for the first display 810 through the input unit 210 .

Referring to FIGS. 16A and 16B , the first display 810 may be integrally implemented with the touch input unit 210 and configured as a touch screen. The second display 820 may include a second display 820 implemented as a head-up display that projects an image onto a windshield of a vehicle. The second display 820 may implement augmented reality (AR) by displaying an image corresponding to an object outside the vehicle visible through the windshield.

The processor 870 may obtain the user input G converted into an electrical signal by the touch input unit 210 of the first display 810 .

Referring to FIG. 16A , the first display 810 may be divided into a plurality of virtual areas.

For example, the first display 810 may be divided into nine virtual areas, from an upper left virtual area A1 to a lower right virtual area A9.

Referring to FIG. 15B , the second display 820 may be divided into a plurality of virtual areas. The processor 870 displays the second display 820 in a plurality of virtual areas based on the positions of the object 1610 outside the vehicle and/or the graphic objects 1621 and 1622 displayed on the second display 820. can be distinguished by

For example, as shown in FIG. 16B , the processor 870 sets the second display 820 to a virtual area B11 where the external object 1610 is located and a virtual area where the first graphic object 1621 is located. (B12) and a virtual area (B13) where the second graphic object 1622 is located.

The processor 870 may set corresponding virtual areas B11 , B12 , and B13 of the second display 820 to each of the plurality of virtual areas A1 to A9 of the first display 810 .

For example, the processor 870 may set the virtual areas A1, A4, and A7 of the first display to correspond to the virtual area B11. The processor 870 may set the virtual areas A2, A5, and A8 of the first display to correspond to the virtual area B12. The processor 870 may set the virtual areas A3, A6, and A9 of the first display to correspond to the virtual area B13.

When the user input to the first display 810 is sensed to move from the first point to the second point, the processor 870 changes the display position of the graphic object from one point to another point, so that the second point is displayed. The display 820 may be controlled.

Here, the meaning of changing the displayed position of a graphic object is defined as including a change in the position where the same graphic object is displayed or a graphic object displayed in a different position being highlighted more. It can be.

Referring to FIG. 15A , the user may apply a touch input (G) moving from virtual area A8 to virtual area A6 of the first display 810 to the touch screen.

When a user input (G) is detected in the A8 virtual area of the first display 810, the processor 870 displays the first graphic object 1621 of the B12 virtual area corresponding to the A8 virtual area with emphasis. 2 The display 820 can be controlled.

Referring to FIG. 15B , when a user input (G) moving from the virtual area A8 to the virtual area A6 of the first display 810 is detected, the processor 870 displays a second area of the virtual area B13 corresponding to the virtual area A6. The second display 820 may be controlled so that the graphic object 1622 is highlighted and displayed.

Meanwhile, similar to the relative movement of a graphic object to correspond to a user input with reference to FIGS. 15A and 15B , the processor 870 controls the position of the highlighted graphic object to be relatively moved to correspond to the user input. can do.

That is, the processor 870 may highlight and display the first graphic object 1621 located in the central virtual area B12 on the second display 820 when a user input (G) is sensed in the virtual area A8. Subsequently, as the user input (G) moves from virtual area A8 to virtual area A6 on the touch screen, the processor 870 highlights the second graphic object 1622 located in the virtual area B13 on the second display 820. can do.

The processor 870 may display the graphic object with emphasis through a display method that arouses the user's attention, including the color, size, shape, location, and flickering of the graphic object.

Meanwhile, the processor 870 may control the second display 820 to change the selected object or graphic object in the first direction in response to a user input moving in the first direction. That is, a graphic object to be selected (or emphasized) may be relatively changed to correspond to a user input.

For example, when a user input of swiping from A5 to A6 is detected while the first graphic object 1621 located in the virtual area B12 of FIG. 16B is selected, the processor 870 generates a second graphic object The second display 820 may be controlled so that the second graphic object 1622 positioned to the right is selected. The processor 870 may highlight and display the selected graphic object.

Based on the second display 820, when a direction parallel to the ground is referred to as an X-axis direction and a direction perpendicular to the ground is referred to as a Y-axis direction, the processor 870 determines the X-axis direction of the graphic objects 1621 and 1622 Coordinates or Y-axis direction coordinates can be determined.

The processor 870 changes the selected graphic object according to the X-axis or Y-axis coordinates of the graphic objects 1621 and 1622 when the user applies a swipe input in the X-axis direction or the Y-axis direction. can

The X-axis coordinate of the graphic object may be a value calculated for each graphic object according to a predetermined rule. For example, the X-axis coordinate of the graphic object may be a value calculated based on the center of the graphic object. For example, the X-axis coordinate of the graphic object may be a value determined based on the lower left corner of the graphic object. For example, the X-axis coordinate of the graphic object may be determined based on the left end or right end of the graphic object.

The Y-axis coordinate of the graphic object may be a value calculated for each graphic object according to a predetermined rule. For example, the Y-axis coordinate of the graphic object may be a value calculated based on the center of the graphic object. For example, the Y-axis coordinate of the graphic object may be a value determined based on the lower left corner of the graphic object. For example, the Y-axis coordinate of the graphic object may be determined based on the top or bottom of the graphic object.

A swipe input may be defined as an input moving a certain distance in one direction. For example, the swipe input may be moving from left to right while touching the touch screen and releasing the touch. For example, the swipe input may be a gesture that does not come into contact with the display, but moves from left to right within a certain distance and then departs a certain distance or more.

For example, in the state in which the external object 1610 is selected in FIG. 16B, when the user applies a swipe input from left to right, the first graphic object 1621 located to the right of the external object 1610 relative to the X-axis direction can be selected. In this case, if the user applies a swipe input from left to right once more, the second graphic object 1622 located to the right of the first graphic object 1621 based on the X-axis direction may be selected.

In this way, the processor 870 may change the selected object or graphic object when the user inputs an up/down or left/right swipe.

17A and 17B are diagrams for explaining a user interface device according to an embodiment of the present invention.

The processor 870 may obtain a user input (G) for the first display 810 through the input unit 210 .

Referring to FIGS. 16A and 16B , the first display 810 may be integrally implemented with the touch input unit 210 and configured as a touch screen. The second display 820 may include a second display 820 implemented as a head-up display that projects an image onto a windshield of a vehicle. The second display 820 may implement augmented reality (AR) by displaying an image corresponding to an object outside the vehicle visible through the windshield.

The processor 870 may obtain the driver's gaze information from the user's gaze detector 880 provided inside the vehicle.

Alternatively, the processor 870 may acquire an image inside the vehicle from an internal camera and generate driver gaze information through image processing.

The processor 870 implements augmented reality (AR) by controlling the second display 820 to display an image corresponding to an object outside the vehicle seen through the windshield based on the driver's gaze information. can

The processor 870 may obtain the user input G converted into an electrical signal by the touch input unit 210 of the first display 810 .

The touch input unit 210 may be configured to generate different electrical signals according to touch pressure.

The processor 870 may identify different third user inputs and fourth user inputs based on the user input obtained through the input unit 210 .

The third user input may be a user input that satisfies a third user input condition stored in the memory 840 . The fourth user input may be a user input that satisfies the fourth user input condition stored in the memory 840 .

For example, the third user input may be a touch input of pressing the touch screen with a pressure less than a preset value. The fourth user input may be a touch input pressing the touch screen more than a preset value.

For example, the third user input may be a touch input of pressing the touch screen for a time less than a preset value. The fourth user input may be a touch input of pressing the touch screen for a time longer than a preset value.

When a third user input is sensed, the processor 870 may display a graphic object on the second display 820 to correspond to an object located in a short distance from the vehicle. When a fourth user input is sensed, the processor 870 may control the second display to display a graphic object corresponding to an object located far from the vehicle.

17A and 17B illustrate a situation in which a first object 1711 and a second object 1712 appear to overlap each other when the user looks outside the vehicle through the windshield of the vehicle based on the driver's point of view.

When a third user input pressing the first display 810 with a pressure lower than a preset value is detected, the processor 870 selects the first object 1711 and the second object 1712 overlapping with each other to obtain information from the vehicle. A graphic object corresponding to the second object 1712 having a short distance may be displayed.

Referring to FIG. 17A , for example, the processor 870 controls the second display 820 to display a graphic object emphasizing the second object 1712 through the process described with reference to FIGS. 14A to 16B . can do.

Referring to FIG. 17B , in a state where a graphic object emphasizing the second object 1712 is displayed, if the user touches the second object 1712 with a certain strength or more, the processor 870 overlaps with the second object 1712 and appears further away from the vehicle. The second display 820 may be controlled to display a graphic object emphasizing the first object 1711 located far away.

The processor 870 may control the second display 820 to display a graphic object in a set color as an image having the same shape as or corresponding to the shape of the object, so that the object appears emphasized.

The graphic object emphasizing the object may be a frame shape surrounding the object or a figure shape pointing to the object. The processor 870 may control the second display 820 to display a graphic object having a shape of a border surrounding an object or a figure pointing to the object.

The user interface device 200 configured as described above has an advantage of providing a user-friendly control environment in manipulating content displayed on the head-up display.

<Second Embodiment>

18 is a diagram for explaining a user interface device according to another embodiment of the present invention.

The user interface device 200' according to this embodiment may include an interface unit 830', a display 820', and a processor 870'.

The user interface device 200 ′ according to this embodiment may further include a user gaze detector 880 .

The configuration of the user interface device 200' in the second embodiment of the present invention may be the same as that of the user interface device 200 in the first embodiment.

The display 820' may be implemented as a head-up display that projects an image onto a windshield of a vehicle. The display 820 ′ may implement augmented reality (AR) by displaying an image corresponding to an object outside the vehicle visible through the windshield.

Based on the driver's gaze information obtained from the user's gaze sensor 880, the processor 870' displays an image corresponding to an object outside the vehicle visible through the windshield to obtain augmented reality (AR). can be implemented

The description of the second display 820 in the first embodiment may be applied to the display 820'.

The interface unit 830' may be electrically connected to the mobile terminal M by wire or wirelessly.

The mobile terminal M may be defined as a portable computing device such as a mobile phone or a tablet computer.

The interface unit 830' may be automatically connected to the mobile terminal M when the mobile terminal M is mounted in a specific location in the vehicle.

For example, when the mobile terminal M is mounted on a cradle in a vehicle, the interface unit 830' may be automatically connected to the mobile terminal M.

The interface unit 830' may obtain user input information from the mobile terminal M.

The user input information may be information generated by a user input to the mobile terminal (M).

For example, the mobile terminal M may include a touch screen, and a touch sensing unit of the touch screen may convert a user input into an electrical signal and transmit the converted electrical signal to a processor of the mobile terminal M. The processor of the mobile terminal M may generate user input information based on the user input converted into an electrical signal.

The interface unit 830' may serve as a passage through which electrical energy is supplied to the connected mobile terminal M.

As for the interface unit 830', the description of the interface unit 830 of the first embodiment described above may be applied.

The processor 870' may control the overall operation of each unit in the user interface device 200'.

As for the processor 870', the description of the processor 870 of the first embodiment described above may be applied.

The processor 870' may obtain user input information from the mobile terminal M through the interface unit 830'. The processor 870' may control the display 820' based on user input information obtained from the mobile terminal M.

Similar to how the processor 870 of the first embodiment controls the first display 810 and/or the second display based on the user input information acquired from the input unit 210, the processor 870' of the second embodiment , The display 820' can be controlled based on user input information obtained from the mobile terminal M.

The processor 870' can control the display 820' in the same or similar manner using the acquired user input information, except that only the object to which the user input information is provided has been changed.

For example, when a user input is applied to the mobile terminal M while the screen of the mobile terminal M is turned off, the processor 870' processes the user input information obtained from the mobile terminal M Based on this, it is possible to control the display 820'.

Meanwhile, in a state in which the screen of the mobile terminal M is turned on, the processor of the mobile terminal M may control the mobile terminal M based on user input information.

As another example, the processor 870' may control the display 820' so that the graphic object 1820 is displayed on the display 820' to correspond to a user input for the mobile terminal M.

For example, when a user input for one area of the display of the mobile terminal M is detected, the processor 870' displays a graphic object in one area of the display 820' corresponding to one area of the display of the mobile terminal. 1820 can be controlled to be displayed.

For example, the processor 870' moves from one point on the display 820' to another point when a user input moving from a first point to a second point on the touch screen of the mobile terminal M is applied. It is possible to control the graphic object 1820 to be displayed.

In addition, the method of controlling the second display 820 based on the user input to the first display 810 described with reference to FIGS. 9 to 17 is displayed based on the user input to the mobile terminal M. 820' can be equally applied.

The user interface device 200' configured as described above uses the mobile terminal as an input unit for manipulating the head-up display and heads-up based on user input information obtained from the mobile terminal without having a separate input unit in the vehicle. You can control the contents of the display.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. , and also includes those implemented in the form of a carrier wave (eg, transmission over the Internet). Also, the computer may include a processor or a control unit. Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: vehicle 200: user interface device
210: input unit 810: first display
820: second display 830: interface unit
840: memory 870: processor
880: user gaze detection unit 890: power supply unit

Claims (20)

a first display;
a second display disposed at a position spaced apart from the first display;
an input unit that obtains a user input;
A processor for controlling the second display based on a user input for the first display acquired through the input unit according to a selected control mode;
The control mode is
A first mode for controlling the first display based on a user input for the first display;
A second mode for controlling the second display based on a user input for the first display;
the processor,
Based on the image information displayed on the second display, the control mode is determined;
The in-vehicle user interface device determines that the control mode is the second mode when a user input to the first display is detected in a state in which information requiring user input is displayed on the second display. According to claim 1,
the processor,
Controlling the second display based on a user input to the first display while the screen of the first display is turned off;
When a pre-stored mode switch user input is detected,
Switching the control mode from the first mode to the second mode or from the second mode to the first mode. According to claim 1,
Further comprising a user gaze detector,
the processor,
When the user's gaze toward the second display is detected by the user's gaze detection unit,
A vehicle user interface device configured to control the second display based on a user input to the first display. According to claim 1,
the processor,
When a user input to the first area of the first display is detected, control is performed in the second mode;
Controlling the vehicle user interface device to the first mode when a user input to an area other than the first area of the first display is detected. According to claim 1,
It further includes an interface unit;
the processor,
The control mode is determined based on driving situation information obtained from at least one of an object detection device, a communication device, and another device through the interface unit;
When the speed of the vehicle is equal to or greater than the predetermined value, it is determined that the control mode is the second mode. delete According to claim 1,
the processor,
Based on the user input obtained through the input unit, different first user input and second user input are identified;
When the first user input is detected,
The control mode is determined to be the first mode,
When the second user input is detected,
The vehicle user interface device determines that the control mode is the second mode. According to claim 1,
the processor,
When a user input to the first display is detected and the control mode is the first mode,
The in-vehicle user interface device controls the second display to display at least a portion of images displayed on the first display on the second display. According to claim 1,
the processor,
Control the second display to display a graphic object corresponding to a user input on the first display;
When a user input to one area of the first display is detected,
The in-vehicle user interface device controls the second display to display a graphic object on one area of the second display corresponding to one area of the first display. According to claim 1,
the processor,
Control the second display to display a graphic object corresponding to a user input on the first display;
When it is detected that a user input to the first display moves from a first point to a second point,
Control the second display so that a graphic object moving from one point to another point is displayed;
A vehicle user interface device that controls the second display to change a location where a graphic object is displayed from one point to another point. According to claim 1,
the processor,
Controlling the second display to display a graphic object corresponding to a user input on the first display;
Based on the user input obtained through the input unit, different third user inputs and fourth user inputs are identified;
When the third user input is detected,
Displaying a graphic object on the second display to correspond to an object located at a short distance from the vehicle;
When the fourth user input is detected,
The vehicle user interface device controls the second display to display a graphic object on the second display, corresponding to an object located far from the vehicle. interface unit;
a first display; and
Obtaining user input information from a mobile terminal through the interface unit;
A processor for controlling the display based on user input information obtained from the mobile terminal according to a selected control mode;
The control mode is
A first mode for controlling a second display of the mobile terminal based on the user input;
A second mode for controlling the first display based on the user input;
the processor,
Based on the image information displayed on the first display, the control mode is determined;
In a state where information requiring user input is displayed on the first display, when the user input is sensed, it is determined that the control mode is the second mode. delete delete delete delete delete delete delete delete

Images