KR20150009120A - Mobile terminal and control method thereof - Google Patents

Abstract

Disclosed are a mobile terminal and a control method thereof, wherein the mobile terminal includes a wireless communications part, a user tracking part, and a control part. The wireless communications part connects to an image display device in a vehicle. The user tracking part detects the viewing direction of a driver based on driver images recorded from the point of a first display and a second display of the connected image display device in the vehicle. If the detected viewing direction of the driver is changed from the first display to the second display, the control part controls the wireless communications part in order to enable at least a part of a first image displayed on the first display to be displayed on one region of the second display. As the viewing direction of a driver is changed at various angles during the driving of a vehicle, the present invention displays an image corresponding to the previous viewing angle of the driver on a display corresponding to the current viewing angle of the driver, thereby enabling the driver to drive safely and also improving the convenience of the driver.

Description

[0001] MOBILE TERMINAL AND CONTROL METHOD THEREOF [0002]

The present invention relates to a mobile terminal and a control method thereof, and more particularly, to a mobile terminal connectable to a video display device in a vehicle and a control method thereof.

The terminal is movable And may be divided into a mobile / portable terminal and a stationary terminal depending on whether the mobile terminal is a mobile terminal or a mobile terminal. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

As the functions of the terminal are diversified, the terminal is implemented in the form of a multimedia device having a combination of functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, have. Further, in order to support and enhance the function of the terminal, it may be considered to improve the structural and software parts of the terminal.

Such a terminal can display at least one content by connecting to an in-vehicle video display device. The in-vehicle video display device is fixedly or detachably attached to the vehicle, and has a function of photographing the outside of the vehicle through a camera for the convenience of the driver, a function of guiding the vehicle to a destination, And a device for outputting sound.

Such an in-vehicle video display device is generally attached to a dash board located in front of a driver's seat, and may be referred to as a head unit (H / U), for example. In this way, as the in-vehicle video display device becomes electronic and the functions of the mobile terminal become diversified, a service interlocked between the head unit and the mobile terminal is further developed. For example, the navigation application or the service screen in the mobile terminal may be equally displayed on the display of the in-vehicle head unit, or the operation of the mobile terminal may be performed in the head unit.

However, for safe driving, the control of the application and the service screen as described above is limited through the head unit and the mobile terminal while the vehicle is running. Further, when there is no passenger other than the driver, it is also impossible to control an application or a service screen provided to the head unit using the mobile terminal through the passenger.

Accordingly, it is an object of the present invention to provide a mobile terminal and a control method thereof, which can control a screen provided to an in-vehicle head unit according to a direction of a driver's sight without disturbing operation.

To this end, a mobile terminal of a vehicle according to an embodiment of the present invention includes: a wireless communication unit connected to an in-vehicle video display device; A user tracking unit for sensing a direction of the driver's gaze based on an image of a driver photographed on the basis of a first display of the connected in-vehicle video display device and an image of a driver photographed on the basis of the second display; Wherein at least a portion of the first screen displayed on the first display is displayed in one area of the second display in response to the direction corresponding to the sensed driver's gaze being moved from the first display to the second display And a control unit for controlling the wireless communication unit.

In one embodiment, the first and second displays are connected to at least one camera detachably disposed in the vehicle, and the at least one camera displays images of a driver present within a predetermined range on the basis of each display, And the image of the image is acquired.

In one embodiment, the control unit extracts a partial screen of a predetermined range in which the driver's previous gaze is fixed on the first screen in real time, and controls the extracted partial screen to be displayed in one area of the second display .

In one embodiment, the control unit requests real-time transmission of a first screen displayed on the first display to the in-vehicle video display apparatus, and transmits the first screen received in response to the request to the in- And the wireless communication unit is controlled so that the wireless communication unit is displayed in real time in the area.

In one embodiment, the control unit controls the first screen to be overlaid on the second screen of the second display.

In one embodiment, the control unit controls the first screen to be displayed in an area distinguished from the second screen displayed on the second display, and the frame is displayed in the boundary area of the first screen. do.

In one embodiment, when the direction corresponding to the sight line of the sensed driver is moved from the second display to the first display, the control unit causes the first screen output to one area of the second display to disappear, And at least a part of the second screen displayed on the second display is outputted to one area of the first display.

In one embodiment, the first display is a head unit display (HUD) disposed in front of the vehicle, the second display is a reflector for monitoring a rear view of the vehicle, The second image is an external image of the vehicle obtained through a camera or the second image is an external image of the vehicle obtained through a lateral or rear camera.

In one embodiment, in response to the movement of the direction corresponding to the perceived driver's gaze from the first display to the display unit of the mobile terminal, the controller displays a predetermined range in which the driver's gaze is fixed And outputting at least a part of the first screen to be output to the first display.

In one embodiment, the control unit controls to output at least a part of the first screen to one area of the display unit in response to the driver's gaze being moved to the display unit, when the stop signal of the vehicle operation is input .

In one embodiment, the control unit determines that the stop signal is input if the signal included in the image photographed through the in-vehicle front camera is a signal indicating 'stop'.

In one embodiment, the control unit is configured to output an alarm notification to the display unit to notify the resuming state of the vehicle operation so that the sight line of the driver is out of the display unit when the resume signal of the vehicle operation is input .

In one embodiment, when the signal lamp included in the image photographed through the in-vehicle front camera changes from a "stop" to a "forward" signal, the control unit determines that the resume signal of the vehicle operation is input .

In one embodiment, the control unit controls the output range of the first screen to be changed to correspond to the first input when a first input in a predetermined direction is detected on the first screen output to the display unit .

In one embodiment, when a second input in a predetermined direction is sensed on a first screen output to the display unit, a screen displayed on another display of the connected in-vehicle video display device is displayed on the display unit And outputs the control signal.

In one embodiment, the control unit controls the first display to disappear from the display unit when the driver's line of sight is not detected on the display unit for a preset time, Is output to one area of the display unit.

In one embodiment, the controller controls the display unit to display a first screen corresponding to the indicator when the touch input is detected in the predetermined direction with respect to the indicator.

In one embodiment, the user tracking unit further includes an image of a driver photographed on the basis of the third display of the connected in-vehicle video display unit to sense a direction in which the driver's gaze is directed, In response to the movement of the display unit corresponding to the sight line of the driver from the display unit to the third display unit, the content screen and the first screen displayed on the display unit are output together with the third display .

In one embodiment, the portable terminal further includes a second image display device connected to the mobile terminal body, the second image display device being adapted to be worn by a driver on a part of the body, The first screen displayed on the first display and the second screen displayed on the second display are controlled to be displayed together on the screen of the second display device in response to the movement from the display to the second display .

Further, a method of controlling a mobile terminal according to an embodiment of the present invention includes: connecting to an in-vehicle video display device; Detecting a direction of the driver's gaze based on the image of the driver photographed on the basis of the first display of the in-vehicle video display device and the image of the driver photographed on the basis of the second display; Wherein at least a portion of the first screen displayed on the first display is displayed in real time in one area of the second display in response to the movement of the direction corresponding to the sensed driver's gaze from the first display to the second display, And outputting the control signal.

As described above, according to the control method of the mobile terminal and the mobile terminal according to the embodiment of the present invention, since the driver's line of sight changes variously while the vehicle is running, By displaying these together on the corresponding display, there is an effect of assisting the safe driving of the driver and at the same time improving the driver's convenience.

In addition, it is not necessary for the driver to perform a separate input during vehicle operation by controlling the display of the vehicle head unit in accordance with the natural gaze change of the driver during the operation of the connected mobile terminal.

1 is a diagram illustrating an example of an in-vehicle video display device connected to a mobile terminal and the mobile terminal according to an embodiment of the present invention.
2 is a diagram illustrating a configuration of a mobile terminal according to an embodiment of the present invention.
3 is a diagram illustrating an exemplary configuration of an in-vehicle video display device connected to a mobile terminal according to an embodiment of the present invention.
4 and 5 are flowcharts for explaining a control method of a mobile terminal according to an embodiment of the present invention.
6A to 6C are views illustrating a control method of a mobile terminal according to an embodiment of the present invention in which a screen displayed on a first display of an in-vehicle video display device is displayed on a second display in accordance with a direction of a driver's gaze FIG.
FIG. 7A and FIG. 7B are diagrams for explaining an example of a display screen of a mobile terminal in which only a partial screen in a range in which a line of sight of a driver is fixed is displayed on a second display in a screen displayed on a first display of the in- And a conceptual diagram for explaining the control method.
8 is a view for explaining an example in which the size and position of the previous screen displayed on the display of the in-vehicle video display device are changed according to the direction in which the driver's gaze is directed, according to an embodiment of the present invention.
9A to 9D are views for explaining an example in which a screen displayed on the in-vehicle video display device is displayed on a screen of a connected mobile terminal according to a direction in which the driver's gaze is directed during shutdown, according to an embodiment of the present invention These are the drawings.
10 is a conceptual diagram for explaining a control method of a mobile terminal for confirming a screen corresponding to a previous driving viewing angle by a driver or a passenger in a vehicle in a mobile terminal during a shutdown according to an embodiment of the present invention.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. It is also to be understood that the technical terms used herein are to be interpreted in a sense generally understood by a person skilled in the art to which the present invention belongs, Should not be construed to mean, or be interpreted in an excessively reduced sense. Further, when a technical term used herein is an erroneous technical term that does not accurately express the spirit of the present invention, it should be understood that technical terms that can be understood by a person skilled in the art are replaced. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Also, the singular forms "as used herein include plural referents unless the context clearly dictates otherwise. In the present application, the term "comprising" or "comprising" or the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps.

Further, the suffix "module" and "part" for components used in the present specification are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

Furthermore, terms including ordinals such as first, second, etc. used in this specification can be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or similar elements throughout the several views, and redundant description thereof will be omitted.

In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings.

1 is a diagram illustrating an example of an in-vehicle video display device connected to a mobile terminal and the mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 described herein may be a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a slate PC ), Tablet PC (tablet PC), ultrabook (ultrabook) can be included.

In addition, the in-vehicle image display apparatus 200 described herein may be configured such that a screen transmitted from the connected mobile terminal 100 is displayed on a reflector (e.g., a side mirror 201b 201c, and rear view mirror 201d), on the display 201e of the navigation device, on a dashboard of the vehicle, or on a head-up display (HUD) And can be displayed on the windshield (e.g., 201a) of the vehicle. That is, the display unit of the in-vehicle image display apparatus 200 may be provided at a position spaced apart from the main body of the image display apparatus 200, and may be formed as a plurality of as described below according to the operation angle of view.

1, a head mounted display (HMD) 300a or another mobile terminal 300b such as a notebook located in a vehicle may be mounted on a vehicle other than the in-vehicle video display device 200, And may be included as an image display device.

In this way, the in-vehicle video display device 200, the mobile terminal 100, and the other video display devices 300a and 300b in the vehicle are connected through the mobile terminal 100, (For example, an appearance of an object in a blind spot, etc.), internal state information of the vehicle (for example, a brake operation signal), and contents for information provision or entertainment effect , And can operate as a single vehicle system.

The in-vehicle video display devices 200, 300a, and 300b may be connected to a display located on the back of the mobile terminal 100 to communicate with each other. In addition, the in-vehicle video display devices 200, 300a and 300b may be connected to one mobile terminal or may be connected to a plurality of mobile terminals.

On the other hand, the in-vehicle video display devices 200, 300a, and 300b may transmit traffic information and various data (for example, programs, programs, and the like) through a wired / wireless communication network such as a LAN (Local Area Network) Executable files, etc.). In addition, the vehicle refers to all possible carriers that are implemented using mechanical or electronic devices for the purpose of moving people or objects such as a general passenger car, a bus, a train, a ship, an aircraft and the like.

Hereinafter, for the purpose of explanation, the mobile terminal 100 and the in-vehicle video display device 200 connected thereto will be described as an example. Accordingly, it is needless to say that the present invention can be similarly applied to other video display devices 300a and 300b in the vehicle.

1, the mobile terminal 100 is connected to the in-vehicle image display apparatus 200 in a wired or wireless manner, receives a screen displayed on the in-vehicle image display apparatus 200, It is possible to transmit to a device or other display. Also, the mobile terminal 100 may transmit the execution result (e.g., screen image or voice) of the application executed in the mobile terminal 100 to the video display device 200 via the interface You can send it.

Here, the mobile terminal 100 may be located in the vehicle or may be located outside the vehicle within a wireless connection range.

In addition, the in-vehicle video display device 200 is connected to at least one mobile terminal 100 by wire or wireless, and receives information on a screen or voice transmitted from the mobile terminal 100, or a destination or a traveling route of the vehicle The in-vehicle image display device 200, and the like. In this way, the state in which the mobile terminal 100 is connected to the video display device 200 located in the vehicle and is linked with the video display device 200 can be referred to as a 'connected car'.

In this way, when the 'connected car' state is established, the mobile terminal 100 displays a screen or voice displayed on the mobile terminal 100 or a screen displayed on the other video display devices 300a and 300b through the interface To the device (200). The interface for the screen transmission may be a means for supporting wireless communication such as a wireless fidelity (WiFi) transceiver, a Bluetooth transceiver, or a wired communication such as a USB (Universal Serial Bus) It can be a means of supporting.

The image display apparatus 200 may acquire an external image of the vehicle through a camera installed in the vehicle, for example, a front camera 277 or a forward camera (not shown) in a black box, 100). Then, the mobile terminal 100 may analyze the photographed image to grasp the face direction of the driver and / or the direction of the eyes of the driver. Thus, it is possible to detect the change of the driver's gaze while the vehicle is running. The mobile terminal 100 can perform at least one function corresponding to the change of the perceived driver's gaze. For example, if the driver's gaze is facing the front for a lane change and is directed to a side mirror on the left, the mobile terminal 100 may be moved It is possible to receive an image of the outside of the vehicle in real time and control it to be displayed on one side of the left side mirror.

In addition, the in-vehicle video display device 200 receives a specific command from the driver or a passenger of the vehicle through the interface, and can transmit the specific command to the connected mobile terminal 100. For example, when the driver touches or presses a touch pad or a key pad provided on the screen of the in-vehicle image display apparatus 200 to input a specific command, the image display apparatus 200 displays the position of the point where the input is applied And transmit information of the sensed position to the connected mobile terminal 100. [

Then, the mobile terminal 100 determines that a touch event has occurred at a point where the touch input is applied, and can perform an operation corresponding to the generated event. That is, the driver or the passenger of the vehicle may control the operation of the mobile terminal 100 using a touch pad, a keypad, or the like provided in the image display device 200. [

In addition, in the 'connected car' state, the passenger of the vehicle executes the navigation application (or dialing, phonebook, email, moving picture playback application, etc.) installed in the mobile terminal 100, In-vehicle image display device 200 by transmitting the execution image of the route guidance application to the in-vehicle image display device 200. [ By doing so, the driver can view the execution image of the route guidance application on the large screen of the image display device 200 instead of the small screen of the mobile terminal 100. Also, the driver can listen to the guidance voice through the speaker provided in the vehicle, instead of the speaker of the mobile terminal 100. [

On the other hand, when the driver wears a head mounted display (HMD) 300a shown in FIG. 1 and connects to the mobile terminal 100, the displays 201a, 201b, 201c, 201d, and 201e may be displayed in front of both eyes of the driver wearing the head mount display 300a.

When the passenger of the vehicle is connected to the mobile terminal 100 using another mobile terminal 300b shown in FIG. 1, the displays 201a, 201b, 201c, 201d, 201e of the in- The additional information of the screen and / or the vehicle to be displayed on the other mobile terminal 300b may be displayed on the display of the other mobile terminal 300b.

In addition, the in-vehicle video display device 200 and the mobile terminal 100 connected thereto can exchange information that is additionally required based on a separate protocol. For example, the in-vehicle image display apparatus 200 may provide the vehicle status information such as vehicle running information, speed information, fuel information, and the like to the mobile terminal 100. In addition, some applications installed in the mobile terminal 100 may use vehicle status information received from the video display device 200 using a separate protocol. Such applications may include a type of application (e.g., guidance, multimedia, game, etc.), a type of GUI (Graphic User Interface) , The status of the application (e.g., whether it is running in the foreground or background), and so on.

2, the mobile terminal 100 includes a wireless communication unit 110, a user tracking unit 120, a user input unit (not shown) A sensing unit 140, an output unit 150, a memory 160, an interface unit 170, a control unit 180, a power supply unit 190, and the like. However, not all of the components of the mobile terminal 100 shown in FIG. 1 are essential components, and the mobile terminal 100 may be implemented by more components than the components shown in FIG. 1, The mobile terminal 100 may also be implemented by components.

The wireless communication unit 110 may include at least one component for performing wireless communication between the mobile terminal 100 and the wireless communication system or wireless communication between the mobile terminal 100 and the network on which the mobile terminal 100 is located have. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114 and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may be a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving previously generated broadcast signals and / or broadcast related information and transmitting the broadcast signals and / or broadcast related information to the mobile terminal 100. The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, it may be received by the mobile communication module 112. The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcasting systems that provide broadcasting signals as well as the digital broadcasting system described above. The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data related to transmission and reception of a voice call signal, a video call signal, and / or a text / multimedia message.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 can be embedded in the mobile terminal 100 or externally. Here, the wireless Internet technologies include a wireless LAN (WLAN), a Wi-Fi, a wireless broadband (Wibro), a WiMAX, a high speed downlink packet access (HSDPA) Can be used.

The short-range communication module 114 means a module for short-range communication. Bluetooth, RFID (Radio Frequency Identification), IrDA (Infrared Data Association), UWB (Ultra Wideband), ZigBee, Wireless LAN (Bluetooth, 802.11n etc. protocol) .

The position information module 115 is a module for confirming or obtaining the position of the mobile terminal (the position of the vehicle can be confirmed when the mobile terminal is mounted on the vehicle). One example is the GPS (Global Position System) module. The GPS module receives position information from a plurality of satellites. Here, the location information may include coordinate information indicated by latitude and longitude. For example, the GPS module can accurately calculate the current position according to the triangle method by measuring the precise time and distance from three or more satellites and measuring three different distances. A method of obtaining distance and time information from three satellites and correcting the error by one satellite may be used. In particular, the GPS module can acquire latitude, longitude and altitude as well as three-dimensional velocity information and accurate time from the position information received from the satellite. As the position information module 115, a Wi-Fi Positioning System and / or a Hybrid Positioning System may be applied.

The user tracking unit 120 receives an audio signal and a video signal and tracks a user using the input signal. The user tracking unit 120 includes a camera 121, a microphone 122, and an image analysis module 123 . The camera 121 processes image frames such as still images and moving images obtained by the image sensor in the video communication mode or the photographing mode. The image frame processed by the camera 121 can be displayed on the display unit 151. [ The image frame may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ More than two cameras 121 may be provided depending on the use environment.

The microphone 122 processes the sound signal input from the outside in the communication mode, the recording mode, the voice selection mode, and the like as electrical voice data. The voice data processed by the microphone 122 in the communication mode can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 and output. The microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated when an external sound signal is input.

The image analysis module 123 analyzes the image transmitted through the interface to determine a direction in which the user's gaze is directed. Specifically, the user can recognize the direction of the user's face and / or the position of the user's eyes in the image transmitted from the external device, thereby detecting a change in the direction of the pupil. The analysis result is transmitted to the control unit 180, and can be controlled so as to execute a function corresponding to the direction in which the user's gaze is directed.

The user input unit 130 generates input data for the operation control of the mobile terminal by the driver. The user input unit 130 may include a key pad dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like. Particularly, when the touch pad has a mutual layer structure with the display unit 151, it can be called a touch screen.

The sensing unit 140 senses the position of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence of the driver, the orientation of the mobile terminal 100, 100, and generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. In addition, the sensing unit 140 may sense a sensing function related to whether the power supply unit 190 is powered on, whether the external unit of the interface unit 170 is connected, and the like.

The sensing unit 140 may include a proximity sensor 141, a pressure sensor 143, a motion sensor 145, and the like.

The proximity sensor 141 can detect an object approaching the mobile terminal 100 or the presence of an object existing in the vicinity of the mobile terminal 100 without mechanical contact. The proximity sensor 141 can detect a nearby object by using a change of an alternating magnetic field or a change of a static magnetic field, or by using a rate of change of capacitance. The proximity sensor 141 may include two or more sensors according to the configuration.

The pressure sensor 143 can detect whether the pressure is applied to the mobile terminal 100, the magnitude of the pressure, and the like. The pressure sensor 143 may be installed at a position where the pressure of the mobile terminal 100 is required according to the use environment. If the pressure sensor 143 is installed on the display unit 151, a touch input through the display unit 151 and a pressure larger than a touch input may be performed according to a signal output from the pressure sensor 143 To identify the applied pressure touch input. Also, according to the signal output from the pressure sensor 143, the magnitude of the pressure applied to the display unit 151 when the pressure touch is inputted can be known.

The motion sensor 145 detects the position or movement of the mobile terminal 100 using an acceleration sensor, a gyro sensor, or the like. The acceleration sensor, which can be used for the motion sensor 145, is a device that converts an acceleration change in one direction into an electric signal and is widely used along with the development of MEMS (micro-electromechanical systems) technology.

Here, the acceleration sensor measures the acceleration of a large value which is built in the airbag system of an automobile and is used for detecting a collision. Therefore, the acceleration sensor recognizes the minute movement of a human hand and uses the acceleration There are various kinds of things to measure. Acceleration sensors are usually constructed by mounting two or three axes in one package. Depending on the usage environment, only one axis of Z axis is required. Therefore, when the acceleration sensor in the X-axis direction or the Y-axis direction is used instead of the Z-axis direction for some reason, the acceleration sensor may be mounted on the main substrate by using a separate piece substrate. The gyro sensor is a sensor for measuring the angular velocity, and it can sense the direction of rotation with respect to the reference direction.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 includes a display unit 151, an audio output module 152, an alarm unit 153, A haptic module 154, and the like.

The display unit 151 displays information to be processed by the mobile terminal 100 and outputs the information. For example, when the mobile terminal 100 is in the call mode, a UI (User Interface) or GUI (Graphical User Interface) related to the call is displayed. If the mobile terminal 100 is in the video communication mode or the photographing mode, the captured image and / or the received image or UI and GUI are displayed.

The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display (3D display). In addition, there may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, the mobile terminal 100 may be provided with an external display unit (not shown) and an internal display unit (not shown) at the same time.

(Hereinafter, referred to as a 'touch screen') has a mutual layer structure with the display unit 151 and a sensor (hereinafter, referred to as 'touch sensor' It can also be used as an input device in addition to the device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch. If there is a touch input to the touch sensor, the corresponding signal (s) is sent to a touch controller (not shown). The touch controller processes the signal (s) and transmits the corresponding data to the control unit 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

The proximity sensor 141 may be disposed in an inner region of the mobile terminal 100 or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor 141 refers to a sensor that detects the presence of an object approaching a predetermined detection surface or an object existing in the vicinity thereof without mechanical contact by using electromagnetic force or infrared rays. The proximity sensor 141 has a longer lifetime and higher utilization than the contact-type sensor.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch" The act of actually touching the pointer on the screen is called "Contact Touch ". A position where the pointer is proximally touched on the touch screen means a position where the pointer corresponds to the touch screen vertically when the pointer is touched.

Further, the proximity sensor 141 detects a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, do. Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The sound output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, and a broadcast reception mode. The sound output module 152 outputs an acoustic signal related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include receiving a call signal, receiving a message, and inputting a key signal. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than an audio signal or a video signal. For example, it is possible to output a signal in a vibration mode. When the call signal is received or the message is received, the alarm unit 153 can vibrate the mobile terminal through the vibration means to inform it. Alternatively, when the key signal is input, the alarm unit 153 may vibrate the mobile terminal 100 through the vibration means with the feedback of the key signal input. The driver can recognize the occurrence of an event through the above-described vibration. Of course, a signal for notifying the occurrence of an event may also be output through the display unit 151 or the audio output module 152.

The haptic module 154 generates various tactile effects that the driver can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of air through the injection port or the suction port, a spit on the skin surface, contact with an electrode, And various effects of tactile effect such as effect of reproducing the cold sensation using the heat absorbing or heatable element can be generated.

The haptic module 154 can transmit the tactile effect through the direct contact, and the tactile effect can be felt by the driver through the muscular sense such as the finger or the arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the image display device. The haptic module 154 may be provided in a place where frequent contact with the driver is frequently performed in the vehicle. For example, a steering wheel, a shift gear lever, a seat, and the like.

The memory 160 may store a program for processing and controlling the controller 180 and temporarily store the input / output data (e.g., map data, a phone book, a message, a still image, For example.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory) (Random Access Memory) SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) A disk, and / or an optical disk. In addition, the mobile terminal 100 may operate a web storage for storing the memory 150 on the Internet.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, the interface unit 170 may include a headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, An audio input / output (I / O) port, a video I / O (input / output) port, and an earphone port. Here, the identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identity module (SIM) A Universal Subscriber Identity Module (USIM), and the like. In addition, the device having the identification module (hereinafter, referred to as 'identification device') can be manufactured in a smart card format. Accordingly, the identification module can be connected to the mobile terminal 100 through the port. The interface unit 170 receives data from an external device or receives power from the external device and transmits the received data to each component in the mobile terminal 100 or transmits data in the mobile terminal 100 to an external device.

When the mobile terminal 100 is connected to an external cradle, the interface unit 170 may be a path through which power from the cradle is supplied to the mobile terminal 100, or various commands A signal may be transmitted to the mobile terminal 100. The various command signals or the power source inputted from the cradle may be operated as a signal for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal 100. For example, the control unit 180 performs related control and processing for voice call, data communication, video call, and the like. In addition, the controller 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180.

The control unit 180 can perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 and supplies power required for operation of the respective components.

The functions of the components applied to the mobile terminal 100 may be implemented in a computer-readable recording medium using software, hardware, or a combination thereof. According to a hardware implementation, an application specific integrated circuit (ASIC), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs) controllers may be implemented using at least one of micro-controllers, microprocessors, and electrical units for performing functions. In some cases, such embodiments may be implemented by the controller 180 In accordance with a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be stored in a software application written in a suitable programming language The software code may be stored in the memory 160, May also be performed by the < / RTI >

The voice recognition module 182 recognizes a voice uttered by a user and performs a corresponding function according to the recognized voice signal.

A navigation session (or a search unit) 300 applied to the mobile terminal 100 displays a traveling route on the map data.

Meanwhile, the mobile terminal 100 according to the embodiment of the present invention can search for a destination or a transit point of the vehicle by using, for example, the multimedia module 181, And can transmit the searched destination or waypoint by communicating with the internal video display device 200.

In addition, the mobile terminal 100 may receive information related to the vehicle provided from the in-vehicle video display device 200 described below through the wireless communication unit 110. [ For example, the in-vehicle image display apparatus 200 may receive information on the vehicle such as the traveling path of the vehicle, the speed and the gear state of the vehicle, the on / off state of the emergency light of the vehicle, Off state of the horn, whether the occupant is seated through the detection sensor installed in the seat in the vehicle, on / off state of the horn, and the like.

In addition, the mobile terminal 100 may transmit the navigation route to the in-vehicle video display device together with the searched destination or stopover. Then, the in-vehicle video display device can display the driving route on a dashboard of a vehicle, a head-up display (HUD), or a front window of a vehicle using a projector. Here, the HUD (Head-Up Display) may be mounted on a dashboard of a vehicle, and the HUD and the projector may be commonly known methods, so a detailed description thereof will be omitted.

3 is a diagram illustrating an exemplary configuration of an in-vehicle video display device connected to a mobile terminal according to an embodiment of the present invention.

The in-vehicle image display apparatus 200 according to the embodiment of the present invention may include a plurality of displays. The plurality of displays may be spaced apart from the in-vehicle image display apparatus 200, for example, the main body of the vehicle head unit (HUD). For example, the main body of the in-vehicle image display apparatus is a vehicle head unit (HUD), and a plurality of displays included therein may include a windshield of a vehicle, a vehicle instrument panel, side mirrors of a vehicle, have. Further, in order to display a predetermined image on such a plurality of displays, the vehicle head unit (HUD) may include a projector. That is, the vehicle head unit HUD may display images stored in the memory or images received from the connected mobile terminal 100 on one of the plurality of displays using the provided projector.

3, the image display apparatus 200 includes a control unit (for example, a central processing unit (CPU)) 212 for controlling the image display apparatus 200 as a whole, a memory A key controller 211 for controlling various key signals and an LCD controller 214 for controlling a liquid crystal display (LCD).

The memory 213 stores map data (map data) for displaying the guidance information on a digital map. In addition, the memory 213 stores traffic information collection control algorithms for allowing the vehicle to input traffic information according to the current road conditions, and information for controlling the algorithms.

The main board 210 is a wireless communication device for performing wireless communication between the video display device 200 and the external mobile terminal 100 or wireless communication between the video display device 200 and a network where the video display device 200 is located. Module 206, a position information module 207 for receiving a GPS signal for guiding the position of the vehicle, tracing a traveling route from a departure place to a destination, or transmitting traffic information collected by a driver as a Global Positioning System (GPS) signal A CD deck 208 for reproducing signals recorded on a compact disc (CD), a gyro sensor 209, and the like. The wireless communication module 206 and the position information module 207 transmit / receive signals through the antennas 204 and 205 for the respective modules.

The wireless communication module 206 includes a mobile communication module 206a for transmitting and receiving a wireless signal to and from a base station, an external terminal, and a server on a mobile communication network, Wireless Internet access capable of accessing the wireless Internet in a manner such as LAN, Wi-Fi, Wireless Broadband (Wibro), World Interoperability for Microwave Access (WIMAX), HSDPA (High Speed Downlink Packet Access) Module 206b in a manner such as Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (UDA), Ultra Wideband (UWB), ZigBee, Wireless LAN A short range communication module 206c for performing communication, and the like.

The broadcast receiving module 222 is connected to the main board 210 and receives a broadcast signal through the antenna 223. The main board 210 includes a display unit (LCD) 201 under the control of the LCD control unit 214 through an interface board 203, a front board 202 under the control of the key control unit 211, A camera 227 for photographing the inside and / or the outside is connected. The display unit 201 displays various video signals and character signals. The front board 202 has buttons for inputting various key signals. The key signals corresponding to buttons selected by the driver are input to the main board 210 ). In addition, the display unit 201 may include the proximity sensor and the touch sensor (touch screen) of FIG.

The front board 202 may have a menu key for directly inputting traffic information, and the menu key may be controlled by the key controller 211.

The audio board 217 is connected to the main board 210 and processes various audio signals. The audio board 217 includes a microcomputer 219 for controlling the audio board 217, a tuner 218 for receiving a radio signal, a power source 216 for supplying power to the microcomputer 219, And a signal processing unit 215 for processing a signal.

The audio board 217 includes a radio antenna 220 for receiving radio signals and a tape deck 221 for reproducing audio tapes. The audio board 217 may further include a sound output unit (e.g., an amplifier) 226 for outputting a sound signal processed by the audio board 217.

The audio output unit (amplifier) 226 is connected to the vehicle interface 224. That is, the audio board 217 and the main board 210 are connected to the vehicle interface 224. The vehicle interface 224 may be connected to a handsfree 225a for inputting a voice signal, an air bag 225b for occupant safety, a speed sensor 225c for detecting the speed of the vehicle, and the like. The speed sensor 225c calculates the vehicle speed and provides the calculated vehicle speed information to the central processing unit 212. [

The navigation session (or search unit) 300 applied to the image display apparatus 200 generates route guidance information based on the map data and current position information of the vehicle, and notifies the driver of the generated route guidance information.

The display unit 201 senses the proximity touch in the display window through the proximity sensor. For example, the display unit 201 may detect the position of the proximity touch when a pointer (e.g., a finger or a stylus pen) is touched in proximity, and transmit position information corresponding to the detected position And outputs it to the control unit 212.

Also, as will be described in detail below, the display unit 201 may include a plurality of displays, and each display may display different images corresponding to the driving viewing angle of the vehicle. The display unit 201 may be connected to at least one camera 227. The camera 227 photographs an image of a user within a predetermined range on the basis of the display unit 201, To the board 210.

The camera 227 is generally built in the front of the vehicle or detachably mounted to capture an external image as the vehicle travels. In addition, the camera 227 may be replaced with a built-in black box in the vehicle. In addition, a plurality of such cameras 227 may be installed (for example, in a position where a side mirror or a rear view mirror is disposed) to photograph an external image of the corresponding vehicle, Or one camera can rotate to photograph an external image of the vehicle corresponding to various driving viewing angles according to the running of the vehicle.

On the other hand, the map data for searching for the travel route may be stored in the memory 213 or received from the external network through the wireless communication unit 110.

The voice recognition module 301 recognizes a voice uttered by a driver and performs a corresponding function according to the recognized voice signal.

A navigation session 300 applied to the image display apparatus 200 displays a traveling route on the map data and when the position of the mobile terminal 100 is within a predetermined distance from a blind spot included in the traveling route (For example, a car navigation device) mounted on a nearby vehicle and / or a mobile terminal carried by a neighboring pedestrian through a wireless communication (for example, a local area wireless communication network) Receives position information of the neighboring vehicle from a terminal mounted on the neighboring vehicle, and receives position information of the neighboring walker from a mobile terminal carried by the neighboring walker.

Meanwhile, the main board 210 may be connected to an interface unit (not shown), and the interface unit (not shown) may include an external device interface unit and a network interface unit. The external device interface unit serves to connect the external device to the video display device 200. To this end, the external device interface unit may include an A / V input / output unit (not shown) or a wireless communication unit (not shown). The external device interface unit can be connected to an external device such as a DVD (Digital Versatile Disk), a Blu ray, a game device, a camera, a camcorder, a computer (notebook) Further, the network interface unit can access a predetermined web page through the connected network or another network linked to the connected network. That is, the user can access a predetermined web page through the network and exchange data with the server.

In the embodiment of the present invention, the video display device 200 may be a head-up display (HUD), a projector, or a display device located on the back of the vehicle. At this time, if the video display device 200 is a display device located on the back side of the vehicle, it can communicate with a HUD (Head-Up Display) or a projector located in all seats of the vehicle through the wireless communication module 206 .

1) according to an exemplary embodiment of the present invention including at least one of the above-described configurations may be connected to the in-vehicle video display device 200, 1).

When the in-vehicle image display apparatus 200 and the mobile terminal 100 are connected, the user tracking unit 120 (FIG. 1) of the mobile terminal displays each of the displays 201a to 201e of the connected in-vehicle image display apparatus 200 The direction of the driver's gaze is detected based on the image of the driver photographed by the reference. To this end, each display of the in-vehicle video display device 200 is connected to at least one camera (preferably located in the driver's seat) for capturing an image of the driver.

When the direction of the driver's gaze is changed from the first display to the second display of the in-vehicle image display apparatus 200, the controller 180 (FIG. 1) A screen displayed on the first display of the display device 200, for example, at least a part of the screen corresponding to the front view of the vehicle is displayed on a second display on which the driver's gaze is shifted, for example, Can be displayed together.

For example, in one area of the side mirror display of the vehicle. That is, when the driver's line of sight is moved from the front view to the rear view displayed on the side mirror, the front image corresponding to the previous driving view is displayed on the display on which the sight line has been shifted.

At this time, the control unit 180 of the mobile terminal 100 can extract only partial screens of a predetermined range, in which a line of sight is fixed, on the screen before the driver's line of sight is shifted, And the driver's gaze can be transmitted to the display of the moved position in real time.

Here, the first display may be a display configured to photograph a vehicle's driving route ahead and display it, for example, a head-up display (HUD) or a windshield of a vehicle (hereinafter collectively referred to as' Vehicle head unit '). A screen corresponding to the front view of the vehicle is displayed on the first display, and the state information of the vehicle can be further displayed.

The second display is a display which is located on the driver's seat and / or the front passenger's seat of the vehicle and is arranged to photograph and / or display the rear of the vehicle, such as a side mirror connected to at least one camera mirror or a rear view mirror (hereinafter, may be referred to as a 'vehicle reflector'). And the second display may display a screen corresponding to the rear view of the vehicle.

For example, when the driver views the left / right side mirror according to the driving route while looking ahead, or moves his / her eyes to the rear view mirror, the 'connected car' state The mobile terminal 100 in the control unit 100 controls the driver to display the front view of the front view in one area of the side mirror to which the driver's gaze is shifted. Accordingly, even when the driver watches the front of the vehicle and is viewing the rear view angle during the vehicle operation, it is possible to know the situation in front of the driver in real time, so that when an object suddenly appears in the front view, the accident can be prevented.

As described above, since the sight line of the driver changes variously along the driving route during the vehicle operation, a screen corresponding to the previous driving viewing angle is displayed on the display corresponding to the current driving viewing angle, thereby assisting the safe driving of the driver, The convenience of the apparatus is improved.

Hereinafter, with reference to FIG. 4, a control method of a mobile terminal for displaying an image of a previous driving view angle on a display corresponding to a driver's line of sight change while the vehicle is traveling will be described in detail. In this regard, FIG. 4 is a flowchart illustrating a method of controlling a mobile terminal according to an embodiment of the present invention.

4, the mobile terminal 100 (FIG. 1) according to the embodiment of the present invention can be connected to the in-vehicle video display device 200 (FIG. 1) (S401).

For example, the mobile terminal 100 and the in-vehicle video display device 200 may be connected to each other by a wireless communication standard such as Bluetooth, a wireless internet standard such as Wi-Fi, a universal serial bus (USB) , And the like, thereby becoming a 'connected car' state as shown in FIG. To this end, the mobile terminal 100 may output a message or voice to inform that it can operate in the connected car state. It is needless to say that there may be a plurality of in-vehicle video display devices 200 connected (for example, 300a, 300b, Fig. 1).

In this way, when the 'connected car' state is established, the control unit 180 (FIG. 2) of the mobile terminal 100 displays the image of the driver photographed on the basis of the first display of the connected in- The direction of the driver's gaze can be sensed based on the image of the driver photographed on the basis of the display (S402).

To this end, each display of the in-vehicle image display apparatus 200 is connected to at least one camera 227, and the connected camera 227 can photograph an image of a driver within a predetermined range based on each display. On the other hand, in another example, the image of the driver can be photographed only on the basis of one display (e.g., a windshield display of the vehicle). According to this, an effect similar to that of taking an image of a driver based on each of the above-described displays based on the motion of the driver's head and the motion of the driver's pupil in the image of one driver taken can be obtained.

When the image of the driver is photographed on the basis of the display, the user tracking unit 120 (FIG. 2) of the mobile terminal displays the predetermined image on the basis of the first display of the vehicle image display apparatus 200 The image of the driver in the range and the image of the driver in the predetermined range based on the second display. When the image of the driver is received in this manner, the image analysis module 123 of the user tracking unit 120 can grasp a change in the direction of the driver's face and / or a change in the direction of the driver's gaze from the received image. The analysis of such an image may be performed by the control unit 180. [

In this way, each display can be connected to at least one camera 227 (FIG. 3) to know the change in the face direction of the driver and / or the change in the direction in which the line of sight is directed. That is, the image display apparatus 200 continuously acquires the image of the driver through the camera, for example, the front camera 227 (FIG. 1), and transfers it to the mobile terminal 100.

In this case, when an image of a driver in a predetermined range is taken on the basis of each display through a plurality of cameras, the image display device 200 causes the captured images of the driver to be transmitted to the mobile terminal 100 in chronological order Or each image to be transmitted may include the photographing time as additional information.

That is, the first and second displays of the in-vehicle image display apparatus 200 may be connected to at least one camera detachably disposed in the vehicle, and may acquire images of a driver within a predetermined range on the basis of the respective displays .

In addition, the first and second displays of the in-vehicle image display apparatus 200 may be connected to the front camera and the rear camera, respectively, to acquire an external image of the vehicle.

A method in which each display of the in-vehicle image display device 200 acquires an external image of the vehicle is as follows, for example.

First, the first display of the in-vehicle image display apparatus 200 may be a head unit display (HUD) or a windshield of a vehicle disposed in front of the vehicle. In addition, the second display of the in-vehicle image display device 200 may be a side mirror or a rear mirror of the vehicle.

When the first display is the windshield of the vehicle, the front view of the vehicle illuminated through the transparent glass can be displayed on the vehicle at the same time, and at the same time, the same image can be captured using the front camera to acquire the external image of the vehicle . When the first display is a vehicle head unit, the map information is mapped with the stored map information and the current vehicle position and travel route, and the corresponding image is displayed on the vehicle head unit. At the same time, It is possible to take an image of the forward view of the vehicle using the camera.

 On the other hand, the second display can display an image reflected from an object or an image positioned behind the vehicle, that is, the rear view of the vehicle. In addition, the rear view camera of the vehicle can be acquired by capturing and storing the rear view of the vehicle in real time.

A camera for photographing an image of a driver to detect a change in the driver's gaze and a camera for photographing an image corresponding to the driving view of the vehicle for acquiring an external image of the vehicle may be separately provided. That is, a first camera for capturing an image of the driver and a second camera for capturing an external image corresponding to the driving view of the vehicle (the second camera may include a front camera and a rear camera) .

As a result of sensing the direction of the driver's eyes, the controller 180 of the mobile terminal 100 determines whether the driver's gaze has moved from the first display to the second display of the vehicle image display apparatus 100 (S403).

If it is determined that the driver has moved his / her gaze, the controller 180 controls at least a part of the first screen displayed on the first display to be displayed in real time in one area of the second display on which the driver's gaze is shifted S404).

Here, the first screen is a screen displayed on the first display, for example, an image including all objects deployed in a front view of the vehicle, a traffic signal display, a moving object, and the like. In addition, in the first screen, status information of the current vehicle can be displayed in addition to the images shown in the front view of the vehicle.

6A, when the driver 900 looks at the front display 201a, for example, the vehicle head unit HUD or the windshield of the vehicle (A), and gazes at the left mirror display 201b for lane change, The mobile terminal 100 in the 'connected car' state moves the screen 601 displayed on the front display 201a to the left side In real time in one area of the mirror display 201b.

That is, as shown in FIG. 6B, the left mirror display 201b to which the driver's line of sight has been moved includes a screen 620 corresponding to the rear view of the vehicle displayed on the left mirror display 201b, And the screen 610 "corresponding to the forward view of the vehicle which is looking at the left mirror display 201b is displayed together with the display screen of the left mirror display 201b. .

At this time, the first screen displayed in one area of the second display on which the driver's gaze is shifted can be moved through the user input. That is, the controller 180 can determine whether a predetermined gesture input has been performed by receiving an image of a driver within a predetermined range based on the second display. If it is determined that there is a gesture input, The first screen can be moved and displayed.

6C, a screen 610 "corresponding to the front view of the vehicle is displayed in one area of the left mirror display 201b of the in-vehicle video display device 200, for example, The controller 180 of the mobile terminal 100 displays the screen 610 "in the direction corresponding to the flicking touch input, that is, in the direction of the second display 201b It can be moved to the upper right area and displayed.

In the case where the first screen and the second screen are simultaneously displayed on the second display and there is no change in the driver's gaze for a predetermined period of time or the driver does not intentionally look at the first screen (e.g., The control unit 180 determines that the screen 610 " corresponding to the front view of the vehicle, which was displayed in real time on the second display 201b as shown in (b) of Fig. 6, It can be prevented from being displayed.

Hereinafter, with reference to FIGS. 7A, 7B, and 8, a method of displaying the screen of the previous display together with the display of the driver whose sight line corresponding to the above-described step S404 has been shifted, according to the embodiment of the present invention More specifically, it will be described.

In this regard, FIGS. 7A and 7B are diagrams illustrating a partial screen in a range where the driver's line of sight is fixed on the screen displayed on the first display of the in-vehicle video display device 200 (FIG. 1) And a control method of the mobile terminal for displaying the second display on the second display. 8 is a diagram for explaining an example in which the size and position of the previous screen displayed on the display are changed according to the direction of the driver's gaze, according to an embodiment of the present invention.

4, the user tracking unit 120 of the mobile terminal 100 in the 'in-car video display' 200 and the 'connected car' state may be connected to the driver of a plurality of displays of the vehicle video display 200, The controller 180 may determine whether the driver's line of sight has moved from the first display to the second display of the in-vehicle image display apparatus 100. In this case,

If it is determined that the driver's line of sight has been moved to another display of the in-vehicle image display apparatus 200, the controller 180 determines at least part of the first screen displayed on the first display, 2 display in real time in one area of the display.

Here, the first display may be a head unit display (HUD) disposed in front of the vehicle, and the second display may be a side mirror or a rear mirror of the vehicle. Also, the first screen displayed on the first display is the same as the outside image of the vehicle obtained through the front camera, that is, the forward view of the vehicle, and the second screen displayed on the second display is the same It may be the same as the external image of the vehicle, that is, the image of the rear view of the vehicle.

The control unit 180 of the mobile terminal 100 may control the in-vehicle video display device 200 through the wireless communication unit 110 (FIG. 2) to display the first screen displayed on the first display on the second display. And requests to transmit the first screen in real time. The first display in response to the request transmits an image of the front view of the vehicle obtained through the connected camera to the mobile terminal 100. Then, the control unit 180 processes the received forward view image and transmits the processed image to the second display where the driver's eyes are moved. Then, the second display displays the image of the forward view of the vehicle transmitted from the mobile terminal 100 in one area of the screen in real time.

At this time, it is preferable to limit the image of the front view of the vehicle displayed on the second display to be displayed within a range that does not obscure the image displayed on the second display. That is, since the second screen of the second display displays a screen corresponding to the rearward view of the vehicle, the driver is allowed to display the forward view of the vehicle previously viewed in a range that does not cover the forward view.

Specifically, the control unit 180 may control the first screen to be displayed on the second display to be overlaid on the second screen of the second display. At this time, it is preferable that the first screen displayed overlaid is displayed as a screen smaller than the second screen. For example, the controller 180 may limit the size of the first screen, which is overlaid on the second screen, such that the size of the first screen does not exceed 1/4 of the total screen size.

In addition, the controller 180 may enlarge or reduce the size of the first screen to be displayed on the second display. For example, if the driver's line of sight is fixed only on the first screen for a predetermined period of time on the second display, the size of the first screen can be enlarged by a predetermined range. At this time, the maximum size of the first screen can be limited within a predetermined range as described above. Also, for example, when the driver's line of sight is fixed only on the second screen for a predetermined period of time on the second display, the size of the first screen may be reduced or eliminated.

For example, referring to FIG. 8 (a), a screen A displayed on a display previously viewed by a driver may be displayed floating or overlayed on a screen B displayed on a display which is currently viewed. At this time, the size of the screen A can be enlarged / reduced centered on the fixed point of the user's gaze. Further, according to the change of the driver's gaze in the current display or the occurrence of an event in accordance with the change of the external situation, the screen A can be displayed in a different area of the driver.

Here, the occurrence of an event in accordance with an external situation change means that, for example, when an object suddenly appears in a display area hidden by a screen A displayed by floating or overlaying or an object is detected in a dead zone, There may be cases where it should be displayed.

In the former case, the controller 180 may control the screen A so that it is moved to another area in the display, for example, an area where there is no object. In the latter case, the control unit 180 can control, for example, that the screen A disappears from the display and an alert notification is displayed. To this end, the mobile terminal 100 may receive the external image of the vehicle and the state information of the vehicle, for example, a signal corresponding to an object detected in a blind spot from the connected in-vehicle image display device 200 .

Also, the controller 180 may display the first screen to be displayed on the second display in an area distinguished from the second screen of the second display. To this end, the second screen displayed on the second display, for example, a screen corresponding to the rear view of the vehicle, may be replaced with an image shot through the camera. At this time, the photographed image is reduced and displayed in real time corresponding to the horizontal and vertical ratios of the area in which the second screen is to be displayed.

For example, referring to FIG. 8B, the second screen displayed on the entire display is reduced and displayed in the first area, for example, the left area. Further, the first screen displayed on the display that the driver has previously viewed is displayed in the second area, for example, the right area. In this case, the size of the second area may be limited to a certain range so as not to interfere with the operation of the driver, that is, to prevent the image of the rear view of the vehicle from being blocked.

In addition, the controller 180 controls the display unit 100 so that the first and second screens displayed simultaneously on the second display of the in-vehicle video display device 100 are visually distinguished from each other in the boundary region of the first screen displayed on the second display A predetermined frame may be displayed, or a highlighting effect may be displayed.

For example, referring to Fig. 7B, an image 720 of the front view of the vehicle is displayed on the windshield 201a of the vehicle and an image 711 "of the rear view is overlaid on one area of the windshield 201a When displayed, the frame 715 can be displayed in the border area of the image 711 " so that the driver can visually distinguish different images. At this time, the control unit 180 of the mobile terminal 100 may control the highlighting effect such as flickering of the frame 715 at predetermined time intervals.

In addition, if it is determined that the driver's line of sight has been moved to another display, the controller 180 may extract a partial screen of a predetermined range in which the driver's line of sight is fixed on the first screen in real time and display the partial screen on the second display . To this end, the control unit 180 receives the first screen in real time from the first display, determines the range in which the driver's line of sight is fixed in the received first screen, extracts the range from the first screen, To the second display.

For example, as shown in FIG. 7A, when the driver 900 is looking at the left side mirror 201b while moving his / her eyes to the front display 201a in order to change the vehicle during operation, the 'connected car' The control unit 180 of the mobile terminal 100 extracts the partial screen 711 within a predetermined range on the basis of the point A of the image 720 displayed on the left side mirror 201b, do. The partial screen 711 "thus extracted is displayed in real time on the windshield 201a of the vehicle whose driver's gaze is shifted as shown in FIG. 7B.

In this way, when a plurality of driving view angles (e.g., a forward view image and a rear view image) are simultaneously displayed on the display that the driver is currently viewing, The controller 180 can display an image corresponding to the previous operation angle of view on another display on which the operator's gaze is shifted.

Specifically, when the direction corresponding to the sight line of the sensed driver is moved from the second display to the first display, the controller 180 causes the first screen, which has been outputted to one area of the second display, to disappear, It is possible to control so that at least a part of the second screen displayed on the second display is outputted to one area of the first display facing the current driver's sight line.

On the other hand, if the driver's line of sight is moved to the third display (e.g., a rear view mirror), both the first screen and the second screen displayed on the second display are displayed together in one area of the third display .

At this time, if the driver's line of sight is not fixed to the third display for a predetermined time, the controller 180 controls the first display and the second display to disappear at the same time, or in order of display, 2 You can control the screen to disappear. In another example, the first screen and the second screen may disappear according to a voice command of the driver (e.g., a voice command of 'erase') or a gesture input.

Hereinafter, a control method of the mobile terminal for displaying an image of the previous driving view angle together with the display of the position corresponding to the driver's line of sight change during the vehicle operation and the vehicle operation interruption will be described in detail with reference to FIG. .

Referring to FIG. 5, a mobile terminal 100 (FIG. 1) according to an embodiment of the present invention connects to an in-vehicle video display device 200 (FIG. 1) (S501). When there are a plurality of in-vehicle video display devices 200, each video display device 200 can be connected to the terminal 100.

For example, the mobile terminal 100 and the in-vehicle video display device 200 may be connected to each other by a wireless communication standard such as Bluetooth, a wireless internet standard such as Wi-Fi, a universal serial bus (USB) , And the like, thereby becoming a 'connected car' state as shown in FIG. To this end, the mobile terminal 100 may output a message or voice to inform that it can operate in the connected car state.

When the connected car state is established as described above, a predetermined notification indicating that the mobile terminal 100 and / or the in-vehicle video display device 200 are connected may be output (S502).

For example, the notification may be output in the form of a beep such as a beep or a voice signal such as 'connected', or a predetermined message output to the display.

In this way, when the mobile terminal 100 and the in-vehicle video display device 200 are in the 'connected car' state, the control unit 180 (FIG. 2) of the mobile terminal 100 transmits the in- (For this purpose, connected to at least one camera) and the image of the driver photographed on the basis of the first display of the first display of the driver (S503).

The user tracking unit 120 (FIG. 2) of the mobile terminal displays the image of the driver in a predetermined range on the basis of the first display of the vehicle image display apparatus 200 through the wireless communication unit 110 (FIG. 2) It is possible to receive an image of a driver in a predetermined range on the basis of the display and to grasp a change in the direction of the driver's face and / or a change in the direction of the driver's gaze from the received image. For example, when the direction of the driver's eyeball in the first received image is directed to the first display and the direction of the driver's pupil in the second received image is directed to the second display, the user tracking unit 120 determines that the driver's gaze change Can be detected.

The control unit 180 of the mobile terminal 100 determines whether the driver's line of sight is on the first display (e.g., the side mirror) of the vehicle image display apparatus 100 It can be determined in real time whether the vehicle has moved to a display (e.g., a windshield or a vehicle head unit of the vehicle) (S504).

If it is determined that there is such a movement, the controller 180 notifies the in-vehicle video display device (not shown) to transmit the first screen displayed on the first display (for example, the rear view of the vehicle viewed through the side mirror) 200, and extracts a partial screen of a predetermined range, in which the driver's gaze is fixed, in the first screen received in real time according to the request (S505). In addition, a process of reducing the ratio of the received image to a size suitable for the display to be displayed may be accompanied.

When the image processing process is performed, the controller 180 controls the extracted partial screen to be displayed in real time in one area of the second display where the driver's gaze is shifted (S506).

The control unit 180 may determine whether the driver's gaze is moved from the in-vehicle image display apparatus 200 to the display unit 151 of the mobile terminal 100 when the stop signal of the vehicle operation is input S507).

The control unit 180 can determine that the stop signal of the vehicle operation is inputted if the signal included in the image photographed through the in-vehicle front camera (or the front camera of the black box) is a signal indicating 'stop'. In addition, the controller 180 may determine whether the operation state information of the vehicle, such as the brake operation signal of the vehicle, is received to determine whether the interruption signal is input or not, as a secondary determination condition.

To this end, the mobile terminal 100 may receive an image corresponding to the forward view of the vehicle from the video display device 200 of the vehicle. Also, the mobile terminal 100 can receive the operation state information of the vehicle from the video display device 200 of the vehicle or the vehicle system (not shown).

When the driver's attention is shifted to the display unit 151 of the mobile terminal 100, the control unit 180 determines whether the driver's gaze is fixed on the display unit 151 And controls to display the screen of the display that the driver has previously watched in real time in an area outside the predetermined range (S508).

9A, when the driver 900 is driving while observing the display 201a on which the image 901 corresponding to the forward view is displayed, when the vehicle is stopped, the mobile terminal 900 in the 'connected car' 100) detects the state of the vehicle received from the vehicle system (not shown), for example, based on the brake operation signal and the ' stop ' signal displayed in the signal lamp 90 included in the image 901 of the front view of the vehicle, It can be seen that the operation is stopped.

9B is displayed on the display unit 151 of the mobile terminal 100 when the driver 900 moves his / her eyes to the display unit 151 of the mobile terminal 100 to which the driver 900 is connected, The screen 901 "corresponding to the front view of the vehicle displayed on the display 201a previously viewed by the driver 900 together with the predetermined content is displayed at the same time. At this time, the screen corresponding to the front view of the vehicle 901 ") is displayed and displayed in an area outside the predetermined range of the point where the eyes of the driver 900 are fixed. For example, in FIG. 9B, a lower area of the display unit 151 is displayed.

In this state, the control unit 180 may sense a predetermined input to the screen 901 " corresponding to the front view of the vehicle displayed on the display unit 151 (S509) For example, a touch input in a predetermined direction applied to the screen 901 '', a gesture input of a predetermined shape, or an input signal or voice command corresponding to a user's gaze.

When the input to the screen 901 'is detected, the controller 180 controls the state of the screen and / or the output range of the screen related to the video display apparatus 100 of the vehicle displayed on the display unit 151 (S510).

Hereinafter, various embodiments in which a screen displayed on the video display device 100 of the vehicle is output to the display unit 151 will be described with reference to FIG. 9C.

In this regard, FIG. 9C illustrates a method of changing the screen of the video display apparatus 100 of the vehicle according to a predetermined input applied to the display unit 151 of the mobile terminal according to an embodiment of the present invention, Fig.

When the vehicle stop signal is received and the driver's gaze is moved from the display indicating the front view of the vehicle to the mobile terminal 100, A content screen 920 corresponding to a user operation is output to the first area, and a front view image 901 "of the vehicle previously looked at by the driver is displayed in real time in the second area distinguished from the first area.

In this state, the controller 180 senses a first input in a predetermined direction with respect to a first image displayed on the display unit 151, for example, a forward view of the vehicle, May be controlled so as to correspond to the first input.

For example, in FIG. 9C, a flicking touch input in a predetermined direction with respect to a screen 901 "a of the vehicle front view displayed on the display unit 151, for example, a flicking touch When the input is applied, the display unit 151 displays an output range of the screen 901 "a displayed on the display unit 151 in the full screen as shown in (b) That is, moved up / down. Accordingly, the driver can see the screen 901 "b of the front view changed to the desired viewing angle.

Also, although not shown, if a zoom-out touch input is detected on a screen 901 "a of the vehicle front view displayed on the display unit 151, a screen 901 The output range of "a) is smaller than before. On the other hand, if a zoom-in touch input is detected with respect to the screen 901 "a, the output range of the screen 901" a displayed on the display unit 151 is enlarged and displayed.

In addition, in a state in which the content screen and the image 901 "corresponding to the forward field of view previously looked at by the driver are distinguished and displayed in the first area and the second area of the display part 151, When a second input in a predetermined direction is sensed for one screen, that is, a forward view of the vehicle, the controller 180 controls the display unit 151 to display the screen of the display corresponding to the different viewing angle in real time .

For example, referring to (c) of FIG. 9C, when a driver applies a flicking touch input in a predetermined direction, for example, a flicking touch input in the left / right direction, to an image 901 " The backward visual field image 902 " may be displayed on the display unit 151 together with the image 901 "of the front visual field of the vehicle. That is, the images corresponding to the various driving visual angles of the vehicle are sequentially displayed The order in which the images are displayed corresponds to the order in which the in-vehicle displays are arranged (for example, from the driver's seat side to the assistant driver's seat side or vice versa) or corresponds to the order in which the driver's line of sight is shifted The video is erased).

Hereinafter, examples of how the screen is output will be described with reference to FIG. 9D, when vehicle operation is resumed and the driver's gaze is moved from the mobile terminal to another display of the in-vehicle video display device 200. FIG.

The mobile terminal 100 in the 'connected car' state can recognize that the vehicle operation is changed from the stopped state to the resumed state when the resume signal of the vehicle operation is input.

The control unit 180 determines that the resume signal of the vehicle operation is inputted when the signal lamp included in the image captured through the in-vehicle front camera is changed from 'stop' to 'progress'.

Specifically, the mobile terminal 100 may recognize that the vehicle has entered into the resume state based on a 'progress' signal (eg, a blue light or a left-turn signal) displayed in a traffic light included in the front view of the vehicle. In addition, the control unit 180 controls the operation of the vehicle by using the vehicle state information (e.g., the break signal is stopped) received from the in-vehicle video display device 100 or the vehicle system Can be determined.

When the vehicle operation is changed to the resumed state, the mobile terminal 100 outputs a warning notification to the display unit 151 requesting the driver to focus on the driving, so that the user's gaze is guided toward the front view of the vehicle can do.

For example, when a vehicle restart signal is inputted, the control unit 180 notifies the driver of a warning notification (e.g., 'Look forward') for informing the resumed state of the vehicle operation so that the driver's line of sight is deviated from the display unit ! ') To the display unit 151, as shown in FIG.

Accordingly, when the driver's gaze is moved from the mobile terminal 100 to the in-vehicle image display apparatus 100, the controller 180 displays the content screen displayed on the display unit 151 of the mobile terminal, For example, in a region of the vehicle head unit (HUD).

Specifically, the user tracking unit 120 of the connected mobile terminal 100, together with the first and second displays of the in-vehicle image display apparatus 200, displays the image of the driver photographed on the basis of the third display So that the direction of the driver's gaze can be detected.

The control unit 180 of the mobile terminal 100 displays the contents screen displayed on the display unit 151 and the front of the vehicle in response to the detected driver's gaze being moved from the display unit to the third display unit (The first screen) corresponding to the field of view is displayed on the third display on which the driver's gaze is shifted (even if the driver's gaze is shifted from the display unit 180 to the third display during vehicle operation) 1 screen can be displayed on the third display).

For example, referring to FIG. 9D, when it is detected that the vehicle operation is resumed and the driver 900's gaze is moved from the mobile terminal 100 to the head unit display 201e of the in-vehicle video display device 100, The head unit display 201e displays a map screen 930 for guiding the route along with a content screen 920 "displayed on the display unit 151 before the driver's gaze is moved and a screen 901 ") May be displayed together.

At this time, the content screen 920 "displayed on the head unit display 201e and the screen 901" of the front view of the vehicle are displayed when a predetermined time elapses or when a control command is input (e.g., , And disappear from the screen.

On the other hand, the external image of the vehicle corresponding to the change of the driver's gaze may be output to the other image display devices 300a and 300b in the vehicle.

For example, as described above with reference to FIG. 1, the mobile terminal 100 may include a head mounted display (HMD) 300a that is mounted on a part of the driver's body, for example, a head, Respectively. However, the present invention is not limited thereto and can be implemented in various wearable devices such as a spectacle type, a ring type, a necklace type, a clothing type, etc., which can be worn on the body of a driver.

When the first image display device (e.g., the vehicle head unit) 200 and the second image display device (e.g., the head mount display) 300a of the vehicle are in the 'connected car' state with the mobile terminal 100 The control unit 180 determines whether the driver's line of sight is displayed on the first display of the first image display apparatus 200 (e.g., the display showing the front sight of the vehicle) ). ≪ / RTI >

When the driver's line of sight is moved from the first display to the second display of the first video display device 200, the controller 180 displays the second screen displayed on the second display (e.g., (For example, the image of the front view of the vehicle) displayed on the first display is displayed on the display of the second image display device together with the image of the first image displayed on the first display.

Accordingly, the display on the plurality of displays of the video display device 200 of the vehicle does not need to be changed in accordance with the driver's visual line change.

Hereinafter, with reference to FIG. 10, a description will be given of a method in which the mobile terminal 100 connected to the vehicle or the passenger of the vehicle confirms the external image of the vehicle corresponding to the change of the driver's gaze during the stop of the vehicle operation.

When the vehicle stop signal is received and the driver's gaze is moved from the display indicating the front view of the vehicle to the mobile terminal 100, the display portion of the mobile terminal 100 A content screen 1020 corresponding to the user's operation is output to the first area of the vehicle, and the image 1001 "of the front view of the vehicle in which the driver has previously looked at the second area, which is distinguished from the first area, Is displayed.

In this state, if the driver's gaze is not detected on the display unit 151 for a preset time (for example, 5 seconds or more), the controller 151 displays the image 1001 " Screen ') disappears.

The control unit 180 may display an indicator for displaying the first screen in one area of the display unit 151. In addition, Here, the indicator is an icon for displaying a screen corresponding to the front view or the rear view of the vehicle, and may be displayed in the form of a status display bar, for example, at the upper or lower end of the display unit 151. In addition, the indicator may include a direction indicator for displaying an image to be displayed as shown in FIG. 10 (b).

In this way, when the indicator is displayed on the display unit 151, when the driver or passenger touches the indicator in the predetermined direction, the controller 180 displays the touch input corresponding to the first screen corresponding to the indicator So as to be displayed along the direction.

10 (b), when the user flickes and touches the indicator 1050 displayed in the upper area of the display unit 151 downward, the display unit 151 displays Images corresponding to various driving viewing angles of the vehicle, for example, a front view 1001 "and a rear view 1002 " are unfolded along the direction of the flicking touch input.

When the user applies a flicking touch input to the opened screen, the controller 180 displays the expanded front view screen 1001 " and the rearward view screen 1002 " along the direction of the touch input, And disappear from the memory 151.

As described above, according to the control method of the mobile terminal and the mobile terminal according to the embodiment of the present invention, when the driver's line of sight changes variously while the vehicle is running, By displaying these together on the corresponding display, there is an effect of assisting the safe driving of the driver and at the same time improving the driver's convenience. In addition, it is not necessary for the driver to perform a separate input during vehicle operation by controlling the display of the vehicle head unit in accordance with the natural gaze change of the driver during the operation of the connected mobile terminal.

Further, according to the embodiment disclosed herein, the above-described method can be implemented as a code that can be read by a processor on a medium on which the program is recorded. Examples of the medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of a carrier wave (e.g., transmission over the Internet) .

The above-described operation methods of the mobile terminal and the mobile terminal are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified so that all or some of the embodiments Or may be selectively combined.

Claims (20)

An in-vehicle video display device;
A user tracing unit for sensing a direction of the driver's gaze based on an image of a driver photographed on the basis of a first display of the connected in-vehicle video display apparatus and an image of a driver photographed on the basis of the second display; And
Wherein at least a portion of the first screen displayed on the first display is displayed in one area of the second display in response to the direction corresponding to the sensed driver's gaze being moved from the first display to the second display And a control unit for controlling the wireless communication unit. The method according to claim 1,
The first and second displays are connected to at least one camera detachably disposed in the vehicle, and the at least one camera acquires images of a driver and an external image of the vehicle existing within a predetermined range on the basis of each display Wherein the mobile terminal is a mobile terminal. The method according to claim 1,
Wherein,
Wherein the control unit extracts a partial screen of a predetermined range in which the driver's previous gaze is fixed on the first screen in real time and controls the extracted partial screen to be displayed in one area of the second display. The method according to claim 1,
Wherein,
The in-vehicle video display device requests real-time transmission of a first screen displayed on the first display and transmits the first screen to the first display in real time so that the first screen is displayed in real- To the mobile terminal. The method according to claim 1,
Wherein the control unit controls the first screen to be overlaid on the second screen of the second display. The method according to claim 1,
Wherein,
Wherein the control unit causes the first screen to be output in an area distinguished from the second screen displayed on the second display and controls the frame to be displayed in the boundary area of the first screen. The method according to claim 1,
Wherein,
When the direction corresponding to the sensed driver's gaze is moved from the second display to the first display, a first screen displayed on one area of the second display disappears and a second screen displayed on the second display disappears Wherein at least a part of the first display is output to one area of the first display. 8. The method according to any one of claims 5 to 7,
Wherein the first display is a head unit display (HUD) disposed in front of the vehicle, the second display is a reflector for monitoring a rear view of the vehicle,
Wherein the first screen is an external image of a vehicle obtained through a front camera and the second screen is an external image of a vehicle obtained through a side or rear camera. The method according to claim 1,
Wherein,
In response to the movement of the direction corresponding to the sensed driver's gaze from the first display to the display portion of the mobile terminal,
Wherein the control unit controls the display unit to output at least a part of the first screen displayed on the first display in an area outside the predetermined range in which the driver's line of sight is fixed. 10. The method of claim 9,
Wherein,
Wherein the controller controls to output at least a part of the first screen to one area of the display unit in response to the driver's gaze being moved to the display unit when the stop signal of the vehicle operation is inputted. 11. The method of claim 10,
Wherein,
Wherein the controller determines that the stop signal is input if the signal included in the image photographed through the in-vehicle front camera is a signal indicating 'stop'. 11. The method of claim 10,
Wherein,
And outputs a warning notice to the display unit to notify the resumed state of the vehicle operation so that the driver's line of sight is deviated from the display unit when the vehicle resume signal is inputted. 13. The method of claim 12,
Wherein,
Wherein when the signal included in the image photographed through the in-vehicle front camera is changed from a 'stop' to a signal indicating 'progress', it is determined that the resume signal of the vehicle operation is inputted. 10. The method of claim 9,
Wherein,
Wherein the controller controls the output range of the first screen to be changed to correspond to the first input when a first input in a predetermined direction is detected with respect to a first screen output to the display unit. 10. The method of claim 9,
Wherein,
And controls the display unit to display a screen displayed on another display of the connected in-vehicle video display device when a second input in a predetermined direction is detected on the first screen displayed on the display unit Mobile terminal. 10. The method of claim 9,
Wherein,
If the driver's line of sight is not detected on the display unit for a preset time,
Wherein the display unit controls the first screen to disappear and controls the indicator for displaying the first screen to be output to one area of the display unit. 17. The method of claim 16,
Wherein,
Wherein the controller controls the display unit such that a first screen corresponding to the indicator is displayed along the direction when a touch input in a predetermined direction is detected with respect to the indicator. 10. The method of claim 9,
The user tracking unit may further include an image of a driver photographed on the basis of the third display of the connected in-vehicle video display unit to sense a direction in which the driver's gaze is directed,
Wherein,
In response to the direction corresponding to the sensed driver's gaze being moved from the display unit to the third display unit, controlling the display unit to display the content screen and the first screen displayed on the display unit together with the third display Wherein the mobile terminal is a mobile terminal. The method according to claim 1,
Further comprising a second video display connected to the mobile terminal body and adapted to be worn by a driver on a part of the body,
Wherein,
In response to the movement of the direction corresponding to the perceived driver's gaze from the first display to the second display,
Wherein the controller controls the first display on the first display and the second display on the second display to be displayed together on the screen of the second display. An in-vehicle video display device;
Sensing a direction of the driver's gaze on the basis of the image of the driver photographed on the basis of the first display of the connected in-vehicle video display device and the image of the driver photographed on the basis of the second display; And
Wherein at least a portion of the first screen displayed on the first display is displayed in real time in one area of the second display in response to the movement of the direction corresponding to the sensed driver's gaze from the first display to the second display, And outputting the control signal to the mobile terminal.

Images