KR101602265B1 - Mobile terminal and method for controlling the same - Google Patents

Abstract

The present invention is to provide a mobile terminal and a vehicle control device, wherein the mobile terminal determines the carelessness of a driver and transmit a dangerous driving state to another terminal, thereby enabling other person to alert the driver to the carelessness. An embodiment disclosed in the present specification provides the mobile terminal worn on a specific body portion of a user to be used, which comprises: a body configured to be wearable on the specific body portion; a bio-information acquiring unit acquiring bio-information on the user from the specific body portion; a control unit detecting a dangerous driving state of the user based on the bio-information; a communications unit transmitting the dangerous driving state to another terminal; and a user input unit receiving a mode selection input for selecting targets to which the dangerous driving state is to be transmitted, wherein the control unit transmits the dangerous driving state to another terminal corresponding to the mode selection input.

Description

[0001] MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME [0002]

TECHNICAL FIELD [0002] The present invention relates to a mobile terminal and a control method thereof, and more particularly, to a mobile terminal for judging a carelessness of a driver of a vehicle and outputting the carelessness through another terminal and a control method thereof.

A terminal can be divided into a mobile / portable terminal and a stationary terminal depending on whether the terminal is movable. Again, the mobile terminal may be divided into a handheld terminal and a vehicle mount terminal according to whether the user can carry the mobile phone directly.

As the functions of the terminal are diversified, the terminal is implemented in the form of a multimedia device having a combination of functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, have. Further, in order to support and enhance the function of the terminal, it may be considered to improve the structural and software parts of the terminal.

Recently, various types of mobile terminals have been developed in consideration of user convenience. Among these, a wearable device includes all kinds of electronic devices that a user can wear on the body or clothes. Such a wearable device may be, for example, a smart watch, a wearable computer, a google glass, a Bluetooth headset, a smart wear, or the like.

Such a wearable device is implemented in the form of a multimedia device having a complex function according to various functions. For example, in addition to the basic functions provided as a clock in the case of a smart watch, the multimedia contents may be reproduced through a function of photographing a still image through a provided camera, recording a moving picture, or a display unit formed in the main body. Or various e-mail plug-ins or the like to check messages via the wireless Internet or access the social network.

By utilizing the various functions of such mobile terminal or wearable device (hereinafter referred to as "mobile terminal" in the present specification, but not intended to limit the devices to which the technology disclosed herein applies) Researches are being actively carried out in order to enable the researchers to provide such information.

In this regard, in recent years, when a driver wearing the mobile terminal has boarded the vehicle, he or she uses the various kinds of biometric information of the driver of the vehicle acquired by the mobile terminal to perform a carelessness (drowsiness, distraction, stress, and the like) of the human body.

The technical problem disclosed in this specification relates to a mobile terminal and a vehicle control apparatus which allow a driver to be aware of carelessness by allowing a mobile terminal to judge a carelessness of a driver and transmit a dangerous driving state to another terminal .

One embodiment disclosed herein is a mobile terminal used by being worn on a specific body part of a user, comprising: a body formed to be worn on the specific body part; a body part for acquiring biometric information on the user from the specific body part; A control unit for detecting a dangerous driving state for the user based on the biometric information, a communication unit for transmitting the dangerous driving state to another terminal, and a control unit for receiving a mode selection input to select an object to which the dangerous driving state is to be transmitted Wherein the control unit transmits the dangerous driving state to another terminal corresponding to the mode selection input.

Also, according to an embodiment, the other terminal may include at least one of the mobile terminal, the vehicle control device, and a vehicle output unit provided in the vehicle.

In addition, according to an embodiment, the dangerous driving state may be at least one of a drowsy driving state and a stressed state.

According to an embodiment, the communication unit may receive image information about the user from an image acquisition device provided in the vehicle, and the control unit may perform the dangerous driving based on at least one of the image information and the biometric information State can be detected.

According to an embodiment, the controller may determine whether the dangerous driving state is a drowsy driving state, a line-of-sight dispersion driving state, or a stressed state based on at least one of the image information and the biometric information.

Also, according to one embodiment, the controller may generate risk level information indicating a risk level of the dangerous driving state.

According to an embodiment, when the user input unit receives the automatic mode input, the control unit transmits information on the dangerous driving state to another terminal corresponding to the in-vehicle occupant when the dangerous driving state is detected , The other terminal having received the information on the dangerous driving state outputs the dangerous driving state through the output unit to the outside, and the other terminal corresponding to the occupant, And may be a terminal corresponding to the recognized occupant based on the image information of the occupant in the vehicle.

According to an embodiment of the present invention, when the user input unit receives the automatic mode input, the control unit transmits information on the dangerous driving state to another terminal recognized through the communication unit when the dangerous driving state is detected, Upon receiving the information on the dangerous driving state, the other terminal can output the dangerous driving state to the outside through the output unit.

Also, according to an exemplary embodiment, the other terminal receiving the information on the dangerous driving state may be at least one other terminal selected through the user input unit of the other terminal.

According to an embodiment, the control unit may transmit information on the dangerous driving state to another terminal within a predetermined distance from the other terminal.

According to an embodiment of the present invention, when the user input unit receives the automatic mode input, the control unit transmits information on the dangerous driving state to the vehicle control device having a plurality of display units when the dangerous driving state is detected, Wherein the vehicle control apparatus outputs the dangerous driving state via the display unit corresponding to the boarding position of the occupant, wherein the boarding position is a position in which the vehicle control apparatus displays image information on the occupant in the vehicle It can be judged based on the position of the occupant recognized as a basis.

According to an exemplary embodiment, when the user input unit receives the personal mode input, the control unit may output the dangerous driving state to the outside through the output unit included in the mobile terminal when the dangerous driving state is detected .

According to an embodiment of the present invention, when the user input unit receives the open mode input, the control unit transmits information on the dangerous driving state to another terminal capable of establishing a communication connection when the dangerous driving state is detected, The other terminal having received the information on the driving state can output the dangerous driving state to the outside through the output unit.

Also, according to an exemplary embodiment, the other terminal receiving the information on the dangerous driving state may be at least one other terminal selected through the user input unit among other terminals capable of establishing communication.

According to an embodiment, when the user input unit receives the custom mode input, the control unit transmits information on the dangerous driving state to the other established terminal upon detecting the dangerous driving state, The other terminal receiving the information on the dangerous operation state can output the dangerous driving state to the outside via the output unit.

Also, according to an exemplary embodiment, the other terminal having received the information on the dangerous driving state may be at least one terminal selected through the user input unit of the other established terminals.

According to an embodiment, when the user input unit receives a selection input for the other terminal, the output unit included in the mobile terminal determines whether or not the occupant of the passenger is wearing the terminal, At least one can be output together.

According to one embodiment, the output unit of the other terminal outputs an output according to at least one of the type of the dangerous driving state, the level of the danger level, whether the other terminal is located in the vehicle, Can be different.

According to an embodiment, when the other terminal is the mobile terminal, the output unit of the mobile terminal may be configured to detect a vibration frequency, an amplitude, a period, a voice or a sound of the main body Wherein the vehicle output unit or the vehicle control device outputs at least one of a type, a content, a frequency, an amplitude, and a cycle of the vehicle emergency output when the other terminal is a vehicle output unit or a vehicle control unit provided in the vehicle, An output LED of the LED, an output period of the LED, a vibration frequency of the vehicle vibration sheet, a vibration amplitude of the vehicle vibration sheet, a vibration period of the vehicle vibration sheet, Wherein at least one of a vibration frequency of the steerable vibration device, a vibration amplitude of the steering device, and a vibration cycle of the steering device is changed and output The.

According to another aspect of the present invention, there is provided a method of controlling a mobile terminal used in a specific body part of a user, the method comprising: receiving a mode selection input to select an object to which a dangerous driving state for the user is to be transmitted; Detecting the dangerous driving state based on the biometric information, and receiving information on the dangerous driving state from the other terminal corresponding to the mode selection input And transmitting the data to the mobile station.

According to the mobile terminal or the control method thereof according to the embodiment disclosed herein, it is possible to accurately determine the type of carelessness and the degree of carelessness of the driver.

In addition, according to the mobile terminal or the control method thereof according to the embodiment disclosed herein, it is possible to predict a sleepiness level of a user and to alert the user in advance through various means, thereby preventing an accident caused by drowsiness It can be prevented in advance.

In addition, according to the mobile terminal or the control method thereof according to the embodiment disclosed herein, it is determined based on at least one of the image information and the biometric information when the driver's carelessness is determined, The carelessness of the driver can be judged even if an abnormality occurs in any one of the acquiring device and the biometric information acquiring unit for acquiring the biometric information.

In addition, according to the mobile terminal or its control method according to the embodiment disclosed in this specification, by transmitting the dangerous driving state to the other terminal, it is possible for the driver to have the driver's awareness of carelessness, So as to be able to escape from.

1 is a block diagram illustrating a mobile terminal according to one embodiment disclosed herein.
2A is a conceptual diagram of a communication system in which a mobile terminal according to the present invention is operable.
2B is a reference diagram for explaining a method of positioning a mobile terminal according to the present invention.
3A shows a front view of a smart watch according to an embodiment disclosed in the present invention.
3B is a rear view of a smart watch according to an embodiment disclosed in the present invention.
4A is a conceptual diagram illustrating an embodiment of a wearable eyewear terminal according to an embodiment disclosed herein.
FIG. 4B is a view of the spectacles-shaped terminal of FIG.
5A is a perspective view showing an example of the mobile terminal disclosed in the present specification from the front.
5B is a rear perspective view of an example of the mobile terminal disclosed in this specification.
6A is a block diagram for explaining a vehicle control apparatus disclosed in this specification.
6B is an exemplary view showing an example of external devices connected to the vehicle control apparatus disclosed in the present invention.
7 is an exemplary view for explaining a display unit of the vehicle control device according to the embodiment disclosed herein.
8A is a block diagram showing the configuration of a vehicle control apparatus according to another embodiment disclosed herein.
8B is a block diagram showing the configuration of a vehicle control apparatus according to another embodiment disclosed herein.
FIG. 8C is an exemplary view illustrating a screen related to the navigation function according to the embodiment disclosed herein.
FIG. 9A is an exemplary view showing an image acquisition apparatus according to an embodiment disclosed in this specification installed in a vehicle. FIG.
FIG. 9B is another example of the image acquisition apparatus according to the embodiment disclosed herein installed in a vehicle. FIG.
10 is a diagram illustrating a drowsiness prediction line for a user according to an embodiment disclosed herein.
11A is an exemplary view of a user's gaze dispersion state according to an embodiment disclosed herein.
FIG. 11B is another exemplary diagram of a user's gaze dispersion state according to an embodiment disclosed herein.
12 is an exemplary diagram illustrating various output methods for the danger level information according to an embodiment disclosed herein.
13 is a block diagram illustrating a process for determining a drowsy driving state or a drowsiness level according to an embodiment disclosed herein.
FIG. 14 is an exemplary view of a screen for receiving a mode selection input by a mobile terminal according to an embodiment disclosed herein.
FIGS. 15A to 15D are diagrams illustrating an example in which a mobile terminal according to an embodiment disclosed herein transmits dangerous operation status information to a terminal corresponding to each mode.
16 is a diagram illustrating an example of a screen displayed in order to receive an input for selecting another terminal to transmit dangerous operation status information according to an embodiment of the present invention.
17 is a schematic configuration diagram of an electric vehicle having a battery charging device according to an embodiment disclosed herein.
18 is a flowchart illustrating steps of a method of controlling a mobile terminal according to an embodiment disclosed herein.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC , A tablet PC (tablet PC), an ultrabook, a wearable device, and the like. However, it will be understood by those skilled in the art that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless the configuration is applicable only to a mobile terminal.

The vehicle control apparatus described in the present specification may be applied to a telematics terminal, a navigation terminal, an AVN (Audio Video Navigation) terminal, a television, a 3D television, an A / V (Audio / A center, a call center, and the like.

In addition, the vehicle control apparatus described in this specification may also be in the form of a mobile terminal connected to the vehicle by wire or wirelessly. In this case, the vehicle control device may be a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP) A slate PC, a tablet PC, an ultrabook, a wearable device, and the like.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. . Also, the technical terms used herein should be interpreted as being generally understood by those skilled in the art to which the presently disclosed subject matter belongs, unless the context clearly dictates otherwise in this specification, Should not be construed in a broader sense, or interpreted in an oversimplified sense. In addition, when a technical term used in this specification is an erroneous technical term that does not accurately express the concept of the technology disclosed in this specification, it should be understood that technical terms which can be understood by a person skilled in the art are replaced. Also, the general terms used in the present specification should be interpreted in accordance with the predefined or prior context, and should not be construed as being excessively reduced in meaning.

Also, the singular forms "as used herein include plural referents unless the context clearly dictates otherwise. In this specification, the terms "comprising ", or" comprising ", etc. should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

Furthermore, terms including ordinals such as first, second, etc. used in this specification can be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals are used to designate identical or similar elements, and redundant description thereof will be omitted.

Further, in the description of the technology disclosed in this specification, a detailed description of related arts will be omitted if it is determined that the gist of the technology disclosed in this specification may be obscured. In addition, it should be noted that the attached drawings are only for easy understanding of the concept of the technology disclosed in the present specification, and should not be construed as limiting the idea of the technology by the attached drawings.

Mobile terminal

Hereinafter, the configuration and the mode of a mobile terminal according to an embodiment disclosed herein will be described in detail with reference to FIGS. 1 to 5B.

1 is a block diagram illustrating a mobile terminal 100 in accordance with one embodiment disclosed herein.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, A controller 170, a controller 180, a power supply 190, and the like. The components shown in FIG. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

For example, the broadcast receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S), a Media Forward Link Only (DVF-H) And a Digital Broadcasting System (ISDB-T) (Integrated Services Digital Broadcast-Terrestrial). Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The mobile communication module 112 is configured to implement a video communication mode and a voice communication mode. The video call mode refers to a state of talking while viewing a video of the other party, and the voice call mode refers to a state in which a call is made without viewing the other party's video. In order to implement the video communication mode and the voice communication mode, the mobile communication module 112 is configured to transmit and receive at least one of voice and image.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. Examples of the wireless Internet technology include a wireless LAN (WLAN), a wireless fidelity (WiFi) direct, a DLNA (Digital Living Network Alliance), a Wibro (Wireless broadband), a Wimax (World Interoperability for Microwave Access), HSDPA Can be used.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, NFC (Near Field Communication), etc. are used as short range communication technology .

The position information module 115 is a module for obtaining the position of the mobile terminal, and representative examples thereof include a Global Position System (GPS) module or a Wireless Fidelity (WiFi) module.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to an external device through the wireless communication unit 110. [ Further, the position information of the user and the like can be calculated from the image frame obtained by the camera 121. [ Two or more cameras 121 may be provided depending on the use environment.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data according to a control command for controlling the operation of the mobile terminal 100 applied by the user. The user input unit 130 may include a key pad, a dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like.

The sensing unit 140 may sense the position of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence of the user, the orientation of the mobile terminal, And generates a sensing signal (or sensing signal) for sensing the current state and controlling the operation of the mobile terminal 100. For example, the sensing unit 140 may detect whether the slide phone is opened or closed when the mobile terminal 100 is in the form of a slide phone. The sensing unit 140 may sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like.

The output unit 150 may include a display unit 151, an audio output module 153, an alarm unit 154, a haptic module 155, and the like in order to generate output related to visual, auditory, have.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the display unit 151 displays the photographed and / or received video or UI and GUI.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, a 3D display, and an e-ink display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see the object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another or may be disposed integrally with each other, or may be disposed on different surfaces.

Also, the display unit 151 may be configured as a stereoscopic display unit 152 for displaying a stereoscopic image.

Here, a stereoscopic image represents a 3-dimensional stereoscopic image, and a 3-dimensional stereoscopic image represents a progressive depth and reality in which objects are located on a monitor or a screen, It is an image that makes you feel the same as the space. 3D stereoscopic images are implemented using binocular disparity. The binocular parallax means a parallax caused by the position of two eyes away from each other. When two eyes see two different images and the images are transmitted to the brain through the retina and fused, the depth and real feeling of the stereoscopic image can be felt .

The stereoscopic display unit 152 may be applied to a three-dimensional display system such as a stereoscopic system (eyeglass system), an autostereoscopic system (non-eyeglass system), and a projection system (holographic system). The stereoscopic method, which is widely used in home television receivers, includes a Wheatstone stereoscopic method.

Examples of the autostereoscopic method include a parallax barrier method, a lenticular method, an integral imaging method, and a switchable lens method. The projection method includes a reflection type holographic method and a transmission type holographic method.

Generally, 3D stereoscopic images consist of left image (left eye image) and right image (right eye image). A top-down method of arranging a left image and a right image in one frame according to a method in which a left image and a right image are combined into a three-dimensional stereoscopic image, A checker board system in which pieces of a left image and a right image are arranged in a tile form, a left-to-right (right-side) Or an interlaced method in which rows are alternately arranged, and a time sequential (frame-by-frame) method in which right and left images are alternately displayed in time.

In addition, the 3D thumbnail image can generate a left image thumbnail and a right image thumbnail from the left image and right image of the original image frame, respectively, and combine them to generate one 3D thumbnail image. In general, a thumbnail means a reduced image or a reduced still image. The left image thumbnail and the right image thumbnail generated in this way are displayed on the screen with a difference of the left and right distance by the depth corresponding to the parallax between the left image and the right image, thereby exhibiting a stereoscopic spatial feeling.

The left and right images necessary for realizing the three-dimensional stereoscopic image can be displayed on the stereoscopic display unit 152 by a stereoscopic processing unit (not shown). The stereoscopic processing unit receives a 3D image and extracts a left image and a right image from the 3D image, or receives a 2D image and converts it into a left image and a right image.

On the other hand, when a display unit 151 and a sensor (hereinafter, referred to as 'touch sensor') that detects a touch operation form a mutual layer structure (hereinafter referred to as a 'touch screen' It can also be used as an input device in addition to the device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area where the touch object is touched on the touch sensor but also the pressure at the time of touch. Here, the touch object may be a finger, a touch pen, a stylus pen, a pointer, or the like as an object to which a touch is applied to the touch sensor.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

Referring to FIG. 1, a proximity sensor 141 may be disposed in an inner region of the mobile terminal or in the vicinity of the touch screen, which is surrounded by the touch screen. The proximity sensor 141 may be provided as an example of the sensing unit 140. The proximity sensor 141 refers to a sensor that detects the presence of an object approaching a predetermined detection surface or an object existing in the vicinity of the detection surface without mechanical contact using an electromagnetic force or an infrared ray. The proximity sensor 141 has a longer life than the contact type sensor and its utilization is also high.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. When the touch screen is electrostatic, it is configured to detect the proximity of the pointer by a change of the electric field along the proximity of an object having conductivity (hereinafter, referred to as a pointer). In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch & The act of actually touching the pointer on the screen is called "contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor 141 detects a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

In the case where the three-dimensional display unit 152 and the touch sensor have a mutual layer structure (hereinafter referred to as a 'three-dimensional touch screen') or a three-dimensional sensor that detects the touch operation and the stereoscopic display unit 152 are combined with each other The stereoscopic display unit 152 may also be used as a three-dimensional input device.

The sensing unit 140 may include a proximity sensor 141, a solid touch sensing unit 142, an ultrasonic sensing unit 143, a camera sensing unit 144, a biometric information acquiring unit 145, . ≪ / RTI >

The proximity sensor 141 measures the distance between the sensing surface (for example, a user's finger or a stylus pen) to which the touch is applied without mechanical contact using the force of the electromagnetic field or infrared rays. The terminal recognizes which part of the stereoscopic image has been touched using the distance. In particular, when the touch screen is of the electrostatic type, the proximity of the sensing object is detected by a change of the electric field according to the proximity of the sensing object, and the touch on the three-dimensional is recognized using the proximity.

The stereoscopic touch sensing unit 142 senses the strength or duration of a touch applied to the touch screen. For example, the three-dimensional touch sensing unit 142 senses a pressure to apply a touch, and when the pressing force is strong, recognizes the touch as a touch to an object located further away from the touch screen toward the inside of the terminal.

The ultrasonic sensing unit 143 is configured to recognize the position information of the sensing target using ultrasonic waves.

The ultrasound sensing unit 143 may include, for example, an optical sensor and a plurality of ultrasound sensors. The light sensor is configured to sense light, and the ultrasonic sensor is configured to sense ultrasonic waves. Since light is much faster than ultrasonic waves, the time it takes for light to reach the optical sensor is much faster than the time it takes for the ultrasonic waves to reach the ultrasonic sensor. Therefore, it is possible to calculate the position of the wave generating source using the time difference with the time when the ultrasonic wave reaches the reference signal.

The camera sensing unit 144 includes at least one of a camera 121, a photo sensor, and a laser sensor.

For example, the camera 121 and the laser sensor are combined with each other to sense a touch of a sensing target with respect to a three-dimensional stereoscopic image. When the distance information detected by the laser sensor is added to the two-dimensional image photographed by the camera, three-dimensional information can be obtained.

As another example, a photosensor may be stacked on a display element. The photosensor is configured to scan the movement of the object proximate to the touch screen. More specifically, the photosensor mounts photo diodes and TRs (Transistors) in a row / column and scans the contents loaded on the photosensor using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor performs coordinate calculation of the object to be sensed according to the amount of light change, thereby acquiring position information of the object to be sensed.

The biometric information acquiring unit 145 acquires biometric information through a specific body part of the user.

According to one embodiment, the biometric information acquiring unit 145 can measure a bio-signal generated by a physiological potential difference of the human body.

For example, the bio-information acquiring unit 145 may acquire the bio-information from the bio-information acquiring unit 145, such as Pulse Plethysmographic (ECG), Electro-CardioGram, Galvanic Skin Reflex, Electro-EncephaloGram, Electro- OculoGraphy sensors. Such sensors can measure vital signs for pulse blood flow, electrocardiogram, electrical skin reflection, brain waves, EMG, and eye movement.

The audio output module 153 can output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 153 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The sound output module 153 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 154 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events generated in the mobile terminal 100 include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 154 may output a signal for notifying the occurrence of an event by using a form other than the video signal or the audio signal, for example, vibration. The video signal or the audio signal may be output through the display unit 151 or the sound output module 153 so that the display unit 151 and the sound output module 153 may be classified as a part of the alarm unit 154 .

The haptic module 155 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 155 may be vibration. The intensity and pattern of the vibration generated by the haptic module 155 can be controlled by the user's selection or setting of the control unit. For example, the haptic module 155 may combine and output different vibrations or sequentially output the vibrations.

In addition to the vibration, the haptic module 155 may be configured to perform various functions such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or a suction force of the air through the injection port or the suction port, a touch on the skin surface, contact with an electrode, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 155 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sense of the finger or arm. At least two haptic modules 155 may be provided according to the configuration of the mobile terminal 100.

The memory 160 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 160 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read- And may include a storage medium of at least one type of disk and optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data to the external device. For example, a port for connecting a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, An audio I / O port, a video I / O port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM) A universal subscriber identity module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Therefore, the identification device can be connected to the terminal 100 through the interface unit 170. [

The interface unit 170 may be a path through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, And various command signals may be transmitted to the mobile terminal 100. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal 100. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

 In addition, the control unit 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

In addition, if the state of the mobile terminal meets a set condition, the controller 180 can execute a lock state for restricting input of a control command of the user to the applications. In addition, the controller 180 may control the lock screen displayed in the locked state based on a touch input sensed through the display unit 151 in the locked state.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays , Microprocessors, microprocessors, microprocessors, and other electronic units for carrying out other functions. In some cases, the embodiments described herein may be implemented by the controller 180 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein.

The software code may be implemented in a software application written in a suitable programming language. The software code is stored in the memory 160 and can be executed by the control unit 180. [

Next, a communication system that can be implemented through the mobile terminal 100 according to the present invention will be described.

2A and 2B are conceptual diagrams of a communication system in which the mobile terminal 100 according to the present invention can operate.

First, referring to FIG. 2A, the communication system may use different wireless interfaces and / or physical layers. For example, wireless interfaces that can be used by a communication system include Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA) , Universal Mobile Telecommunications Systems (UMTS) (especially Long Term Evolution (LTE)), Global System for Mobile Communications (GSM), and the like.

Hereinafter, for the sake of convenience of description, the description will be limited to CDMA. However, it is apparent that the present invention can be applied to all communication systems including CDMA wireless communication systems.

2A, a CDMA wireless communication system includes at least one terminal 100, at least one base station (BS) 270, at least one base station controller (BSCs) 275 , And a mobile switching center (MSC) The MSC 280 is configured to be coupled to a Public Switched Telephone Network (PSTN) 290 and BSCs 275. BSCs 275 may be coupled in pairs with BS 270 through a backhaul line. The backhaul line may be provided according to at least one of E1 / T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL or xDSL. Thus, a plurality of BSCs 275 may be included in the system shown in FIG. 2A.

Each of the plurality of BSs 270 may include at least one sector, and each sector may include an omnidirectional antenna or an antenna pointing to a particular direction of radials from the BS 270. In addition, each sector may include two or more antennas of various types. Each BS 270 may be configured to support a plurality of frequency assignments and each of the plurality of frequency assignments may have a specific spectrum (e.g., 1.25 MHz, 5 MHz, etc.).

The intersection of sector and frequency assignment may be referred to as a CDMA channel. The BS 270 may be referred to as a Base Station Transceiver Subsystem (BTSs). In this case, one BSC 275 and at least one BS 270 may be collectively referred to as a " base station ". The base station may also indicate a "cell site ". Alternatively, each of the plurality of sectors for a particular BS 270 may be referred to as a plurality of cell sites.

As shown in FIG. 2A, a broadcasting transmitter (BT) 295 transmits a broadcasting signal to terminals 100 operating in the system. The broadcast receiving module 111 shown in FIG. 1 is provided in the terminal 100 to receive a broadcast signal transmitted by the BT 295.

In addition, Figure 2A shows a satellite 300 of a Global Positioning System (GPS). The satellite 300 aids in locating the mobile terminal 100. Although FIG. 2A shows two satellites, useful location information may be obtained by two or more satellites. The location information module 115 shown in FIG. 1 cooperates with the satellite 300 shown in FIG. 2A to obtain desired location information. Here, the position of the mobile terminal 100 can be tracked using all the techniques capable of tracking the location as well as the GPS tracking technology. Also, at least one of the GPS satellites 300 may optionally or additionally be responsible for satellite DMB transmission.

Of the typical operation of a wireless communication system, the BS 270 receives a reverse link signal from the mobile terminal 100. At this time, the mobile terminal 100 is connecting a call, transmitting or receiving a message, or performing another communication operation. Each of the reverse link signals received by the particular base station 270 is processed by the particular base station 270. The data resulting from the processing is transmitted to the connected BSC 275. The BSC 275 provides call resource allocation and mobility management functions, including the organization of soft handoffs between the base stations 270. BSCs 275 also transmit the received data to MSC 280 and MSC 280 provides additional transport services for connection with PSTN 290. [ Similarly, the PSTN 290 is connected to the MSC 280, the MSC 280 is connected to the BSCs 275, and the BSCs 275 are connected to the BS 100 so that the forward link signal is transmitted to the mobile terminal 100 270 can be controlled.

Next, a method of acquiring location information of a mobile terminal using a WiFi (Wireless Fidelity) Positioning System (WPS) will be described with reference to FIG. 2B.

A WiFi Positioning System (WPS) 300 uses a WiFi module included in the mobile terminal 100 and a wireless access point 320 transmitting or receiving a wireless signal with the WiFi module, Is a technology for tracking the position of the terminal 100, and refers to a WLAN (Wireless Local Area Network) based positioning technology using WiFi.

The WiFi location tracking system 300 includes a Wi-Fi location server 310, a mobile terminal 100, a wireless AP 330 connected to the mobile terminal 100, and a database 330 in which certain wireless AP information is stored .

The WiFi location server 310 extracts information of the wireless AP 320 connected to the mobile terminal 100 based on the location information request message (or signal) of the mobile terminal 100. The information of the wireless AP 320 connected to the mobile terminal 100 may be transmitted to the Wi-Fi position server 310 through the mobile terminal 100 or may be transmitted from the wireless AP 320 to the Wi- Lt; / RTI >

SSID, RSSI, channel information, Privacy, Network Type, Signal Strength, and Noise Strength based on the location information request message of the mobile terminal 100, Lt; / RTI >

The WiFi location server 310 receives the information of the wireless AP 320 connected to the mobile terminal 100 and transmits the information included in the built-in database 330 and the received wireless AP 320, And extracts (or analyzes) the location information of the mobile terminal 100 by comparing the information.

2B, the wireless APs connected to the mobile terminal 100 are illustrated as first, second, and third wireless APs 320 as an example. However, the number of wireless APs connected to the mobile terminal 100 can be variously changed according to the wireless communication environment in which the mobile terminal 100 is located. The WiFi location tracking system 300 is capable of tracking the location of the mobile terminal 100 when the mobile terminal 100 is connected to at least one wireless AP.

Next, the database 330 may store various information of arbitrary wireless APs located at different locations, as will be described in more detail with respect to a database 330 in which arbitrary wireless AP information is stored.

The information of any wireless APs stored in the database 300 includes at least one of MAC address, SSID, RSSI, channel information, privacy, network type, radar coordinate of the wireless AP, (Available in GPS coordinates), address of the AP owner, telephone number, and the like.

Since the database 330 stores any wireless AP information and location information corresponding to the wireless AP, the WiFi location server 310 can search the mobile terminal 100 in the database 330, The location information of the mobile terminal 100 can be extracted by searching for wireless AP information corresponding to the information of the wireless AP 320 connected to the wireless AP 320 and extracting location information matched with the retrieved wireless AP information.

The extracted location information of the mobile terminal 100 is transmitted to the mobile terminal 100 through the Wi-Fi location server 310 so that the mobile terminal 100 can acquire the location information.

Hereinafter, an embodiment of various mobile terminals according to an embodiment disclosed herein will be described with reference to FIGS. 3A to 5B.

3A and 3B are front and rear views of a smart watch according to an embodiment of the present invention.

That is, FIGS. 3A and 3B show a case where the mobile terminal 100 has a smart watch type which is a watch type mobile terminal in the form of a wearable device.

Although the Smart Watch 200 disclosed in the present specification is illustrated as a rectangular shape, the present invention is not limited thereto, and may be applied to a smart watch having various polygonal shapes such as a circle or a triangle Of course, this is possible.

According to one embodiment, the smart watch 200 comprises a band 230 and a body 220. The outer case of the main body 220 may be divided into a front case 210 and a rear case 250. Various electronic parts are embedded in the space formed between the front case 210 and the rear case 250. [ At least one intermediate case may be additionally disposed between the front case 210 and the rear case 250. [ The cases may be formed by injection molding a synthetic resin or may be formed to have a metal material such as stainless steel (STS) or titanium (Ti) or the like.

3A, a display unit 151, a camera 121, a microphone 122, and the like may be disposed on the front case 210.

The display unit 151 occupies most of the main surface of the front case 210. A camera 121 and a microphone 122 may be disposed at both ends of the display unit 151.

On the other hand, the display unit 151 may display various types of time information. These pieces of information can be displayed in the form of letters, numbers, symbols, graphics, or icons.

At least one of the letters, numbers, symbols, graphics, or icons may be displayed in a predetermined arrangement for inputting such information, thereby being implemented as a keypad. Such a keypad may be called a so-called " soft key ".

The display unit 151 may operate as an entire area or may be divided into a plurality of areas and operated. In the latter case, the plurality of areas can be configured to operate in association with each other.

Also, a wired / wireless headset port (not shown) and a wired / wireless data port (not shown) may be disposed on one side of the main body 220 of the smart watch 200. The ports are configured as an example of the interface 170 (see FIG. 1).

Referring to FIG. 3B, the first sensor 145a may be disposed on the rear surface of the main body 220, that is, on the rear case 250. The first sensor 145a may be a sensor for detecting a user's skin condition or a user's biological signal. In addition, a second sensor 145b may be disposed in at least one region of the band 230 that is in contact with the skin of the rear part of the wrist of the actual user to detect movement of the user's muscles.

An antenna for receiving broadcast signals may be additionally disposed on the side surface of the main body 220. An antenna constituting a part of the broadcast receiving module 111 (see FIG. 1) may be installed to be able to be drawn out from the main body 220.

The main body 220 of the smart watch 200 may include an audio output module (not shown), an interface, and the like. A user input unit 340 and a connection port may be disposed on the side surfaces of the front case 210 and the rear case 250.

The user input unit 240 is operated to receive a command for controlling the operation of the smart position 200 and may include at least one operation unit 240a, 240b, 240c, and 240d. The operating units may also be referred to as manipulating portions and may be employed in any manner as long as the user is operating in a tactile manner with a tactile impression.

Contents inputted by a plurality of operation units can be set variously. For example, inputting a command such as start, end, scroll, and the like, adjusting the size of sound output from the sound output unit 152, or switching the touch recognition mode of the display unit 151 have.

The connection port receives data from an external device or supplies power to each component in the smart watch 200 or allows data in the smart watch 200 to be transmitted to an external device. The connection port may be configured as an example of the interface 170 (see FIG. 1).

A main body 220 of the smart watch 200 is provided with a power supply unit (not shown) for supplying power to the smart watch 200. The power supply unit may be built in the main body 220.

Figures 4A and 4B are conceptual diagrams illustrating one embodiment of a wearable eyewear terminal in accordance with one embodiment disclosed herein.

4A and 4B illustrate a case where the mobile terminal 100 is equipped with smart glasses, which is a glasses type mobile terminal in the form of a wearable device.

Referring to FIG. 4A, a wearable eyewear type terminal 200 'according to an embodiment disclosed herein includes a body 100', a display portion 151, and a control portion 180.

In addition, the wearable eyewear type terminal 200 'according to the embodiment disclosed herein may further include a user input unit, a voice recognition unit, and a motion detection unit.

The wearable eyewear type terminal 200 'according to one embodiment disclosed herein may be implemented with a head mounted display (HMD). As a concrete example, it can be implemented as smart glasses.

The body 100 'is formed to be mountable on the head. For example, it can be implemented as a frame and leg in smart glasses.

The display unit 151 may be disposed at a position corresponding to both eyes in association with the main body 100 '. Also, the display unit 151 is optically transparent and can output time information.

The time information refers to a virtual object created in the wearable eyewear type terminal 200 'or input from an external device. For example, the virtual object may refer to an application, a corresponding icon, a content, a UI of a call mode, and the like. Which may be generated by the control unit 180 or input from a mobile terminal such as a smart phone. At this time, since the display unit 151 has light transmittance, the user can see the external environment through the display unit 151. [

In addition, in one embodiment, the display unit 151 may output information about an arbitrary external object configuring the external environment while viewing the external environment. For example, the external object may be a business card, a person, or an external device capable of mutual communication.

As described above, the control unit 180 can control the wearable eyewear type terminal 200 '. Specifically, the control unit 180 may output the information of the external device sensed by the wireless communication unit 110 to the display unit 151.

For example, the controller 180 may identify the location of the detected external device. At this time, the controller 180 may determine whether the detected external device is located within the user's field of view, and may determine whether to output the sensed information of the external device according to the determination result.

The control unit 180 may be mounted on the body 100 'of the wearable eyewear-type terminal 200' or may be integrally formed with the body 100 '. In another embodiment, it may be spaced apart from the body 100 '.

The camera 121 may be disposed on the front face of at least one of the left and right eye display units 151. Alternatively, a space other than the wearer's view can be photographed by being disposed on one or both sides of the frame 100 '.

The user input unit 130 may be implemented as a separate touch panel on one or both sides of the frame 110 '. Alternatively, it may be implemented with a physical key. For example, a power ON / OFF switch may be implemented on one side of the frame 110 '.

In another embodiment, it may be implemented as a separate external device connected to the main body 100 '. Accordingly, the user can input a specific command to a separate external device. Alternatively, the display unit 151 may be implemented as a touch screen to directly receive a control command from a user.

In another embodiment, it may be implemented as a module that recognizes a voice command of a user. Thus, the user can input a specific command to the main body 100 'through voice.

Meanwhile, smart glasses are being marketed as an example of a wearable eyewear-type terminal. The smart glasses implemented as a wearable device can easily perform the functions performed in the existing mobile terminal.

The display unit 151 of the smart glasses may display an external environment and output time information displayed on the display unit 151 together. This allows the user to more easily grasp information about an arbitrary object constituting the external environment.

In addition, the smart glasses can perform wireless communication between smart glasses or an external device capable of communicating with smart glasses. At this time, information related to the external device may be output to the display unit 151. [

4B shows a wearable device on which an eye proximity display according to one embodiment disclosed herein may be mounted.

4B, the wearable device shown is a glass-type terminal 200 '', and the glass-type terminal 200 '' is configured to be worn on the head of a human body, and a frame part (case, . The frame portion may be formed of a flexible material to facilitate wearing. This figure illustrates that the frame portion includes a first frame 201 '' and a second frame 202 '' of different materials.

The frame portion is supported on the head portion, and a space for mounting various components is provided. As shown in Fig. 4B, electronic parts such as the control module 280 ", the sound output module 252 " and the like may be mounted on the frame part. Further, a display portion 251 " covering at least one of the left and right eyes may be detachably mounted on the frame portion.

The control module 280 " is configured to control various electronic components included in the glass type terminal 200 ". The control module 280 '' can be understood as a configuration corresponding to the control unit 180 (see FIG. 1) described above. This figure illustrates that the control module 280 " is installed in the frame portion on one side of the head. However, the location of the control module 280 " is not limited to this.

The display unit 251 '' may be implemented as a head mounted display (HMD). The HMD type refers to a display method that is mounted on a head and displays an image directly in front of the user's eyes. When the user wears the glass-type terminal 200, the display unit 251 may be arranged to correspond to at least one of the left and right eyes so that the user can directly provide an image in front of the user's eyes.

The display unit 251 " can project an image with the user's eyes. Further, the display unit 251 may be formed to be transmissive so that the user can view the projected image and the general view of the front (the range that the user views through the eyes) together.

As described above, the image output through the display unit 251 '' can be seen overlapping with the general view. The mobile terminal 100 can provide an Augmented Reality (AR) in which a virtual image is superimposed on a real image or a background and displayed as a single image using the characteristics of the display.

The camera 221 " is disposed adjacent to at least one of the left and right eyes, and is configured to capture a forward image. Since the camera 221 " is located adjacent to the eye, the camera 221 " can acquire a scene viewed by the user as an image.

Although the camera 221 '' is provided in the control module 280 '' in the figure, it is not limited thereto. The camera 221 '' may be provided in the frame part, or may be provided in plural to acquire stereoscopic images.

The glass-type terminal 200 " may have user inputs 223a ", 223b " that are manipulated to receive control commands. The user input units 223a '' and 223b '' can be employed in any manner as long as the user touches, pushes, etc. in a tactile manner. This figure illustrates that the user input portions 223a '' and 223b '' of the push and touch input type are provided in the frame portion and the control module 280 '', respectively.

In addition, a glass-type mobile terminal 200 '' may be provided with a microphone (not shown) for receiving sound and processing it as electrical voice data and an acoustic output module 252 '' for outputting sound. The sound output module 252 may be configured to transmit sound in a general sound output mode or a bone conduction mode. In a case where the acoustic output module 252 '' is implemented in a bone conduction manner, when the user wears the glass-type terminal 200 '', the acoustic output module 252 '' is brought into close contact with the head, .

5A and 5B, a structure of a mobile terminal or a mobile terminal in which components of the mobile terminal or the mobile terminal according to the embodiment disclosed in FIG. 1 are arranged will be described.

5A is a perspective view of a mobile terminal 100 according to an embodiment of the present invention.

The mobile terminal 100 disclosed in the present specification has a bar-shaped terminal body. However, the present invention is not limited thereto and can be applied to various structures such as a watch type, a clip type, a spectacle type, or a folder type, a flip type, a slide type, a swing type and a swivel type in which two or more bodies are movably coupled have.

The body includes a case (frame, housing, cover, etc.) which forms an appearance. In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. A battery cover 103 covering the battery 191 may be detachably attached to the rear case 102 have.

The cases may be formed by injection molding of synthetic resin, or may be formed of metal such as stainless steel (STS), aluminum (Al), titanium (Ti), or the like.

A first sound output module 153a, a first camera 121a, a first operation unit 131 and the like are disposed on the front surface of the terminal body, and a microphone 122, an interface unit 170 ), A second operation unit 132, and the like.

The display unit 151 is configured to display (output) information processed by the mobile terminal 100. The display unit 151 may include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) ), A flexible display, a 3D display, and an e-ink display.

The display unit 151 may include touch sensing means for receiving a control command by a touch method. When a touch is performed on any one of the display units 151, the touch sensing unit senses the touch and the content corresponding to the touched position is input. The content input by the touch method may be a letter or a number, an instruction in various modes, a menu item which can be designated, and the like.

The touch sensing means may be formed of a translucent material so that the visual information output from the display unit 151 can be seen, and a structure for enhancing the visibility of the touch screen in a bright place. 5A, the display unit 151 occupies most of the front surface of the front case 101.

A first sound output module 153a and a first camera 121a are disposed in an area adjacent to one end of both ends of the display unit 151 and a first operation unit 131 and a microphone 122 . The second operation unit 132 (see FIG. 5B), the interface unit 170, and the like may be disposed on the side of the terminal body.

The first sound output module 153a may be implemented in the form of a receiver for delivering a call sound to a user's ear or a loud speaker for outputting various alarm sounds or multimedia playback sounds.

The sound generated from the first sound output module 153a may be configured to be emitted along the assembly gap between the structures. In this case, the hole formed independently for the apparent acoustic output is hidden or hidden, so that the appearance of the mobile terminal 100 can be simplified. However, the present invention is not limited thereto, and a hole for emitting the sound may be formed in the window.

The first camera 121a processes an image frame such as a still image or moving image obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100 and may include first and second operation units 131 and 132. The first and second operation units 131 and 132 may be collectively referred to as a manipulating portion and may be any type of tactile manner such as touch, push, scroll, etc., .

In the figure, the first operation unit 131 is a touch key, but the present invention is not limited thereto. For example, the first operation unit 131 may be a mechanical key, or a combination of a touch key and a touch key.

The contents input by the first and / or second operation units 131 and 132 may be variously set. For example, the first operation unit 131 receives a command such as a menu, a home key, a cancellation, a search, and the like, and the second operation unit 132 adjusts the size of the sound output from the first sound output module 153a And a switch to the touch recognition mode of the display unit 151 and the like.

The microphone 122 is configured to receive a user's voice, other sounds, and the like. The microphone 122 may be provided at a plurality of locations to receive stereophonic sound.

The interface unit 170 is a path for allowing the mobile terminal 100 to exchange data with an external device. For example, the interface unit 170 may include a connection terminal for connecting to an earphone by wire or wireless, a port for short-range communication (for example, an IrDA Port, a Bluetooth Port, a Wireless LAN LAN Port) or the like, or power supply terminals for supplying power to the mobile terminal 100. [ The interface unit 170 may be implemented as a socket for receiving an external card such as a SIM (Subscriber Identification Module) or a UIM (User Identity Module) or a memory card for storing information.

5B is a rear perspective view of the mobile terminal 100 shown in FIG. 5A.

Referring to FIG. 5B, a second camera 121b may be additionally mounted on the rear surface of the terminal body, that is, the rear case 102. FIG. The second camera 121b may have a photographing direction substantially opposite to that of the first camera 121a (see FIG. 5A), and may be a camera having different pixels from the first camera 121a.

For example, the first camera 121a has low pixels so that it is easy to capture a face of a user in the case of a video call or the like and transmit the face to the other party. The second camera 121b photographs a general object and immediately transmits It is preferable to have a high pixel. The first and second cameras 121a and 121b may be installed in the terminal body so as to be able to rotate or pop-up.

A flash 123 and a mirror 124 are further disposed adjacent to the second camera 121b. The flash 123 shines light toward the subject when the subject is photographed by the second camera 121b. The mirror 124 enables the user to illuminate the user's own face or the like when the user intends to photograph the user himself / herself (self-photographing) using the second camera 121b.

And a second sound output module 153b may be further disposed on the rear surface of the terminal body. The second sound output module 153b may implement the stereo function together with the first sound output module 153a (see FIG. 5A) and may be used for the implementation of the speakerphone mode during a call.

An antenna (not shown) for receiving a broadcast signal may be additionally disposed on the side of the terminal body in addition to an antenna for a call or the like. An antenna constituting a part of the broadcast receiving module 111 (see FIG. 1) can be installed to be able to be drawn out from the terminal body.

The terminal body is provided with a power supply unit 190 (see FIG. 1) for supplying power to the mobile terminal 100. The power supply unit 190 may include a battery 191 built in the terminal body or detachable from the outside of the terminal body. This figure illustrates that the battery cover 103 is coupled to the rear case 102 so as to cover the battery 191 to restrict the detachment of the battery 191 and to protect the battery 191 from external impact and foreign matter .

It will be apparent to those skilled in the art that the techniques for mobile terminals or wearable devices disclosed herein may be embodied in other specific forms without departing from the spirit of the invention disclosed herein.

Vehicle control device

Hereinafter, the vehicle control device disclosed in this specification will be described in detail with reference to Figs. 6A to 8C.

6A is a block diagram for explaining a vehicle control apparatus related to the present invention.

6B is an exemplary view showing an example of external devices connected to the vehicle control device related to the present invention.

6A and 6B are block diagrams for explaining a vehicle control apparatus according to an embodiment disclosed herein, and FIG. 6B is a block diagram illustrating a vehicle control apparatus according to an embodiment of the present invention, Fig. 8 is an example showing an example of devices. Fig.

6A, a vehicle control apparatus 400 according to an embodiment of the present invention includes a control unit 410, a sensing unit 430 connected to the control unit 410, a vehicle driving unit 420, And a memory 440, and may further include an output unit 450. The vehicle control apparatus 400 may be formed in a body portion of a vehicle including an outer frame forming an outer appearance of the vehicle, a window, and an inner frame formed so that the user can ride. Here, the components shown in FIG. 6A are not essential in implementing the vehicle control apparatus 400 related to the present invention, so that the vehicle control apparatus 400 described in this specification may be configured to have more , Or may have fewer components.

The sensing unit 430 may include at least one sensor for sensing at least one of information in the vehicle control device 400, surrounding environment information surrounding the vehicle control device 400, and user information. For example, the sensing unit 430 may include a proximity sensor 432, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor A G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, A sensor such as an optical sensor, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation sensor, a heat sensor, a gas sensor, And the like). Meanwhile, the vehicle control apparatus 400 disclosed in this specification can combine and utilize the information sensed by at least two of the sensors.

Meanwhile, the sensing unit 430 may further include a near field communication module 431. The short-range communication module 431 is for short-range communication. The short-range communication module 431 may be a Bluetooth ™, a Radio Frequency Identification (RFID), an Infrared Data Association (IrDA), an Ultra Wideband (UWB) (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology. The short range communication module 431 may support wireless communication between the vehicle control device 400 and the external device 170 (see FIG. 6B) through the wireless area networks.

The external device may be the mobile terminal 100 described above. In particular, the external device may be in the form of a wearable device 200, 200 ', which is a form of the mobile terminal described above.

The vehicle drive unit 420 can release the locked state of the vehicle or switch the state of the vehicle to the locked state. Here, the locked state of the vehicle may be a state in which the function of the vehicle is limited in whole or in part, or the state in which the vehicle is not started or the door is not opened. On the contrary, the state in which the lock state is released means that at least one of a door of a driver's seat or a passenger seat of a vehicle, a door of a rear seat or a trunk is opened, or all the functions of the vehicle, Or a variety of functions such as a navigation function and a ventilation function can be made available.

In addition, the vehicle drive unit 420 may change various settings of the vehicle or perform functions automatically. For example, the vehicle drive unit 420 may control the components of the vehicle such as the driver's seat of the front seat, the degree of opening of the window of the passenger seat, the angle of the side mirror, And it is possible to adjust the seat height of at least one of the driver's seat or the rear seat and rear seats or the horizontal position of the seat (for example, the distance between the seats). In addition, the vehicle driver 120 may set the steering wheel handle setting, that is, the height of the steering wheel, the sensitivity of the steering wheel, etc., under the control of the controller 410. In addition, the vehicle drive unit 420 may cause the gear of the vehicle to operate in the automatic shift mode or the manual shift mode under the control of the control unit 410. In the case of a hybrid vehicle, Or a mode in which an electric motor is used may be preferentially selected.

The vehicle driving unit 420 may change not only the physical setting state of the vehicle but also the software setting according to the control of the controller 410. [ For example, the vehicle drive unit 420 may display a preset music list under the control of the control unit 410, or may automatically reproduce one of the music on the preset list. Or the vehicle driving unit 420 may automatically set a predetermined specific destination and automatically display the route to the specific destination through the provided navigation. Alternatively, the vehicle drive unit 420 may automatically set the vehicle-to-vehicle distance or the vehicle speed under the control of the control unit 410 at the time of cruise control of the vehicle.

To this end, the vehicle driving unit 420 may have at least one different lower driving unit, and each of the lower driving units may change a physical setting state or a software setting state of the vehicle. Hereinafter, the lower driving unit for changing the physical setting state of the vehicle will be referred to as a first driving unit 421, and the lower driving unit for changing the software setting state of the vehicle will be referred to as a second driving unit 422. [

The first driving unit 421 may include other components that can change an outer frame or an inner frame of the vehicle to change a physical setting of the vehicle. For example, the first driving unit 421 may further include a physical driving unit for adjusting the height of the seat or the angle of the backrest, and may include an elastic member such as a coil or a spring for increasing or decreasing the height of the handle, And a handle height adjusting unit including the handle height adjusting unit.

Meanwhile, the second driver 422 may be implemented by at least one application program or application. For example, the second drive unit 122 may be implemented as an application program for playing back any one of application programs for driving navigation or pre-stored media data (for example, MP3) These application programs or applications may be any of those for driving control of the vehicle.

The output unit 450 includes at least one of a display unit 451, an acoustic output unit 452, a haptrip module 453, and a light output unit 454 to generate an output related to visual, auditory, can do. The display unit 451 for displaying various image information may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. This touch screen may function as a user input portion 423 that provides an input interface between the vehicle control device 100 and a user and may provide an output interface between the vehicle control device 400 and the user.

Such a touch screen can be implemented in various parts of the vehicle. For example, the touch screen may be implemented on all or part of a windshield glass of a vehicle, and may be an outer surface (a surface exposed to the outside of the vehicle) or an inner surface (a surface facing the inside of the vehicle) Lt; / RTI > The touch screen may also be implemented on a window on the side of the driver's seat, a window on the side of the driver's seat, or an exterior or interior surface of the window in the rear seat of the vehicle. Alternatively, the touch screen may be implemented in a side mirror of a vehicle or a sunroof of a vehicle.

Such a touch screen may also be implemented not only in the gals portion of the vehicle's window or sunroof, but also in the outer or inner frame of the vehicle. For example, the touch screen may be implemented on a surface of an outer frame of a vehicle, such as an A-pillar, a B-pillar, a C-pillar, between a windshield glass and a window, or between a window and a window. Or at least a portion of the exterior surface of the vehicle door (e.g., near the handle portion of the vehicle door). In addition, the touch screen may be formed on a cover surface of a gear box inside the vehicle or on a cover portion of a console box. It should be understood that a plurality of touch screens may be formed on at least one or more different portions of the vehicle.

The memory 440 stores data supporting various functions of the vehicle control device 400. [ The memory 440 may store a plurality of application programs (application programs or applications) driven by the vehicle control device 400, data for operation of the vehicle control device 400, and commands. At least some of these applications may be downloaded from an external server via wireless communication. At least some of these application programs may also be present on the vehicle control device 400 from the time of departure for the basic functions of the vehicle control device 400 (e.g., start function, navigation function, vehicle lock and unlock function) . On the other hand, the application program is stored in the memory 440, installed on the vehicle control device 400, and can be driven by the control unit 410 to perform the operation (or function) of the vehicle control device.

According to one embodiment, the application program may be a navigation program that performs a navigation function.

In addition, the memory 440 may store information related to at least one or more users. Here, the user-related information may be information on the authentication information of the user and the setting status of various vehicles set by the user directly or appropriately set based on the user's biometric information. For example, it may be internal temperature or humidity of a vehicle set by a specific user, or setting information according to a user's driving habits. Or the travel route record of the user may be such information. The authentication information may be information on a password, a pattern preset by the user, or information based on the user's biometric information such as fingerprint or iris recognition information. Alternatively, the authentication information may be information related to a user's gesture.

According to one embodiment, the biometric information of the user may be obtained by the mobile terminal 100 (or the wearable device 200, 200 ').

In this case, the wearable device may further include a communication unit that communicates with a vehicle control device installed in the vehicle, and a biometric information obtaining unit that obtains biometric information on the user from the specific body part (e.g., the wrist) .

Here, the communication unit may transmit the acquired biometric information of the user to the vehicle control device 400, and the storage unit 440 may store the biometric information of the user.

The biometric information may be at least one of information on the heart rate, body fat, blood pressure, blood sugar, face shape, fingerprint, brain waves, and iris for the user.

The memory 440 may store a program for the operation of the controller 410 and may temporarily store input / output data (e.g., user authentication information and operation environment setting information). The memory 440 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 440 may be a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), a multimedia card micro type a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable memory (EEPROM) read-only memory (ROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk. The vehicle control apparatus 400 may be operated in association with a web storage that performs a storage function of the memory 440 on the Internet.

In addition to the operation related to the application program, the control unit 410 typically controls the overall operation of the vehicle control device 400. [ The control unit 410 may control the driving of the vehicle by processing signals, data, information or the like inputted or outputted through the above-mentioned components or driving an application program stored in the memory 440. In addition, the control unit 410 may control at least some of the components illustrated in FIG. 1A in order to drive an application program stored in the memory 440. Further, the control unit 410 may operate at least two or more of the components included in the vehicle control device 400 in combination with each other for driving the application program.

Meanwhile, the control unit 410 can receive authentication information from a user, and can determine whether the user is an authenticated user based on the input authentication information. Here, the authentication information may be a fingerprint of the user or recognition information of a predetermined pattern. Or the authentication information may be iris recognition information of a user or information related to a specific gesture of a user. For example, the control unit 410 may recognize information of a pattern of a plurality of knocks (for example, tabs or knocks) applied to a portion of a surface of the vehicle exterior or interior by a user, Or a touch screen area formed on a surface of a part of the interior, a window of a driver's seat or a passenger seat, and a windshield glass, or the like, as the authentication information. Alternatively, the control unit 410 may recognize a user's gesture made inside or outside the vehicle by using a photosensor or a camera provided in the sensing unit 430, or may recognize the iris information of the user.

Then, the control unit 410 can release the locked state of the vehicle only for the authenticated user. Therefore, the vehicle control apparatus 400 has: The user can open the door of the vehicle or open the trunk or the like without using the key. Alternatively, the vehicle may be started using the authentication information of the predetermined user. Or vice versa to switch the state of the vehicle to the locked state. That is, the control unit 410 may maintain the locked state of the vehicle until the authentication information of the authenticated user is input again based on the selection of the authenticated user. On the other hand, when the authentication information of the user is inputted through the outside of the vehicle while the vehicle is unlocked, the control unit 410 can switch the state of the vehicle to the locked state based on the user's authentication information. If the same authentication information is input once again while the vehicle is switched to the locked state, the vehicle may be switched to the unlocked state again.

The controller 410 may be configured to form a touch screen on a part of the vehicle to receive the authentication information of the user. The control unit 410 may receive the fingerprint information of the user, input the pattern information, or receive the preset password through the formed touch screen. For this purpose, the control unit 410 can perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively. Furthermore, the control unit 410 may control any one or a plurality of the above-described components in order to implement various embodiments described below on the vehicle control device 400 according to the present invention.

In addition, the control unit 410 may display various image information on the touch screen formed on a part of the vehicle. For example, the control unit 410 may display a fingerprint input area for user authentication or graphic objects for pattern input on the touch screen, and may display the authentication result of the user or information related to the currently authenticated user .

In the case of the authenticated user, the control unit 410 may change the setting state of the vehicle using the user-related information corresponding to the user. For example, the control unit 410 may control the first driving unit 421 to adjust the seat height or the angle of the backrest of the driver's seat or the like based on the information of the authenticated user, have. Alternatively, the control unit 410 may adjust the degree of opening of the driver's seat and the passenger's window, the angle of the side mirror, or the like, or the height of the steering wheel, based on the information corresponding to the authenticated user.

In addition, the control unit 410 may cause various operation modes of the vehicle to be changed according to the authenticated user. For example, the control unit 410 may change the operation mode of the power steering apparatus of the steering wheel to any one of the operation modes (for example, a normal mode or a sports mode) May be selected. Alternatively, the control unit 180 may set the gear shift mode in any one of the manual shift mode or the automatic shift mode that the user prefers in the shift mode of the gear.

In addition, the control unit 410 may perform software setting change as well as physical setting change. For example, when the authenticated user boarded the vehicle, the control unit 410 can automatically select a music list preferred by the user or a music list currently listened to by the authenticated user. Or the control unit 410 may automatically select a channel of the radio station that the currently authenticated user mainly hears.

Also, the control unit 410 may change various vehicle settings based on the time when the authenticated user boarded the vehicle. For example, the control unit 410 can analyze a destination that the user mainly travels at the time, based on the time at which the authenticated user boarded the vehicle, that is, the time at which the user was authenticated and the travel history of the authenticated user. In other words, when the user mainly has a driving habit toward "home" from 8:00 am to 9:00 am, the control unit 410, based on the user's driving record, When the user is boarding the vehicle, the destination may be automatically set to "home" and the route may be displayed on the display unit of the built-in navigation system.

As described above, the control unit 410 of the vehicle control device 400 related to the embodiment of the present invention allows the user to control the vehicle using the authentication information, thereby making it easier for the user to ride on the vehicle and use the vehicle. In addition, when the user is authenticated, the control unit 410 of the vehicle control device 400 according to the embodiment of the present invention adjusts various environment settings of the vehicle based on the authenticated user, As shown in FIG.

In addition, not only when the user is not boarding the vehicle, but also when the user is boarding the vehicle, the control unit 410 allows the physical setting change or the software setting change of the vehicle to be easily changed based on the user's selection It is possible. For example, the control unit 410 may control at least one of the physical setting changes (for example, a change of the physical setting) based on a plurality of knocks (Tabs) applied to the interior of the vehicle, Or software configuration changes may be made. For example, when the user applies a plurality of taps to the handle portion of the vehicle, the controller 410 recognizes the taps so that the height of the steering wheel is adjusted or the operation mode of the power steering device is changed to another mode.

On the other hand, the control unit 410 can change not only the plurality of tabs but also the physical or software setting state based on the gesture of the user. For example, the controller 410 can detect movement of a driver or passenger on the vehicle using a camera, a photo sensor, a laser sensor, or an infrared sensor provided in the vehicle. And a specific function may be performed based on the movement of the driver or the passenger, or the currently set state may be adjusted. For example, if an occupant seated at the same passenger takes a gesture down the hand from the top to the bottom of the passenger's window, the degree of opening of the passenger's seat can be adjusted based on the occupant's gesture. Alternatively, the control unit 410 may cause the predetermined specific music data to be reproduced when a specific gesture (e.g., a gesture for flicking or clapping a finger) is detected based on the gesture of the driver's seat or the passenger.

At least some of the components may operate in cooperation with one another to implement the operation, control, or control method of the vehicle control apparatus 400 according to the various embodiments described below. In addition, the operation, control, or control method of the vehicle control device 400 may be implemented on the vehicle control device 400 by driving at least one application program stored in the memory 440.

Hereinafter, the components listed above will be described in more detail with reference to FIG. 6A, before explaining various embodiments implemented through the vehicle control device 400 described above.

The sensing unit 430 senses at least one of information in the vehicle control device, surrounding environment information surrounding the vehicle control device, and user information, and generates a corresponding sensing signal. The control unit 410 may control the driving or operation of the vehicle control device 400 or may perform data processing, function, or operation related to the application program installed in the vehicle control device 400 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 430 will be described in more detail.

First, the proximity sensor 432 refers to a sensor that detects the presence of an object approaching a predetermined detection surface, or the presence of an object in the vicinity thereof, without mechanical contact using an electromagnetic force or an infrared ray. Such a proximity sensor 432 may be disposed in each area inside or outside the vehicle or near the touch screen.

Examples of the proximity sensor 432 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is electrostatic, the proximity sensor 432 can be configured to detect the proximity of the object with a change of the electric field along the proximity of the object having conductivity. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of explanation, the act of recognizing that the object is located on the touch screen in proximity to the touch screen without touching the object is referred to as "proximity touch" The act of actually touching an object on the screen is called a "contact touch. &Quot; The position at which the object is closely touched on the touch screen means a position where the object corresponds to the touch screen vertically when the object is touched. The proximity sensor 432 can detect proximity touches and proximity touch patterns (e.g., proximity touch distance, proximity touch direction, proximity touch rate, proximity touch time, proximity touch location, proximity touch movement state, have. Meanwhile, the control unit 410 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor 432 as described above, and further provides visual information corresponding to the processed data It can be output on the touch screen. Further, the control unit 410 may control the vehicle control apparatus 400 so that different operations or data (or information) are processed depending on whether the touch to the same point on the touch screen is a proximity touch or a contact touch have.

The touch sensor uses a touch (or touch input) applied to the touch screen (or the display unit 451) by using at least one of various touch methods such as a resistance film type, a capacitive type, an infrared type, an ultrasonic type, Detection.

For example, the touch sensor may be configured to convert a change in a pressure applied to a specific portion of the touch screen or a capacitance generated in a specific portion to an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, a capacitance at the time of touch, and the like where a touch object touching the touch screen is touched on the touch sensor. Here, the touch object may be a finger, a touch pen, a stylus pen, a pointer, or the like as an object to which a touch is applied to the touch sensor.

Thus, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the control unit 410. Thus, the control unit 180 can know which area of the display unit 451 is touched or the like. Here, the touch controller may be a separate component from the controller 410, and may be the controller 410 itself.

Meanwhile, the control unit 410 may perform different controls or perform the same control according to the type of the touch object, which touches the touch screen (or a touch key provided on the touch screen). Whether to perform different controls or perform the same control according to the type of the touch object may be determined according to the operating state of the current vehicle control device 400 or an application program being executed.

On the other hand, the touch sensors and the proximity sensors discussed above can be used independently or in combination to provide a short touch (touch), a long touch, a multi touch, a drag touch ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, and the like. Touch can be sensed.

The ultrasonic sensor can recognize the position information of the object to be sensed by using ultrasonic waves. Meanwhile, the controller 410 can calculate the position of the wave generating source through the information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the fact that the light is much faster than the ultrasonic wave, that is, the time when the light reaches the optical sensor is much faster than the time the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generating source can be calculated using the time difference with the time when the ultrasonic wave reaches the reference signal.

The sensing unit 430 may include at least one of a camera sensor (e.g., a CCD, a CMOS, etc.), a photosensor (or an image sensor), and a laser sensor to recognize a gesture of a user.

The camera 421 and the laser sensor are combined with each other to sense the touch of the object to be sensed. The photosensor can be laminated to the display element, which is adapted to scan the movement of the object to be detected proximate to the touch screen. More specifically, the photosensor mounts photo diodes and TRs (Transistors) in a row / column and scans the contents loaded on the photosensor using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor performs coordinate calculation of the object to be sensed according to the amount of change of light, and position information of the object to be sensed can be obtained through the calculation.

The display unit 451 may display various image information related to the user's authentication information input as described above. For example, the display unit 451 may display a graphic object displaying an area for receiving a fingerprint of a user or a graphic object for receiving pattern information in a part of a vehicle where a touch screen is formed. Alternatively, the display unit 451 may display information related to the user authentication result and the currently authenticated user when the user authentication is completed. Such image information may be displayed on at least a part of the windshield glass of the vehicle or a window of a driver's seat or a passenger's seat or the like so that the window of the vehicle or the windshield glass of the vehicle equipped with the vehicle control apparatus 400 according to the embodiment of the present invention At least a portion of the electrodes may be designed to sense the user's touch input.

Also, the display portion 451 may be formed on the inner surface as well as the outer surface of the windshield glass and the window. The display unit 451 formed on the inner surface may display (output) information to be processed by the vehicle control device 400. For example, the screen information displayed on the display unit 451 formed on the inner surface may include execution screen information of an application program driven by the vehicle control apparatus 400, UI (User Interface) based on the execution screen information, , And GUI (Graphic User Interface) information.

Also, the display unit 451 may be included in the sensing unit 130. In this case, the display unit 451 may display the detection result of the sensing unit 430, the matching result of the user authentication information, or a part of the unique authentication information of the user (e.g., the name of the user) .

The audio output unit 452 can output the audio data stored in the memory 440. The sound output unit 152 also outputs sound signals related to functions (e.g., user authentication sound, user authentication sound, etc.) performed in the vehicle control device 400. [ The sound output unit 452 may include a speaker, a buzzer, and the like.

Meanwhile, the output unit 450 of the vehicle control apparatus 400 according to the embodiment disclosed herein may include a haptic module 453. [ The haptic module 453 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 453 may be vibration. The controller 410 can output tactile information using the haptic module 453 when a user's touch input is detected on a touch screen implemented in an outer frame of the vehicle, an inner frame, a glass window, or the like. Accordingly, the user can check whether or not the authentication information inputted by himself / herself is correctly inputted by using the tactile information.

The intensity and pattern of the vibration generated in the haptic module 453 can be controlled by the user's selection or the setting of the control unit. For example, the haptic module 453 may combine and output different vibrations or sequentially output the vibrations.

In addition to vibration, the haptic module 453 can be used for a variety of applications such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of air through the injection port or the suction port, a touch on the skin surface, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 453 can transmit the tactile effect through the direct contact, and the tactile effect can be realized by the user through the muscular sense of the finger or arm. More than two haptic modules 453 may be provided in accordance with the configuration of the vehicle control device 400.

In the following, various embodiments may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

Meanwhile, the sensing unit 430 of the vehicle control apparatus 400 according to the embodiment of the present invention includes a body mounted on the body unit to contact a part of the user's body, Or a tab sensing unit 433 for sensing a tab gesture. Here, the tabs sensed by the tap sensing unit 433 of the vehicle control device 400 are means for inputting authentication information of the user, or may be configured to control various functions of the vehicle control device 400 when the user boarded the vehicle As shown in FIG. These tabs can be understood as an operation of lightly tapping the body or the object of the vehicle control device 400 with a tap object such as a finger or an operation of lightly touching the tap object to the body or the object of the vehicle control device 400. [ Here, the body of the sensing unit 430 may be formed in a body portion of the vehicle including an outer frame of the vehicle, an inner frame, and a window or a windshield glass.

The tap object to which the tap is applied may be an object that can apply external force to the body of the vehicle control device 400 or an object. For example, the tap object may include a finger, a stylus pen, a pen, , A fist, and the like. The tap object is not limited to an object to which a touch input can be applied to the vehicle control device 400 according to the present invention. If the object is an object that can apply an external force to the body or the object of the vehicle control device 400, The kind is irrelevant.

On the other hand, the object to which the tap gesture is applied may include at least one of a body of the vehicle control device 400 and an object placed on the vehicle control device 400.

The tap or tap gesture may be sensed by at least one of an acceleration sensor and a touch sensor included in the tap sensing unit 433. [ Here, the acceleration sensor is a sensor capable of measuring dynamic forces such as acceleration, vibration, and impact of the main body of the vehicle control device 400.

That is, the acceleration sensor can sense vibration (or movement) of the main body of the vehicle control device 400 generated by the tap gesture, and detect whether the tap is applied to the object. Therefore, the acceleration sensor is positioned close to the main body of the vehicle controller 400 so as to detect a tap with respect to the main body of the vehicle controller 400 or to detect whether movement or vibration occurs in the main body of the vehicle controller 400 It is possible to detect that an object is tapped.

As long as the acceleration sensor can detect movement or vibration in the main body of the vehicle control device 400, the acceleration sensor can be applied not only to the case where the tap is applied to the main body of the vehicle control device 400, It is possible to detect the tab.

In the vehicle control device 100 according to the present invention, only one of the acceleration sensor and the touch sensor, the acceleration sensor, and the touch sensor are sequentially used for sensing the tab of the main body of the vehicle control device 400, Or an acceleration sensor and a touch sensor can be used at the same time. On the other hand, in order to sense a tap, a mode using only an acceleration sensor may be named as a first mode, a mode using a touch sensor may be referred to as a second mode, and both the acceleration sensor and the touch sensor may be used Mode) may be referred to as a third mode or a hybrid mode.

On the other hand, when the tab is sensed through the touch sensor, it is possible to more accurately grasp the position where the tab is sensed.

In the vehicle control apparatus 400 according to an embodiment of the present invention, in order to sense the tap through the acceleration sensor or the touch sensor, the display unit 451 of the vehicle control device 400 may be operated with a minimum current or power Can be operated in a specific mode consumed. This particular mode may be named "Doze mode ".

For example, in the doze mode, in a touch screen structure in which a touch sensor and a display unit 451 form a mutual layer structure, a light emitting element for outputting a screen in the display unit 451 is turned off, The sensor may be in the on-state state. Alternatively, the doze mode may be a mode in which the display unit 451 is turned off, and the acceleration sensor is kept in an on-state. Alternatively, the dose mode may be a mode in which the display unit 451 is turned off, and the touch sensor and the acceleration sensor are all kept on.

Therefore, in the doze mode state, that is, in a state where the display unit 451 is turned off (the display unit 451 is in an inactive state), the user can perform at least one point on the touch screen formed on a part of the vehicle, 400, when the tab is attached to a specific point of the main body, it is possible to detect that the tab is applied from the user through at least one of the on-touch sensor and the acceleration sensor.

In order to distinguish the taps as the means for inputting the user's authentication information or the taps as the means for controlling the functions of the vehicle control device 400 and simply any external objects, It is judged that a "tab" for the purpose of inputting the user's authentication information or controlling the vehicle control device 400 is detected only when at least two or more tabs are applied to the touch screen formed on a part of the vehicle within the reference time can do. For example, if it is determined that one tap is applied to the touch screen formed on a part of the vehicle as a result of the detection by the tap detection unit 433, the control unit 410 inputs the user's authentication information It may be recognized that an arbitrary external object or human body has collided.

Accordingly, the control unit 410 may be configured to input the authentication information of the user only when at least two or more tabs (or a plurality of times) that are continuously applied within the reference time are detected by the tab detecting unit 433 Quot; tab " as a means for controlling the function of the vehicle control device 400 is detected.

That is, tap gestures may mean that at least two or more tap gestures are continuously sensed within a reference time. Therefore, in the following description, the detection of the "tap " may mean that it has been sensed that a user's finger, a touch pen, or the like is struck by the body of the vehicle control apparatus 400 substantially plural times.

In addition, the controller 410 may determine whether the tab is sensed by the user's other finger or one finger, as well as that the tab is sensed within the reference time. For example, the control unit 410 may control a predetermined area of the vehicle, that is, a window of the vehicle or a part of the windshield glass or a portion of the A, B, C pillars or the vehicle roof, When the tabs are detected in a box, a gear box, or the like, the user can sense whether the tabs are made through one finger or through different fingers, using fingerprints sensed at the portions where the tabs are applied. The control unit 410 recognizes the position of the taps on the display unit 451 or the acceleration generated due to the taps through at least one of the touch sensor or the acceleration sensor provided in the tap sensing unit 433, You can detect if the tabs are made with one finger or each with a different finger.

Further, the control unit 410 determines whether the user has input the tabs through one hand or one finger, or both hands or at least two fingers It is possible to judge whether or not it is inputted.

The taps may be a plurality of tapes continuously sensed within the reference time. Here, the reference time may be a very short time, for example, 300 ms to 2 s or less.

If the tap detecting unit 433 detects that the main body of the vehicle control apparatus 400 has been tapped for the first time, the tap detecting unit 433 detects the tap for the first time after the first tapping You can detect if there is a tap. If the next tap is detected within the reference time, the tap sensing unit 433 or the control unit 410 may input the authentication information of the user or the specific function of the vehicle control device 400 according to the embodiment of the present invention. It can be determined that a tab for controlling is detected. In this way, the controller 410 recognizes the second tab only when the second tab is detected within a predetermined time period after the first tab is detected, thereby enabling the user to input the authentication information, Or whether it is merely a user's mistake or an unintended object crashing outside or inside the vehicle.

There are various methods for allowing the "valid tab" to be recognized. For example, the control unit 410 may control the second tap for tapping the body by the second reference number or by the second reference number within a predetermined time after the first tap is detected by the first reference count or by the first reference count or more, May be recognized as the "valid tap ". Here, the number of the first reference times and the number of the second reference times may be equal to or different from each other. For example, the first reference number may be one, and the second reference number may be two. As another example, the first reference number and the second reference number may all be one.

Also, the control unit 410 may determine that the "tabs " are detected only when the tab is within the" predetermined area ". That is, when it is detected that the main body of the vehicle controller 400 is tapped for the first time, the controller 410 may calculate a predetermined area from the point where the first tapping is detected. Then, the control unit 410 determines whether or not the first hit is followed by the first or second reference count or the first or second reference count or more within the reference time from the time when the first hit is sensed It can be determined that the first tab or the second tab is applied.

It is to be understood that the reference time and the predetermined region may be variously modified according to the embodiment.

Meanwhile, the first tap and the second tap may be sensed not only as a reference time and a predetermined area, but also as separate tabs depending on the detected position of each tab. That is, the controller 410 can determine that the first tab and the second tab are applied when the second tab is detected at a position spaced apart by a predetermined distance from the position where the first tab is sensed. When the first tap and the second tap are recognized based on the sensed position, the first tap and the second tap may be simultaneously detected.

Also, when the first tap and the second tap are configured by a plurality of touches, that is, a plurality of tapes, a plurality of touches constituting the first tap and the second tap may be simultaneously detected. For example, when the first touch constituting the first tab is detected and the first touch constituting the second tab is positioned at a position distant from the detected position of the first touch of the first tab If it is sensed, the controller 180 can sense the first touch constituting the first tap and the second tap, respectively. The controller 410 senses an additional touch input sensed at each position and determines that the first tab and the second tab are applied when a touch is detected at a first reference frequency or at a second reference frequency or more at each position can do.

If the tab of the main body of the vehicle control device 400 is detected a plurality of times from the tap sensing unit 133, the control unit 410 not only authenticates the user, 400 may be controlled so that at least one of the executable functions is controlled. Here, the functions executable on the vehicle control device 400 may mean all kinds of functions that can be executed or driven by the vehicle control device 400. [ Here, one of the executable functions may be an application installed in the vehicle control apparatus 400. And "an arbitrary function is executed" may mean that "any application program is executed or driven in the vehicle control apparatus 400 ". For example, the control unit 410 may play a music file based on a plurality of tabs of the user detected in the console box, or may control the navigation to automatically set a path to a predetermined destination.

As another example, a function executable in the vehicle control device 400 may be a function necessary for basic driving of the vehicle control device 400. [ For example, the functions required for basic driving may be a function of turning on / off an air conditioner or a hot air fan provided in the vehicle, and may be a function for starting the vehicle, switching the vehicle from the locked state to the unlocked state, And a function of switching from a state to a locked state. Or to turn the cruise control function of the vehicle on or off.

On the other hand, the control unit 410 can form a position where the user's authentication information is input based on the body of the user's tab or a point on the touch screen. For example, the control unit 410 forms an area for receiving the pattern information, centering on a point where the first user's taps are applied, or forms an area for receiving the user's biometric information, . In this case, the point at which the user's pattern information or the biometric authentication information is input may be changed every time, even if the user taps the body or another point on the touch screen every time. Accordingly, the user can minimize the exposure of the authentication information, and thus can be safer from a threat such as theft of the vehicle.

Needless to say, the user authentication information may be input to the vehicle control device 400 via a predetermined external device based on the user's selection. That is, for example, the sensing unit 430 may be connected to a predetermined external device outside the vehicle using the short-range communication module 431, and authentication information of the user input through the external device may be transmitted to the short- 431 to the sensing unit 430 and may be authenticated by the control unit 410.

The vehicle control apparatus 400 can acquire vehicle information from the vehicle.

According to one embodiment, the vehicle information may be acquired through the sensing unit 430 described above.

According to another embodiment, the vehicle control apparatus 400 may further include a vehicle information obtaining section 460 for obtaining the vehicle information.

Here, the vehicle information includes at least one of an air conditioning function for the vehicle, whether or not the door (including a bonnet, a trunk, and a fuel supply port) is open or closed, whether a window is open or closed, whether a sunroof is opened or closed, A parking position, a navigation function provided in the vehicle, a stolen state of the vehicle, and a state of running of the vehicle.

The vehicle information may further include information related to at least one of an operation speed of the vehicle, an operation acceleration, a travel distance to the present, a rapid acceleration frequency, and a sudden braking frequency.

For this purpose, the vehicle information obtaining unit 460 may perform communication with various sensors provided in the vehicle.

For example, the vehicle information acquiring unit 460 may acquire acceleration information for the vehicle by communicating with an acceleration sensor installed in the vehicle and measuring an acceleration of the vehicle.

In addition, for example, the vehicle information obtaining unit 460 can communicate with the black box provided in the vehicle to obtain an accident image when a vehicle accident occurs. In this case, the memory 440 may store the accident image.

The image information acquiring unit 470 acquires image information about the user from the image acquiring apparatus 900.

The image acquisition apparatus 900 acquires an image for a user as means for processing an image frame such as a still image or a moving image obtained by an image sensor such as a camera.

The image acquiring unit 900 may acquire image information through various communication means from the image acquiring apparatus 900. [

6B shows an example in which the predetermined external device is connected to the vehicle control apparatus according to the embodiment of the present invention.

6B, the predetermined external device 170 may be a mobile terminal such as a smart phone 171 or a smart key 172 of the user. In this case, the controller 180 recognizes the unique information of the external device 170, and automatically recognizes a specific user when the external device 170 is within a predetermined distance. The control unit 410 may receive the authentication information input by the user through the external device 170. The authentication information input by the external device 170 may include a communication module provided in the external device 170, And may be transmitted to the vehicle control device 400 through the short range communication module 431 of the sensing unit 430.

The authentication information may be biometric information of the user.

According to one embodiment, the biometric information may be obtained by the mobile terminal 100 or the wearable device 200, 200 '.

Here, the biometric information may be at least one of information on the heart rate, body fat, blood pressure, blood glucose, face shape, fingerprint, brain waves, and iris for the user.

For example, the authentication information may be a user's heart rate, fingerprint or iris recognition information, or predetermined password or pattern information. Such authentication information may also be information related to a specific user's gesture.

The authentication for the user may be based on a plurality of biometric information or a plurality of authentication information.

For example, authentication of the user may be performed by checking both the heart rate of the user and the gesture of the user.

The external device 170 may be configured to receive authentication information from a user, that is, to receive at least some of the sensors provided in the sensing unit 430 of the vehicle control device 400, And may further include configurations that are the same, similar or corresponding to those of the sensors.

For example, the external device 170 such as the smart key 172 or the smart phone 171 may be provided with a touch screen on which a user can input pattern information or a touch screen on which the user can input pattern information, A sensing unit may be provided that is the same as, or similar to, the tap sensing unit 433. The external device 170 may further include a fingerprint recognition unit for recognizing the fingerprint of the user. In addition, the external device 170 may further include an inertial sensor, a gyro sensor, or an acceleration sensor for recognizing the user's gesture.

For example, the smart watch 173, which is a watch type mobile terminal, may include a sensor capable of acquiring a user's heart rate. In the case of the smart glasses 174 as a spectacle type mobile terminal, And an iris recognition camera for recognizing the iris of the subject.

In this case, the user can input his / her authentication information using at least one of the fingerprint, predetermined pattern information, and iris recognition information through the external device 170. Alternatively, the user may be allowed to input his / her authentication information to the external device 170 by taking a specific gesture while wearing the external device 170. In this case, the controller 180 of the external device 170 uses the value of the position change of the external device 170, that is, the measured value of the acceleration measurement value, the gravity change amount, or the inertia change amount according to the gesture of the user So that the gesture of the user can be recognized and utilized as authentication information. Alternatively, the external device 170 may recognize that the position has been changed using a user's image input through a camera or the like, and may measure the changed value.

Meanwhile, when the authentication information is input through the external device 170, the control unit 410 of the vehicle control device 400 can control driving of the vehicle using the input authentication information. For example, the control unit 410 may recognize the current user according to the authentication information, cancel the locked state of the vehicle, and set the vehicle internal environment corresponding to the recognized user. Alternatively, the control unit 410 may change the unlocked state of the vehicle to the locked state when the current vehicle state is the unlocked state and the authentication information is input again once the startup is completed.

Meanwhile, the driving of the vehicle may be immediately controlled using the authentication information of the user input through the external device 170. However, the control unit 410 may request the user for a single authentication process. In this case, when the external device 170 is within a predetermined distance or when authentication information is inputted through the external device 170, the control unit 410 changes the state of the vehicle to a wake up state , And prepare to drive the vehicle according to the authentication information input from the authenticated user. When the user enters the authentication information again in a predetermined area (for example, a window of a driver's seat or a passenger seat, A, B, C pillars, etc.) outside or inside the vehicle in a state of being switched to the wakeup state Receives the input, authenticates the user, and drives the vehicle accordingly.

In addition, although it has been described in the above description that only one authentication procedure is performed, it is needless to say that more authentication procedures can be performed. In the above description, a plurality of authentication procedures are performed when the user's authentication information is input through the external device 170. However, the plurality of authentication procedures may be performed by a user directly in a touch screen area It is needless to say that the present invention can also be applied to the case of inputting the authentication information of the user.

7 is an exemplary view for explaining a display unit of the vehicle control device according to the embodiment disclosed herein.

7 shows a case where the vehicle control apparatus 400 is implemented in the form of a head unit of a vehicle.

The vehicle control apparatus 400 may include a plurality of display units D100 to D102.

For example, as shown in FIG. 7, the vehicle control apparatus 400 includes a first display unit D100 in front of the driver's seat and two second display units D101 and D102 in front of the rear seats .

At this time, in general, only the first display unit D100 is restricted from the driving regulation.

Therefore, only the first display unit D100 is limited by the vehicle display method, and the second display units D101 and D102 can display the entire contents without any limitation.

8A is a block diagram showing a configuration of a vehicle control device according to another embodiment disclosed herein.

FIG. 8A illustrates a case where the vehicle control apparatus 400 is implemented in the form of a video display device, a head unit of a vehicle, or a telematics terminal.

8A, the vehicle control apparatus 400 'includes a control section (for example, a central processing unit (CPU) 412') for controlling the overall operation of the vehicle control apparatus 400 ' A key control unit 411 'for controlling various key signals, and an LCD control unit 412 for controlling an LCD (Liquid Crystal Display) And a main board 410 'having a built-in mother board 414'.

The memory 413 'may store map information (map data) for displaying the route guidance information on the digital map. In addition, the memory 413 'may store traffic information collection control algorithm that allows the vehicle to input traffic information according to the current road conditions, and information for controlling the algorithm.

The main board 410 'is provided with a unique device number and is provided with a CDMA (Code Division Multiple Access) module 406' embedded in the vehicle, a GPS signal for tracking the location of the vehicle, A GPS module 207 that receives traffic information collected by a user or transmits traffic information collected by a user as a Global Positioning System (GPS) signal, a CD deck 408 'for reproducing a signal recorded on a compact disc (CD) A gyro sensor 409 ', and the like. The CDMA module 406 'and the GPS module 407' can transmit and receive signals through the antennas 404 'and 405'.

In addition, the broadcast receiving module 422 'is connected to the main board 410' and can receive broadcast signals through the antenna 423 '. The main board 410 'includes a display unit (LCD) 401' under the control of the LCD control unit 414 'through an interface board 430', a front board (not shown) under the control of the key control unit 411 ' 402 'and a camera 427' for photographing the interior and / or exterior of the vehicle. The display unit 401 'displays various video signals and character signals, and the front board 402' includes buttons for inputting various key signals. The key signals corresponding to the user- (410 '). Also, the display unit 401 'includes a proximity sensor and a touch sensor (touch screen).

The front board 402 'has a menu key for directly inputting traffic information, and the menu key may be controlled by the key controller 411'.

The audio board 417 'is connected to the main board 410' and processes various audio signals. The audio board 417 'includes a microcomputer 419' for controlling the audio board 417 ', a tuner 418' for receiving a radio signal, a power source unit for supplying power to the microcomputer 419 ' And a signal processing unit 415 'for processing various audio signals.

The audio board 417 'includes a radio antenna 420' for receiving a radio signal and a tape deck 421 'for reproducing an audio tape. The audio board 417 'may further include a sound output unit (for example, an amplifier) 426' for outputting a sound signal processed by the audio board 417 '.

The audio output unit (amplifier) 426 'is connected to the vehicle interface 424'. That is, the audio board 417 'and the main board 410' are connected to the vehicle interface 424 '. The vehicle interface 424 'may be connected to a handsfree 425a' for inputting a voice signal, an airbag 425b 'for occupant safety, a speed sensor 425c' for detecting the speed of the vehicle, and the like. The speed sensor 425c 'calculates the vehicle speed and provides the calculated vehicle speed information to the central processing unit 412'.

The navigation session 400_1 'applied to the vehicle control device 400' generates route guidance information based on the map data and the vehicle current position information, and notifies the user of the generated route guidance information.

The display unit 401 'senses the proximity touch in the display window through the proximity sensor. For example, the display unit 401 'detects the position of the proximity touch when a pointer (for example, a finger or a stylus pen) is touched proximately, and obtains position information corresponding to the detected position And outputs it to the controller 412 '.

The speech recognition device (or speech recognition module) 401_1 'recognizes the speech uttered by the user and performs the corresponding function according to the recognized speech signal.

The navigation session 400_1 'applied to the vehicle control device 400' displays a traveling route on the map data, and when the position of the mobile terminal 100 is determined in advance from a dead zone included in the traveling route, (For example, a vehicle navigation device) mounted on a nearby vehicle and / or a mobile terminal carried by a neighboring pedestrian through a wireless communication (for example, a short-range wireless communication network) The location information of the surrounding vehicle is received from the terminal mounted on the peripheral vehicle, and the location information of the surrounding pedestrian is received from the mobile terminal carried by the peripheral pedestrian.

Meanwhile, the main board 410 'may be connected to the interface unit 430', and the interface unit 430 'includes the external device interface unit 431' and the network interface unit 432 '.

The external device interface unit 431 'can connect the external device and the vehicle control device 400'. To this end, the external device interface unit 431 'may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 431 'may be connected to an external device such as a DVD (Digital Versatile Disk), a Blu ray, a game device, a camera, a camcorder, a computer (notebook) The external device interface unit 231 transmits external video, audio or data signals to the control unit 412 'of the vehicle control device 400' through the connected external device. Further, the controller 412 'can output the video, audio, or data signal processed by the controller 412' to the connected external device. To this end, the external device interface unit 431 'may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The A / V input / output unit includes a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, an S-video terminal (analog) terminal, A DVI (Digital Visual Interface) terminal, an HDMI (High Definition Multimedia Interface) terminal, an RGB terminal, a D-SUB terminal, and the like.

The wireless communication unit can perform short-range wireless communication with other electronic devices. The vehicle control apparatus 400 'may be, for example, Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, DLNA Alliance), and the like.

Also, the external device interface unit 431 'may be connected to at least one of various set-top boxes and various terminals to perform input / output operations with the set-top box.

On the other hand, the external device interface unit 431 'can receive the application or application list in the adjacent external device and transmit it to the control unit 412' or the memory 413 '.

The network interface unit 432 'provides an interface for connecting the vehicle control device 400' to a wired / wireless network including the Internet network. The network interface unit 432 'may include an Ethernet terminal or the like for connection to a wired network and may be connected to a wireless LAN (WLAN) (Wi -Fi, Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), and HSDPA (High Speed Downlink Packet Access) communication standards.

The network interface unit 432 'may transmit or receive data to another user or other electronic device via the network to which it is connected or another network linked to the network to which it is connected. Particularly, it is possible to transmit a part of content data stored in the vehicle control device 400 'to another user registered in the vehicle control device 400' or a selected user or selected electronic device among other electronic devices.

On the other hand, the network interface unit 432 'can access a predetermined web page through the connected network or another network linked to the connected network. That is, it is possible to access a predetermined web page through a network and transmit or receive data with the server. In addition, content or data provided by a content provider or a network operator can be received. That is, it can receive contents such as a movie, an advertisement, a game, a VOD, a broadcast signal, and related information provided from a content provider or a network provider through a network. In addition, the update information and the update file of the firmware provided by the network operator can be received. It may also transmit data to the Internet or a content provider or network operator.

In addition, the network interface unit 432 'can select and receive a desired one of applications open to the public via the network.

8B is a block diagram showing a configuration of a vehicle control apparatus according to another embodiment disclosed herein.

8B is a configuration diagram showing the vehicle control device 400 with emphasis on the function of the vehicle navigation device.

When the vehicle control apparatus 400 '' is implemented as a vehicle navigation apparatus, the vehicle control apparatus 400 '' may be classified into an in-dash type and an on-dash type, ). The navigation system of the Indy type is inserted into a fixed space allocated in a dashboard of the vehicle and is fixedly mounted. The on-deck type navigation (vehicle navigation) device is mounted on a dashboard of a vehicle or installed near the vehicle by using a fixed support, and can be detachably attached to the vehicle.

The vehicle control apparatus 400 '' according to the present embodiment includes navigation system (vehicle navigation) apparatuses of the intra-city type and the on-scene type. In addition, a navigation message transmitted from GPS (Global Positioning System) And a portable terminal capable of performing a navigation function in conjunction with a GPS receiver that receives the traffic information.

8B, the vehicle control device 400 '' receives a global positioning system (GPS) signal received from a satellite, and transmits the GPS signal to the navigation device 100 ) Which generates second vehicle position data based on the traveling direction of the vehicle and the speed of the vehicle, a GPS module 401 '' for generating first vehicle position data of the vehicle (Or memory) 404 '' for storing map data and various information; a vehicle position estimating unit 402 for estimating a vehicle based on the first vehicle position data and the second vehicle position data; (Map matching link or map matching road) in the map data stored in the storage unit 404 ", and outputs the matched map information Results) A communication unit 408 '' for receiving real-time traffic information from the information providing center and / or a nearby vehicle through the wireless communication network 500 '', receiving the traffic light information, and performing telephone communication through the wireless communication network 500 ' A control unit 407 '' for generating route guidance information based on the matched map information (map matching result), a route guidance map (including interest information) included in the route guidance information, A voice output unit 406 '' for outputting a voice signal corresponding to the voice guidance information (the voice guidance voice message) and the traffic signal light information included in the voice guidance information included in the voice guidance information, . ≪ / RTI >

Here, the communication unit 408 '' may further include a hands-free unit having a Bluetooth module, and may receive a broadcast signal including traffic information in TPEG format from a broadcasting station through an antenna. The broadcast signal may include not only video and audio data according to various standards such as terrestrial or satellite digital multimedia broadcasting (DMB), digital audio broadcasting (DAB), digital video broadcasting (DVB-T, DVB- Service, traffic information according to a Binary Format for Scene (BIFS) data service, and various additional data. Further, the communication unit 408 " tunes the signal band in which the traffic information is provided, demodulates the tuned signal, and outputs the demodulated signal to the TPEG decoder (included in the control unit 407).

The TPEG decoder decodes traffic information in the TPEG format and provides various information including traffic signal information included in the traffic information to the controller 407 ''.

The route guidance information may include not only map data but also various information related to driving such as lane information, traveling speed limit information, turn-by-turn information, traffic safety information, traffic guidance information, vehicle information, .

The signals received through the GPS module 401 " include 802.11, which is a standard specification of a wireless network for a wireless LAN including a wireless LAN and some infrared communication proposed by IEEE (Institute of Electrical and Electronics Engineers) A Broadband Wireless Access (BWA) system including 802.15, which is a standard for a wireless personal area network (PAN) including a UWB, a ZigBee, and the like, and a fixed broadband network (FWA) , 802.20, which is a standard for the mobile Internet, and 802.20, which is a standard for the mobile Internet (MBWA) including WiBro, And to provide positional information of the terminal to the vehicle control device 400 ".

The vehicle control apparatus 400 '' may further include an input unit, and the input unit may be a variety of devices such as a keypad, a touch screen, a jog shuttle, and a microphone, .

The map matching unit 403 '' generates a vehicle estimated position based on the first position data and the second position data, and reads the map data corresponding to the traveling path from the storage unit 404 '' do.

The map matching unit 403 '' matches the vehicle estimated position with a link (road) included in the map data, and outputs the matched map information (map matching result) to the controller 407 ''. . For example, the map matching unit 403 " generates a vehicle estimated position based on the first position data and the second position data, and stores the generated vehicle estimated position and the storage unit 404 & , And outputs the matched map information (map matching result) to the control unit 407 ". The map matching unit 403 '' may output the road attribute information such as a single-storey road or a multi-storey road included in the matched map information (map matching result) to the control unit 407 ''. Also, the function of the map matching unit 403 " may be implemented in the control unit 407 " '.

The storage unit 404 " stores map data. At this time, the stored map data includes geographic coordinates (geographic coordinates or latitude and longitude coordinates) indicating latitude and longitude in units of minutes (DMS unit: Degree / Minute / Second). In addition to the geographical coordinates, UTM (Universal Transverse Mercator) coordinates, UPS (Universal Polar System) coordinates, and TM (Transverse Mercator) coordinates may be used as the stored map data.

The storage unit 404 '' stores various information such as various menu screens, point of interest (POI), and function characteristic information according to a specific position of the map data.

The storage unit 404 '' stores various user interface (UI) and / or graphical user interface (GUI).

The storage unit 404 '' stores data and programs necessary for the navigation apparatus 400 to operate.

The storage unit 404 '' stores destination information inputted from the user through the input unit. At this time, the destination information may be a destination, or a source and a destination.

The display unit 405 " displays image information (or a route guidance map) included in the route guidance information generated by the control unit 407. [ Here, the display unit 405 includes a touch sensor (touch screen) and a proximity sensor. The route guidance information includes not only map data but also various types of information related to driving such as lane information, running restriction speed information, turn-by-turn (TBT) information, traffic safety information, traffic guidance information, May be included.

When displaying the image information, the display unit 405 '' displays various contents such as various menu screens and route guidance information by using the user interface and / or graphical user interface included in the storage unit 404 can do. Here, the content displayed on the display unit 405 " includes various text or image data (including map data and various information data) and a menu screen including data such as icons, list menus, and combo boxes.

The voice output unit 406 '' outputs voice information (or voice message for the route guidance information) included in the route guidance information generated by the control unit 407. Here, the audio output unit 406 " may be an amplifier or a speaker.

The control unit 407 '' generates the route guidance information based on the matched map information and outputs the generated route guidance information to the display unit 405 '' and the voice output unit 406 '' . At this time, the display unit 405 '' displays the guidance information.

Real-time traffic information is received from the control unit 407 '', the information providing center, and / or a terminal (vehicle navigation device) installed in the nearby vehicle, thereby generating the route guidance information.

The control unit 407 '' is connected to the call center through the communication unit 408 '' to perform a telephone conversation or transmit / receive information between the vehicle control device 400 '' and the call center . Here, the communication unit 408 '' further includes a hands-free module having a Bluetooth function using a short-range wireless communication system.

When the POI search menu is selected by the user, the controller 407 '' searches POIs located on the route from the current position to the destination, and displays the retrieved POI on the display unit 405 ". At this time, the control unit 407 '' determines whether the POI located on the route (POI located on the left or right side of the travel road, for example, where the route is not to be changed (re-searching)) and / (A point at which a path should be changed, for example, a point at which a predetermined path is to be changed to pass through a surrounding POI) located in the POI 405, and displays the retrieved POI on the display unit 405 ".

8C is an exemplary diagram illustrating a screen associated with the navigation function according to an embodiment disclosed herein.

The screen shown in FIG. 8C may be a screen displayed by the mobile terminal 100, the wearable device 200, 200 'or the vehicle control device 400.

The mobile terminal 100 may be a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP) 8C, only some of the components shown in FIG. 8C may be displayed, or all of the illustrated components shown in FIG. 8C may not be displayed.

As shown in FIG. 8C, the display unit providing a screen related to the navigation function may display an icon I1 indicating the orientation of the map displayed in one area of the screen. The display unit providing a screen related to the navigation function may display a map such that a specific direction (for example, a true north direction of the earth), a moving direction of the moving object, a direction in which the destination is located, and the like are fixed and displayed in the upward direction of the screen.

The display unit providing the screen related to the navigation function may display an icon (I2) indicating whether the sound output module (162) is activated and volume setting in one area of the screen. The user can activate or deactivate the sound output module 162 or adjust the volume thereof by applying a touch input to the icon I2.

The display unit may display an icon (I3) indicating whether or not to activate a route search function using Transport Portal Experts Group (TPEG) in one area of the screen. TPEG (Transport Portal Experts Group) is a group for establishing a traffic information protocol established by the European Broadcasting Union in 1997. However, the navigation system is widely known as a route guidance function using real time traffic situation information have.

The display unit may display an icon (I4) indicating a scale of the map data in one area of the screen.

The display unit may display an icon (I5) indicating a current time in one area of the screen. In addition, the display unit 161 may display an icon I6 indicating an estimated time to reach a predetermined destination in one area of the screen. In addition, an icon indicating the expected time to reach the predetermined destination may be displayed.

The display unit may display an icon (I7) indicating a remaining distance to a predetermined destination in one area of the screen.

The display unit may display an icon (I8) for enlarging a map displayed in one area of the screen or an icon (I8 ') for zooming.

The display unit may display an icon (I9) indicating a position and a moving direction of the moving object in one area of the screen. The icon I9 may be displayed on the map at a position corresponding to the current position of the mobile unit. In addition, the moving direction of the moving object can be indicated by the arrow I9 in the direction of the vertex of the arrow or the like.

The display unit may display an icon (I10) indicating a place name of the area where the moving object is located in one area of the screen.

The display unit may display an icon (I11) indicating a lane configuration of the traffic lane when the mobile body is a road lane on which a moving object travels in one area of the screen.

The display unit may display a route necessary for reaching the predetermined destination (I12, see FIG. 8). The path may not be displayed if the destination of the moving object is not set.

All the functions (e.g., navigation function) performed by the vehicle control device 400 described above are performed by the mobile terminal 100 or the wearable device 200, 200 'connected to the vehicle control device 400 by wire or wirelessly, ). ≪ / RTI >

Alternatively, the vehicle control device 400 and the mobile terminal 100 may perform all the functions in conjunction with or in association with each other.

To this end, the mobile terminal 100 or the wearable device 200 may include components that are the same, similar or corresponding to those included in the vehicle control apparatus 400.

For example, the acceleration sensor included in the vehicle control apparatus 400 may be provided by the acceleration sensor provided in the mobile terminal 100 or the wearable device 200 or 200 '.

According to one embodiment, a VNC (Virtual Network Computing) scheme may be applied to interworking or linking the vehicle control apparatus 400 and the mobile terminal 100. [

VNC (Virtual Network Computing) may refer to a graphical desktop sharing system that remotely controls another computer (or other terminal) using a RFB protocol in a computer environment.

The VNC may provide a method of transmitting a keyboard, a mouse, or a touch event from one terminal to another and updating the graphic screen via the network.

In addition, all the functions performed by the vehicle control apparatus 400 described above can be performed by dividing the vehicle control apparatus 400 and the mobile terminal 100 or the wearable device 200, 200 '.

That is, when a specific function among all the functions performed by the vehicle control device 400 is performed, some of the specific functions are performed by the vehicle control device 400, and the remaining functions are performed by the mobile terminal 100) or the wearable device (200, 200 ').

For example, in the case of the air conditioning function for the vehicle, an input to the set temperature for the air conditioner provided in the vehicle is performed by the mobile terminal 100 or the wearable device 200 or 200 ' The function of controlling the air conditioner at the set temperature may be performed by the vehicle control device 400. [

It will be apparent to those skilled in the art that the description of the vehicle control apparatus disclosed herein may be embodied in other specific forms without departing from the spirit of the invention disclosed herein.

Hereinafter, the mobile terminal or the vehicle control device for judging the carelessness of the driver will be described in detail with reference to Figs. 9A to 13, according to an embodiment disclosed herein.

A mobile terminal for judging a carelessness of a driver

Hereinafter, a mobile terminal for judging a carelessness of a driver according to an embodiment of the present invention will be described in detail, and the contents overlapping with those described above will be omitted and a detailed description will be omitted.

The mobile terminal according to one embodiment of the present invention includes a biometric information acquiring unit 145, a control unit 180 for detecting a dangerous driving state, a user input unit 130 for receiving a mode selection for selecting a to- And a communication unit 110 for transmitting the dangerous driving state to another terminal.

The biometric information acquiring unit 145 acquires biometric information through a specific body part of the user as described above.

According to one embodiment, the bio-information acquiring unit 145 can measure a bio-signal generated by a physiological potential difference of the human body. For example, as described above, the bio-information acquiring unit 145 acquires bio-signals such as PPG, ECG, GSR, EEG, A sensor, and / or a sensor.

The control unit 180 can detect a dangerous driving state for the user based on the biometric information acquired through the biometric information acquisition unit 145. [

Here, the dangerous driving state for the user may include at least one of a drowsiness driving state, a driver distraction driving state, and a stress state for the user.

In particular, the controller 180 may determine the drowsiness operation state and / or the stress state of the user based on the biometric information.

In addition, the control unit 180 can determine the drowsiness operation state and / or the eyeball dispersion operation state of the user based on the image information about the user received from the image acquisition apparatus through the communication unit 110. [

That is, the control unit 180 can determine the drowsiness operation state, the eyeball dispersion operation state, and the stress state of the user based on the image information and / or the biometric information.

In addition, the controller 180 may generate risk level information for each of the dangerous driving states based on the image information and / or the biometric information.

That is, when the controller 180 determines the drowsy driving state of the user, it is preferable that the controller 180 determines based on both the image information and the biometric information. When the controller 180 determines the drowsiness driving state based on both the image information and the biometric information, it is possible to accurately determine which of the evaluation steps for drowsiness is divided by the user to be.

On the other hand, when the user boarding the first vehicle, the controller 180 may correct the initial value of the image information and / or the biometric information for a predetermined time before determining the dangerous driving state and / And determines the image information and / or the biometric information on the basis of the corrected initial value.

The details of the control unit 180 determining the dangerous driving state will be described later.

The communication unit 110 is a means for establishing a communication connection between the mobile terminal 100 and another terminal, and transmits the dangerous driving state detected by the controller 180 to another terminal.

At this time, the user input unit 130 receives a mode selection input through the communication unit 110 to select an object to which a dangerous driving state is to be transmitted.

Accordingly, the control unit 180 transmits the dangerous driving condition information to the other terminal corresponding to the mode selection input received through the user input unit 130. [

Here, if the counterpart terminal that transmits the dangerous driving condition information through the communication unit 110 receives the dangerous driving condition and can output it to the outside, the type and form of the counterpart terminal are not limited, For example, it may be a mobile terminal 100, a vehicle control device 400, or a vehicle output unit (not shown) provided in a vehicle in the form of a wearable device such as a watch, a spectacle, or the like.

In addition, the dangerous driving condition information includes information on whether the user wearing the mobile terminal 100 is in a drowsy driving state, a drowsy driving state, and a stressed state, And may include cognitive risk level information.

According to a specific example, when the controller 180 detects that the current driver is in the drowsy driving state during the dangerous driving state, the control unit 180 instructs the mobile terminal of the other user, Data including information can be transmitted.

Accordingly, the passenger can recognize that the present driver from the mobile terminal is in the drowsy driving state, and the passenger can directly orally warn the driver of the drowsy driving.

The mobile terminal 100 may notify the wearer of the result of the dangerous driving state determined by the control unit 180 directly. However, the user who is not in the dangerous driving state may not recognize the alarm from the mobile terminal 100 Therefore, the control unit 180 notifies another user who is not in the dangerous driving state of the dangerous driving state and warns the user who is in the dangerous driving state from the other user so that the driver can quickly get out of the dangerous driving state have.

Meanwhile, as described above, the controller 180 according to an exemplary embodiment of the present invention can detect a dangerous driving state for a user using image information about the user from an image acquisition device provided in the vehicle.

Specifically, the control unit 180 can determine the drowsiness operation state and / or the eyeball dispersion operation state of the user based on the image information.

That is, the control unit 180 can determine the drowsiness operation state, the eyeball dispersion operation state, and the stress state of the user based on the image information and / or the biometric information.

In addition, when the controller 180 determines the drowsy driving state of the user, it is preferable to determine the user's risk information based on both the image information and the biometric information in order to accurately determine the degree of drowsiness.

In this case, although the biometric information acquisition unit 145 for acquiring biometric information may be provided in various forms in the mobile terminal 100, it is also possible that the image acquisition device for acquiring image information is arranged in the vehicle Let's take a closer look at the example.

9A and 9B are illustrations of an image acquisition device according to an embodiment disclosed herein installed in a vehicle.

The image acquisition apparatus 900 acquires an image for a user as means for processing an image frame such as a still image or a moving image obtained by an image sensor such as a camera.

At this time, it is preferable that the image capturing apparatus 900 is installed toward the user or the passenger seat in the vehicle to acquire an image for the user, and the image capturing apparatus 900 may be located inside or outside the vehicle.

As will be described later, since the mobile terminal according to the embodiment disclosed herein is based on the user's facial expression in addition to the user's biometric information when determining the user's drowsiness, carelessness, stress, etc., 9), for example, as shown in Fig. 9, is provided inside the vehicle, and is mounted on the rotation axis (see Fig. 9B) of the instrument panel or its periphery .

Of course, the image capturing apparatus 900 may be fixed to one side of the vehicle by a mountable mount, and an image capturing apparatus 900 fixed to the mount may be fixedly installed to face the user.

The control unit 180 may control the image acquisition apparatus 900 to acquire images obtained by the plurality of image acquisition apparatuses 900. The image acquisition apparatus 900 may acquire images for the user, Information of the user can be obtained.

In order to acquire three-dimensional image information about the user, it is preferable that each of the plurality of image capturing apparatuses 900 is installed toward the user, and is installed to face the user in different directions.

As will be described later, the image information to be used is recognized from the plurality of image capturing apparatuses 900 to recognize the face of the user, the eye open time, the eye flicker, the face direction, and the gaze direction, (Or head) from the plurality of image capturing apparatuses 900 may calculate the direction of the coordinates or the face located in the space, and may be a plurality of captured images among the plurality of image capturing apparatuses 900 The controller 180 selects one of the image acquiring apparatuses 900 having a high recognition rate for the user's face and uses the image information acquired on the basis of the selected image acquiring apparatus 900, Eye flicker, face direction, and gaze direction can be recognized.

Further, in order to acquire an image of ROI (Region Of Interest) according to the user's gaze coordinates, the plurality of image capturing apparatuses 900 may be installed toward the plurality of regions so as to acquire image information for a plurality of regions inside and outside the vehicle have.

The image acquisition device 900 may acquire image information of an area inside or outside the vehicle with a small number of image acquisition devices or may acquire image information of the lens or the image acquisition device 900 itself in up, And at least one motor for moving or rotating the motor in the direction of rotation.

When the image capturing apparatus 900 does not face the ROI or does not recognize the user's face, the lens or the image capturing apparatus 900 itself can be tilted until the user's face is extracted from the acquired image information have.

At this time, the communication unit 110 can receive image information about the user from the image acquisition apparatus 900 provided in the vehicle.

The image capturing apparatus 900 may directly transmit the image information to the mobile terminal 100 or the vehicle control apparatus 400 through a wired or wireless communication unit built in or external to the mobile terminal 100, 400 may receive the image information, and may transmit the received image information to the vehicle control device 400 or the mobile terminal 100.

Hereinafter, the drowsy driving state, the stress state, and the line-of-sight dispersion operation in the dangerous driving state will be described in detail.

Drowsy driving status

The control unit 180 can determine the drowsy driving state of the user in a dangerous driving state based on at least one of the image information and the biometric information.

First, as shown in Table 1 below, the evaluation step for drowsiness can be expressed in seven steps based on human fatigue-scale (HFC) and Stanford sleepiness scale (SSS) Karolineska Sleepiness Scale), it can be expressed in 9 steps.

The control unit 180 determines whether the user's drowsy driving state is a user's facial expression, eye open time, blinking of eyes, or face direction, or a combination thereof, using the image information .

For example, the control unit 180 can determine the drowsiness state of the user according to the ratio of the eyelids floating or winding time during a predetermined time.

Here, the time during which the eyelid is wound can be a time at which the eyelid is wound by about 70% or 80% based on the difference between the maximum size and the minimum size of the user's eye.

As another example, the controller 180 may determine the drowsy driving state of the user based on blinking of the eyes.

Specifically, the sleeping driving state of the user is determined according to a value obtained by averaging the number of blinks of the eyes using the values obtained by squaring the degree of closure of the eyelids that are determined to be blinking, or when the eyes are flickering The sleeping driving state of the user can be determined according to the average value of the number of blinks of the eyes by using the closure rate of the judged eyelid.

At this time, the degree of winding of the eyelids judged to be blinking may be squared based on the larger value of the left eye or the right eye, or the user may be judged according to the averaged value.

As another example, when the blinking of the eyes is determined, the control unit 180 determines the sleeping driving state of the user according to the number of times that the blinking of the eyes does not take long, but is longer than a predetermined time (about 0.5 seconds or 1 second) .

As another example, when the flickering of the eyes is determined, the control unit 180 may determine the drowsiness state of the user based on the time taken for the blinking of the eyes to take longer than the predetermined time.

As another example, when the blinking of the eyes is determined, the control unit 180 determines whether the blinking of the eyes is divided by the number of consecutive occurrences, The state can be judged.

For example, when the blinking of the eye is judged and it is assumed that the time for blinking of the eye is equal to or longer than a predetermined time (about 2 seconds), it is assumed to be represented by "1 ", and in the case of 0000110000111100001110000, 4 + 3) / 3 = 3, the drowsy driving state of the user can be determined based on this value. That is, the controller 180 can evaluate the drowsiness according to the magnitude of the calculated value as described above, and the greater the calculated value, the greater the degree of the drowsiness of the user.

As another example, the controller 180 can determine the drowsy driving state of the user based on the period of flickering of the eyes. Specifically, when the flickering of the eyes is determined, About 2 seconds) or more (n).

As another example, the controller 180 can determine the drowsy driving state of the user based on the speed of blinking of the eyes.

Specifically, the sleeping driving state of the user is determined based on the average value (AECS) calculated by measuring the speed at the time of winding while the eyes are left and dividing the sum of the measured velocities by the number of blinking times, The sleeping state of the user can be judged based on the APCV value obtained by dividing the largest value of the closure rate determined by the blinking of the eyelid by the maximum speed value when the eye is left unfurled have.

At this time, APCV can be based on the value obtained by averaging APCVs calculated based on the most recent value or a plurality of times.

As another example, the controller 180 may determine the drowsy driving state of the user based on whether the pupil is contracted (or whether the iris is loosened).

As another example, when the direction of the face changes in the vertical direction based on the direction of the face, the controller 180 determines that the user's head is nodded and can determine the drowsy driving state of the user.

As another example, the control unit 180 may determine that the user is yawned based on the facial expression of the user, and determine the drowsy driving state of the user.

As described above, the controller 180 can determine the drowsy driving state of the user based on any one of the facial expression of the user's face, the open time of the eyes, the blinking of the eyes, and the face direction, In order to make an accurate judgment as to which stage, it is preferable to determine the drowsy driving state by combining the judgment criteria.

At this time, when the control unit 180 determines the drowsiness driving state or the drowsiness driving evaluation level of the user by combining a plurality of judgment grounds, the plurality of judgment grounds (for example, the expression of the user's face, Flickering and direction of the face, etc.) can be applied.

Here, when the control unit 180 determines the drowsiness driving state for the user and the drowsiness driving evaluation level for the user on the basis of a plurality of factors such as the face of the user, the open time of the eyes, the flickering of the eyes, In order to improve judgment accuracy, the weights for the plurality of elements are preferably set by learning based on a large amount of databases for the elements.

At this time, the learning model can be generated based on the controller 180 classifying the image information and the biometric information of a general user including a user or a user into a database, and based on the generated learning model, Or the risk level.

On the other hand, the control unit 180 can determine the drowsy driving state of the user based on the biometric information acquired through the biometric information acquisition unit 145, as described above.

The control unit 180 may determine a drowsiness state of the user based on the bio-signal of the electromyogram measured through the EMG sensor or based on the electrical skin reflection signal measured by the GSR sensor.

When the control unit 180 determines the drowsiness operation state or the drowsiness operation evaluation level of the user on the basis of the biometric information, the weight value for each of the plurality of biometric information may be applied.

Here, when the controller 180 determines the drowsiness operation state for the user and the drowsiness operation evaluation level for the user based on a plurality of factors such as the values measured by the EMG and GSR sensors, The weights for the plurality of elements (for example, the values measured by the EMG and the GSR sensor) are preferably set by learning based on a large amount of databases for the elements.

At this time, the learning model can be generated based on the database of the image information and the biometric information of the general user including the user or the user by the control unit 180, and the risk driving state or the risk level is determined based on the generated learning model .

On the other hand, when the control unit 180 determines the drowsiness driving state for the user based only on the image information or the drowsiness driving state for the user based only on the biometric information, the range of the evaluation step for the drowsiness driving state is limited, There may be a problem that it is not possible to accurately determine the evaluation level of the drowsy driving state of the user.

Therefore, in order to solve the above problem, the controller 180 preferably determines the drowsiness state of the user based on the biometric information in addition to the image information.

When the controller 180 determines the drowsy driving state or the like for the user based on the image information and the biometric information, the control unit 180 applies weights to the image information and the biometric information, It is desirable to determine which stage corresponds to which stage.

Here, the relation between the weight for the image information and the weight for the biometric information may be changed according to the environment when the controller 180 evaluates the drowsy driving state. However, the weight for the image information is not limited Is larger than the weight for the biometric information.

The relationship between the weight for the image information and the weight for the biometric information may be different for each of the drowsy driving state, the visually dispersed driving state, and the stress state, and accordingly, It is preferable to apply different weights to each of the biometric information to determine which dangerous driving state corresponds or an evaluation level for a specific dangerous driving state.

Meanwhile, when the control unit 180 determines the drowsiness driving state for the user or the drowsiness driving evaluation level for the user, it is preferable to consider both the image information and the biometric information as described above, The calculated sleepiness tendency line can be further considered by using at least one of the sleep state information and the biorhythm information for the user before boarding the vehicle (refer to FIG. 13).

FIG. 13 is a block diagram illustrating a process for determining a drowsy driving state or a drowsiness level according to one embodiment disclosed herein, and FIG. 10 illustrates a drowsiness trend line for a user according to an embodiment disclosed herein .

As shown in FIG. 10, the control unit 180 can predict a change in the evaluation level with respect to the drowsy driving state of the user over time.

While the user is wearing the mobile terminal 100, the mobile terminal 100 can acquire biometric information for the user for a predetermined period of time or at regular intervals using the biometric information acquiring unit 145 .

The mobile terminal 100 may analyze the user ' s sleep pattern before boarding the vehicle, and may induce a drowsiness trend line after the user has boarded the vehicle.

That is, the mobile terminal 100 is measured using an acceleration sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor, or the like included in the sensing unit 140 Based on the information on the movement of one user and information on the respiration, pulse change, and EMG of the user measured using the PPG, EMG, and GSR sensors included in the biometric information acquiring unit 145, State information, and biorhythm information.

Specifically, the control unit 180 may use information measured continuously or periodically for a predetermined period of time, such as at the beginning and end of the sleep, the frequency of the motion during sleep, the degree of motion, the sleep phase The sleeping phase), the rhythm of the sleeping surface, and the pattern of the sleeping surface. Based on the analyzed information, the sleepiness tendency line as shown in Fig. 10 can be calculated.

The drowsiness tendency line is a graph showing a drowsiness level according to time based on whether a user is riding a vehicle or running on a vehicle. As shown in Fig. 10, The sleeping level, the time to maintain the initial sleeping level until the sleeping level changes, and the rate of change of the sleeping level with time.

The controller 180 can predict the sleepiness level for the user by using the drowsiness trend line calculated as described above.

In other words, the mobile terminal 100 analyzes the sleep state information using the biometric information acquired before the user's boarding or driving the vehicle, calculates the sleepiness tendency line based on the analyzed sleep state information, You can predict the level of sleepiness over time after boarding or after driving.

As described above, the mobile terminal 100 according to the embodiment of the present invention predicts the sleepiness level of the user and, before the user is disturbed by the degree of drowsiness of the user, Thereby making it possible to prevent a traffic accident caused by drowsiness.

By combining the first sleeping level calculated based on the image information and the biometric information and the second sleeping level calculated based on the drowsiness trend line when the control unit 180 determines the drowsiness evaluation level of the user, The sleeping level of the user can be calculated.

By applying a predetermined weight to each of the first sleeping level and the second sleeping level, the sleeping level of the user can be finally calculated.

At this time, the weights applied to the first sleeping level and the second sleeping level may be set in advance by user input, or may be adaptively changed according to a confirmation input of the finally determined sleeping level of the user. That is, if the finally determined sleeping level of the user is three levels but the confirmation input inputted through the user input unit 130 is 2 rather than 3, the weighted value can be changed so that the sleeping level of the finally determined user becomes two levels have.

Eye-line dispersion operation state

On the other hand, the controller 180 can determine the visually dispersed driving state of the user in a dangerous driving state based on at least one of the image information and the biometric information.

The control unit 180 may determine the user's face direction and the user's gaze direction or a combination thereof when determining the gaze dispersion operation state of the user using the image information.

For example, the controller 180 calculates the face direction of the user and the user's gaze direction based on the image information acquired through the one or more image capturing apparatuses 900, It is possible to judge the user's line-of-sight dispersion operation state based on the gaze direction.

When the control unit 180 determines the gaze dispersion operation state of the user, the gaze dispersion level of the user can be determined in consideration of the direction of the gaze of the user, the degree of change of the gaze, and the staying time of the gaze.

If the controller 180 determines that the user is in the line-of-sight dispersing operation state by using any one of the face direction of the user and the user's gaze direction or a combination thereof, the controller 180 may control the user of the plurality of image capturing apparatuses 900 It is possible to select at least one image acquisition device 900 corresponding to the sight line coordinates to which the line of sight is directed.

That is, at least one of the plurality of image acquisition apparatuses 900 facing the user's ROI can be selected.

Accordingly, the control unit 180 can recognize at least one object from the image information acquired by the selected image acquisition device 900, and the corresponding control command is performed by the mobile terminal 100 according to the recognized object Or the control command may be transmitted to another mobile terminal 100 or another vehicle control device 400 so that the other mobile terminal 100 or the other vehicle control device 400 may execute the control command.

For example, as shown in FIG. 11A, when the user's gaze is directed to the room mirror M, the controller 180 recognizes that the user's gaze is not looking forward, It can be determined that the vehicle is in the driving state.

The control unit 180 determines that the eye coordinate pointed by the user's eyes is the room mirror M and selects the backward image acquisition device (not shown) corresponding to the room mirror M, And acquire image information from the acquiring device.

Accordingly, the control unit 180 recognizes at least one object from the image information acquired from the backward image acquisition apparatus, and recognizes the face when the object T1 is a person.

When the controller 180 recognizes a face according to a control command corresponding to the object T1, the controller 180 can perform a corresponding control command because the recognized object is a face. Specifically, the control unit 180 can extract the name information corresponding to the recognized face using the face database stored in the memory 160. [

The control unit 180 also receives the biometric information of the object T1 from the facial expression of the recognized face and / or the biometric information obtaining unit of the mobile terminal corresponding to the recognized face, determines the state of the object, And output the determination result through the output unit 150. [

Therefore, the control unit 180 outputs a voice saying " the cabin is sleeping "through the sound output unit 153 based on the image information acquired from the backward image acquisition apparatus, or" Quot ;, so that the user can quickly take the correspondence according to the state of the recognition object.

11B, when the user's gaze is directed to the outside of the driver's window of the vehicle, the control unit 180 recognizes that the user's gaze does not gaze forward, and if the user is in a dangerous driving state It can be determined that the visual line dispersion operation state.

Then, the control unit 180 determines that the eye coordinates of the user's gaze, that is, the ROI is outside the driver's side window of the vehicle, selects the corresponding left direction image acquisition device (not shown) It is possible to acquire image information from the apparatus.

Accordingly, the control unit 180 recognizes at least one object T2 and T3 from the image information acquired from the left-side image acquisition apparatus, recognizes the character string when the objects T2 and T3 are strings, You can extract Internet addresses, phone numbers, or reciprocals from a string.

Therefore, the control unit 180 extracts the mutual name of "COFFEE " and the telephone number of" 02-000-0000 "based on the image information acquired from the left- The extracted information can be stored in the telephone directory.

Meanwhile, the control unit 180 can determine the invisible operation state of the user based on the biometric information acquired through the biometric information acquisition unit 145 including the EOG, the EMG sensor, etc., in addition to the image information, When determining the drowsiness driving state or the like for the user based on the image information and the biometric information, weighting is applied to each of the image information and the biometric information, It is preferable to judge whether it corresponds.

Here, the relation between the weight for the image information and the weight for the biometric information may be changed according to the environment when the controller 180 evaluates the drowsy driving state. However, the weight for the image information is not limited Is larger than the weight for the biometric information.

At this time, the relationship between the weight for the image information and the weight for the biometric information may be different from the weight relation applied for the drowsiness driving state and the stress state.

Stress condition

On the other hand, the controller 180 can determine the stress state of the user in a dangerous driving state based on at least one of the image information and the biometric information.

The control unit 180 may determine the facial expression of the user's face when determining the stress state of the user by using the image information.

For example, if the facial expression of the user is determined to be angry, the stress state of the user can be determined.

According to the facial expression of the user, a large amount of database storing the stress level according to the user's facial expression can be used while determining the stress level level of the user.

In addition, the controller 180 can determine the stress state of the user based on the biometric information acquired through the biometric information acquiring unit 145. [

For example, the controller 180 can determine the degree of stress of the user based on the heart rate and / or heart rate variability measured through the PPG sensor. At this time, in order to increase the accuracy of the stress level measurement, it is possible to pass through a predetermined filter or to remove noise.

When the controller 180 determines the stress level and the stress level of the user based on the plurality of pieces of biometric information, the control unit 180 can apply the weights to each of the plurality of biometric information and improve the determination accuracy of the stress state and the stress level , The weights for the plurality of elements are preferably weight values set by learning based on a large amount of databases.

At this time, the learning model can be generated based on the database of the image information and the biometric information of the general user including the user or the user by the control unit 180, and the risk driving state or the risk level is determined based on the generated learning model .

In the case where the controller 180 determines the stress state of the user based on only the image information or determines the stress state of the user based on only the biometric information, the range of the evaluation step for the stress state is limited, There may be a problem that it is not possible to make an accurate judgment on the evaluation level of the state.

Accordingly, in order to solve the above problem, the controller 180 may determine the stress state of the user based on the biometric information in addition to the image information.

When the controller 180 determines a stress state or the like based on the image information and the biometric information, the control unit 180 applies a weight to each of the image information and the biometric information, It is preferable to judge whether the step corresponds to the step.

Here, the relationship between the weight for the image information and the weight for the biometric information in the evaluation of the drowsy driving state by the controller 180 is not particularly limited as long as it can be changed according to the environment. However, It is preferable that the weight of the image information is larger than the weight of the image information.

The relationship between the weight for the image information and the weight for the biometric information may be different from the weight for the drowsiness driving state and the eyeball dispersion driving state.

Meanwhile, when the controller 180 determines the stress level of the user or the stress level of the user, it is preferable to consider both the image information and the biometric information as described above, but more preferably, The calculated sleepiness tendency line can be further considered by using at least one of the sleep state information and the biorhythm information for the user (see FIG. 13).

The stress state and the stress level of the user may be affected by the sleep state of the user, so that the sleepiness tendency line (see FIG. 10) calculated by the mobile terminal 100 can be used.

That is, when the control unit 180 determines the stress level of the user, the control unit 180 determines whether the stress level is lower than the first stress level calculated based on the image information and the biometric information, The stress level of the correct user can be calculated.

By applying a predetermined weight to each of the first stress level and the second stress level, the user can finally calculate the stress level.

At this time, the weights applied to the first stress level and the second stress level may be set in advance by a user input, or may be adaptively changed according to a confirmation input on the finally determined stress level of the user.

Judge dangerous driving condition

As described above, according to one embodiment disclosed herein, the controller 180 can detect a dangerous driving state for a user based on at least one of image information and biometric information.

Here, the dangerous driving state may include a drowsy driving state, a visual line dispersion driving state, and a stress state.

Also, the control unit 180 can generate risk level information for the drowsiness level, the eyeball dispersion level, and the stress level, which indicate the degree of drowsiness, the degree of eyeball dispersion, and the degree of stress, for each dangerous driving condition.

At this time, when the controller 180 generates the danger level information based on the image information and the biometric information, it is preferable to apply the weight to each of the image information and the biometric information. Different values may be applied depending on whether the weight is a drowsy driving state, a visually dispersed driving state, or a stress state.

As described above, in the case of the drowsy driving state and the line-of-sight dispersing operation state, a weight higher than the biometric information is applied to the image information, and in the case of the stress state, the danger level information is generated by applying a weight higher than the image information to the biometric information .

Herein, it is noted that the relationship between the weight for the image information and the weight for the biometric information may be changed according to the environment when the control unit 180 evaluates the drowsy driving state, the visual dispersion operation state, and the stress state .

Specifically, the control unit 180 may determine that the image information acquired from the at least one image acquiring apparatus 900 includes noise or at least a part of the image information is erroneous through various known means or methods .

At this time, when the reliability of the image information determined by the controller 180 does not reach a predetermined level, the weight applied at the time of the evaluation of the dangerous driving state may be gradually or temporarily set to a predetermined value, It is possible to visually and / or audibly notify the user of abnormality of the information through the output unit 150, the other mobile terminal 100, the vehicle and / or the vehicle control device 400. [

Similarly, the control unit 180 can determine that the biometric information acquired through the biometric information acquiring unit 145 includes noise or at least a part of the biometric information is erroneous through various known means or methods.

At this time, when the reliability of the biometric information determined by the controller 180 does not reach the preset level, the weight applied at the time of the evaluation of the dangerous driving state may be gradually or temporarily set to a predetermined value, It is possible to visually and / or audibly notify the user of abnormality of the information through the output unit 150, the other mobile terminal 100, the vehicle and / or the vehicle control device 400. [

For example, when a failure occurs in one of the image acquiring apparatus 900 and the biometric information acquiring unit 145, a weight applied to the image information or the biometric information may be lowered in the evaluation of the dangerous driving state, It is possible to output information on the abnormality among the acquisition device 900 and the biometric information acquisition unit 145 through the output unit 150, the other mobile terminal 100, the vehicle and / or the vehicle control device 400 have.

Meanwhile, when generating the risk level information based on the image information and the biometric information, the control unit 180 may generate risk level information for at least two of the drowsiness operation state, the eyeball dispersion operation state, and the stress state.

At this time, when the control unit 180 determines which of the drowsy driving state, the visual field dispersion driving state, and the stressed state is the dangerous driving state for the user, it is possible to set a priority corresponding to the danger level information of each dangerous driving state .

Accordingly, when the control unit 180 generates a plurality of risk level information, the control unit 180 determines whether or not the risky driving state for the user corresponds to the drowsy driving state, It is possible to determine which of the stress states.

Print

The output unit 150 may output the dangerous driving state determined by the controller 180 or the danger level information thereof in various forms (various types that can stimulate the user's five senses).

According to one embodiment, the output unit 150 outputs the danger level information in the form of vibration of the main body of the mobile terminal 100, output of the light emitting device, screen, or sound, Can be changed.

12 is an exemplary diagram illustrating various output methods for the danger level information according to an embodiment disclosed herein.

12, the mobile terminal 100 is provided with an alarm AS20 that increases the frequency of the vibration as the danger level increases, based on the vibration type alarm AS10 having the basic amplitude and the fundamental frequency, , An alarm AS30 with an increased amplitude of vibration, or a shorter output period of the vibration can be output.

In addition, the mobile terminal 200 may be configured to generate an alarm AS40 in which the frequency or amplitude of the vibration varies with time based on the vibration level alarm AS10 having the basic amplitude and the fundamental frequency, And output it to the outside.

As described above, according to another embodiment, it is possible to output a sound (for example, "current drowsiness driving state is level 5") or a sound according to frequency, The amplitude and / or the output period, and the like.

The mobile terminal 100 outputs sounds according to the danger level information, but the types of sounds output according to the dangerous driving states may be different.

That is, when the dangerous driving state determined for the user is the drowsy driving state, it is preferable that the sound outputted through the output unit 150 is a sound having high amplitude and high frequency capable of chasing drowsiness, When the state is a stress state, it is preferable that the sound output through the output unit 150 is music that can calm the mind.

According to another embodiment, the dangerous driving state and the danger level information determined by the control unit 180 may be transmitted to the vehicle control unit 400 so that the vehicle control unit 400 may control the output unit 450 of the vehicle control unit Or output the danger level information to the outside through a vehicle output unit (not shown) provided in the vehicle using the vehicle driving unit 420. [

As in the case of the mobile terminal 100, the output unit 450 and the vehicle output unit (not shown) of the vehicle control apparatus change output forms or output intensities as the dangerous operation state or the danger level information changes, can do.

According to one embodiment, when the output unit 450 of the vehicle control apparatus and the vehicle output unit output voice or sound as described above, the type of voice or sound , Frequency, amplitude, and / or output period.

In addition, the vehicle output portion may be at least one of an emergency light of the vehicle, a dash panel, or at least one LED installed around the dash panel, a vibration sheet of the vehicle, and a vibration steerable device of the vehicle. Accordingly, the output period of the emergency light, the color of the LED, the output period of the LED, the vibration frequency of the vibration seat or the vibratable steering device, the vibration amplitude of the vibration seat or the vibratable steering device, And the output period may be differently output.

In addition, the vehicle control apparatus 400 can directly open at least one window installed in the vehicle for ventilation inside the vehicle, or operate the air conditioner of the vehicle to lower the dangerous driving level of the user.

Further, when the danger level is higher than the predetermined level, it is preferable that the vehicle control device 400 directly controls the vehicle operation.

At this time, after at least one of the mobile terminal 100, the vehicle control device 400, and the vehicle output unit performs the above-described action with respect to the dangerous level for the dangerous driving state, or after a predetermined time elapses, It is desirable to repeatedly perform the determination of the danger level and the action accordingly.

When the controller 180 determines that the dangerous driving state is the drowsy driving state, at least one LED in the vicinity of the vehicle instrument panel in the vehicle output section is in the first to third drowsiness levels It can be lit in green.

In addition, when the sleep level is 4th to 5th level, the LED is turned on in yellow, and the output unit 150 outputs an operational predictable time (sleep state information and / or bodily rhythm Information can be judged according to the drowsiness main line calculated based on the information).

Also, at the sixth sleeping level, the LED flashes in yellow, and the output unit 150 can output a predicted time that can be operated by voice.

Also, when the seventh level of drowsiness, the LED is turned on in red, and the output unit 150 continuously generates vibration for 5 seconds at intervals of 5 seconds using the haptic module 155, (For example, "take water") for warning the user by using voice 153 and breaking the drowsiness.

In addition, when the drowsy eighth level, the LED is turned on in red, and the output unit 150 continuously generates vibration for 3 seconds at intervals of 3 seconds using the haptic module 155, The vehicle control device 400 can alerts by voice using the control unit 153 and guide the shoulder or rest area close to the current position to the destination or the stopover.

Also, when the drowsy ninth level, the LED is turned on in red, and the output unit 150 continuously generates vibration for 0.5 seconds at intervals of 0.5 seconds using the haptic module 155, The control unit 400 causes the vehicle emergency light to blink through the device 400 and warns by voice using the sound output unit 153. When the vehicle control apparatus 400 decelerates the vehicle speed or stops from the current position It is possible to control the vehicle to move to a possible place.

As another example, when the control unit 180 determines that the dangerous driving state is a stressed state, at least one LED in the vicinity of the vehicle instrument panel in the vehicle output unit may be lit in green when the stress is at the first level.

Also, when the stress is at the second level, the LED may be turned on in yellow.

In addition, when the stress level is the third level, the LED is turned on in red, and the output unit 150 uses the sound output unit 153 to guide the current user's stress state to the voice, And can output sound or voice that can induce the user.

At this time, the controller 180 can acquire the bio-signal for the respiration of the user through the bio-information acquiring unit 145 to determine whether it is in accordance with the breathing method that can alleviate the stress, , It can be audibly output through the mobile terminal 100, the vehicle control device 400, and / or the vehicle output unit to warn the user not to follow the breathing method.

Other terminal outputs dangerous operation status

The mobile terminal 100 may output a dangerous driving condition result (type or degree of the dangerous driving condition) determined by the control unit 180 directly to the wearer, There is a possibility that the alarm about the dangerous driving state output from the terminal 100 may not be recognized.

Therefore, in order to solve such a problem, the control unit 180 notifies another user who is not in a dangerous driving state of a dangerous driving state and warns a user who is in a dangerous driving state from another user, It is desirable to be able to quickly get out of the state.

As described above, according to one embodiment of the present invention, the dangerous driving state or the danger level information determined by the control unit 180 is transmitted to another mobile terminal, and the other mobile terminal transmits the danger level information through the output unit It can be outputted in the form of vibration or voice of the main body, and it is possible to induce the driver to move out of the dangerous driving state by the person carrying the other mobile terminal.

Specifically, when the control unit 180 determines that the dangerous driving state of the driver is a dangerous driving state, the control unit 180 determines whether the dangerous driving state is a dangerous driving state and / Level to the other terminals.

That is, when the occupant adjacent to the driver receives the dangerous driving condition information for the driver via the mobile terminal carried by the occupant, the occupant directly makes the driver wake up drowsy or gazes forward , So that the stress can be mitigated.

At this time, the control unit 180 of the mobile terminal 100 can determine the type and / or the danger level of the dangerous driving state of the user wearing the mobile terminal 100, The control unit 180 determines whether the user wearing the mobile terminal 100 is a driver of the vehicle according to various known determination methods, can do.

For example, when the mobile terminal 100 recognizes the movement of a specific pattern, recognizes the biometric information or its change recognized through various biometric sensors, or recognizes a change of the biometric information through the communication with the vehicle control device 400 Can determine the terminal corresponding to the recognized driver as the mobile terminal 100 worn by the driver by using the image information of the occupant at the specific position acquired through the image acquisition device 900, The mobile terminal 100 can determine that the mobile terminal 100 is worn by the driver according to various determination methods.

If the other terminal receiving the dangerous operation status information is the mobile terminal 100, the output unit 150 of the other terminal may determine the vibration frequency of the main body, the vibration amplitude of the main body, At least one of the vibration periods of the main body can be changed and output. In addition, it is possible to change at least one of the type, content, frequency, amplitude, and output period of the voice or sound to be output according to the type of the dangerous driving state or the degree of dangerous level.

According to another embodiment, when the other terminal receiving the dangerous driving state information is the output unit 450 or the vehicle output unit of the vehicle control apparatus, the output of the vehicle emergency light output in accordance with the type of dangerous driving state or the degree of danger level The output color of the LED, the output period of the LED, the vibration frequency of the vehicle vibration sheet, the vibration amplitude of the vehicle vibration sheet, the vibration period of the vehicle vibration sheet, the vibration of the vehicle At least one of a vibration frequency of the steering device, a vibration amplitude of the vibration-steerable device of the vehicle, and a vibration period of the vibration-steerable device of the vehicle.

The detailed output method according to the type and / or the danger level of the dangerous driving state of the mobile terminal or the vehicle control apparatus can be performed according to the embodiment described above, so a detailed description thereof will be omitted.

In this way, when the mobile terminal 100 determines the dangerous driving state, the mobile terminal 100 transmits the dangerous driving state information including the dangerous driving state type and / or the dangerous level to the specified other terminal, Through the input unit 130, a mode selection input for an object to which a dangerous driving state is to be transmitted can be received.

FIG. 14 is an exemplary view of a screen for receiving a mode selection input by a mobile terminal according to an embodiment disclosed herein.

14, the mobile terminal 100 according to an exemplary embodiment of the present invention may include a mode for at least one of the automatic mode M1, the personal mode M2, the open mode M3, and the custom mode M4, You can receive a selection input.

According to one embodiment, when the mobile terminal 100 receives a selection input for the automatic mode M1 from the user, the control unit 180 controls the display of the image of the passenger acquired by the image acquisition apparatus 900 included in the vehicle The dangerous operation state information is transmitted to the terminal corresponding to the occupant based on the information, and the other terminal having received the dangerous operation state information can output the dangerous operation state to the outside via the output unit.

Here, the control unit 180 of the control unit 180 and / or the control unit 410 of the vehicle control device 400 recognizes the passenger using the image information acquired through the image acquisition device 900 installed inside or outside the vehicle toward the passenger or boarding seat . The control unit 410 of the control unit 180 and / or the vehicle control device 400 controls the operation of the control unit 180 based on the image information acquired from the image acquisition device 900 installed in the vehicle toward the passenger, (Name, address, etc.) of the terminal corresponding to the recognized specific person or information of the specific person (e.g., name, address, etc.) by using the face database stored in the memory 160 of the vehicle control apparatus 400 and / or the memory 440 of the vehicle control apparatus 400 Can be extracted.

The controller 180 determines that the vehicle is in a dangerous driving state as shown in FIG. 15A. The controller 180 determines whether the driver D or the driver D recognized by the controller 180 of the mobile terminal 100 and / or the controller 410 of the vehicle controller 400, It is possible to transmit the dangerous driving condition information to the other terminals corresponding to at least one of the passenger P1 and the passenger P2 in the rear right seat so that the at least one other terminal can output the dangerous driving state to the outside .

At least one of the at least one other terminal corresponding to the occupant recognized by the controller 180 of the mobile terminal 100 and / or the controller 410 of the vehicle controller 400 may be a mobile terminal 100 worn by a driver, And transmits the dangerous driving state information only to the other terminals within a predetermined distance from the position of the dangerous driving state information.

That is, the control unit 180 determines whether the at least one other terminal corresponding to the recognized occupants is in the dangerous driving state (i.e., the at least one other terminal) based on the position information of the other terminal (including the relative position information with respect to the mobile terminal 100) It is desirable to transmit information.

If the controller 180 determines that the dangerous driving state of the driver D is present and the terminal corresponding to the passenger P1 is far from the position of the mobile terminal 100 as shown in FIG. The controller 180 transmits the dangerous driving state information only to the terminal 102 corresponding to the occupant P2 in the right rear seat.

Accordingly, the terminal 102 of the right rear seat occupant can output the dangerous driving state to the outside, so that the occupant P2 can take action against the driver D in the dangerous driving state.

At this time, the other terminal having received the dangerous driving state information can output the dangerous driving state information, but the output unit of the other terminal determines whether the dangerous driving state, the danger level, And an in-vehicle position of the other terminal.

For example, when the other terminal is located in the vehicle and is close to the driver's mobile terminal, the output intensity of the vibration and sound can be increased or the output period can be shortened. In addition, The output intensity of vibration and sound can be made smaller or the output period can be lengthened as compared with other terminals.

According to another embodiment, the mobile terminal 100 (preferably a mobile terminal worn by the driver) receiving a selection input from the user for the automatic mode M1 may be provided with at least one And can transmit the dangerous driving state information to the recognized other terminal through the communication unit 110 when the dangerous driving state is determined by the driver.

At this time, the other terminal receiving the dangerous operation state information can output the dangerous operation state to the outside through the output unit.

Unlike the above-described embodiment, the control unit 180 recognizes the other terminal, not the passenger, and transmits the dangerous operation status information to the recognized terminal.

A method of recognizing another terminal may be performed by various known methods. For example, the terminal may recognize the other terminal whether communication has been established through the communication unit 110 or transmit / receive predetermined data through the communication unit 110 It is possible to recognize other terminals.

At this time, the counterpart terminal that receives the dangerous operation status information among other terminals recognized by the mobile terminal 100 may be limited to the terminal registered in the memory 160, It may be a terminal that is not registered.

Meanwhile, it is preferable that the other terminal that receives the dangerous driving condition information from the other terminal recognized by the controller 180 is a terminal around the driver located within a predetermined distance range around the mobile terminal 100 worn by the driver.

That is, the controller 180 transmits the dangerous driving state information to another terminal selected based on the position information of the other terminal (including the relative position information with respect to the mobile terminal 100) among other recognized terminals.

As shown in FIG. 15A, the controller 180, which has determined that the mobile terminal 100 is in the dangerous driving state, can recognize the terminal located outside the vehicle, It is preferable to transmit the dangerous driving condition information only to the terminal 102 located in the vehicle.

Accordingly, the in-vehicle occupant P2 can recognize that the driver D is in the dangerous driving state and can take action against the driver D in the dangerous driving state.

At this time, the other terminal having received the dangerous driving state information can output the dangerous driving state information, but the output unit of the other terminal determines whether the dangerous driving state, the danger level, And an in-vehicle position of the other terminal.

According to another embodiment, the mobile terminal 100 (preferably a mobile terminal worn by the driver) receiving an input for the automatic mode M1 from the user through the user input unit 130 may determine the dangerous driving state The dangerous driving state information is transmitted to the vehicle control device 400 having a plurality of display units, and the vehicle control device 400 can output the dangerous driving state through at least one display unit.

At this time, it is preferable that a plurality of display portions are provided for each boarding seat in the vehicle. For example, the display unit of the vehicle control apparatus 400 may be installed near the center fascia D100, the driver's seat, and the backrests D101 and D103 of the passenger seat as shown in Fig. 15B.

At least one display unit for outputting the dangerous driving state may be a display unit for displaying the dangerous driving state of the occupant recognized by the control unit 410 of the vehicle control device 400 based on the image information about the occupant in the vehicle And may be a display unit installed at a position corresponding to the boarding position (or boarding seat). For example, the control unit 410 may determine the presence or absence of an occupant on the seat of the vehicle based on the image information of the occupant.

15B, when the control unit 180 of the mobile terminal 100 determines the dangerous driving state for the driver D, the control unit 180 transmits the dangerous driving state information to the vehicle control device (for example, 400, and the vehicle control device 400 recognizes the presence or absence of the occupant and recognizes that the occupant is in the driver's seat, the driver's seat, and the rear-right seat. Accordingly, the vehicle control apparatus 400 can output the dangerous driving state to the display units D100 and D102 corresponding to the boarding position.

The passenger P1 and / or the passenger P2, who have determined that the driver D is in the dangerous driving state, can take action against the driver D in the dangerous driving state in accordance with the dangerous driving state information outputted on the display portions of the driving wheels D100 and D102 have.

At this time, the other terminal having received the dangerous driving state information can output the dangerous driving state information, but the output unit of the other terminal is able to determine the type of dangerous driving state, the degree of danger level, whether the other terminal is worn Whether or not the other terminal is worn by the driver, and the boarding position of the occupant wearing the other terminal.

Reference numeral C in FIG. 15B is an example in which one image acquisition device 900 indicates a position for acquiring an image for all the seats in the vehicle.

When the control unit 180 receives the input of the automatic mode M1 from the user, the control unit 180 recognizes, based on the image information about the occupant, the other terminal corresponding to the occupant recognized around the mobile terminal 100 The dangerous operation state information can be transmitted to the other terminal.

At this time, at least one of the other terminals receiving the dangerous driving status information or the display unit of the vehicle control apparatus displaying the dangerous driving status information may be selected and input through the user input unit 130 according to one embodiment .

As shown in FIG. 16, the output unit 150 may include an output unit 150 for outputting the dangerous operation state information through the output unit 150, (P1 to P4) representing a predetermined area, and outputs them.

At this time, only the display part of the vehicle control device corresponding to the driver's seat and the passenger seat corresponding to S1 and S2 selected by the user input and corresponding to S1 and S2, or the driver and the passenger on the driver's seat and passenger seat, May transmit the dangerous driving condition information.

The output unit 150 may output the graphic object to the passenger seat when the passenger is on the passenger or the occupant's terminal. For example, as shown in Fig. 16, when a passenger seat is displayed as a predetermined area, the area for the driver's seat, the front passenger's seat and the rear-right seat recognized as being occupied by a passenger or occupants wearing a terminal, Or a predetermined graphic object may be added to an area recognized as the presence of the occupant and displayed.

When receiving the input of the personal mode (M2) from the user, the control unit 180 can output the dangerous driving state to the outside through the output unit 150 of the mobile terminal 100 itself which has detected the dangerous driving state.

Upon receiving the personal mode input, the control unit 180 can output the dangerous driving state through the output unit 150 without transmitting the dangerous driving state information to the other terminal when the dangerous driving state is determined.

In the case of the mobile terminal 100 receiving the personal mode input, the output unit 150 preferably outputs the dangerous state information by vibration or a screen rather than outputting the dangerous state information by voice or sound.

However, even when the personal mode input is received, the control unit 180 compulsorily selects the automatic mode M1, the open mode M3, or the custom mode M4, if the risk level of the dangerous driving state is equal to or higher than the predetermined level And transmits the dangerous driving state information to another peripheral terminal, and the other terminal outputs the dangerous driving state information to the outside.

In this case, at least one of the other terminals receiving the dangerous driving status information or the display unit of the vehicle control apparatus displaying dangerous driving status information according to one embodiment may receive selection input through the user input unit 130 have.

As shown in FIG. 16, the output unit 150 may include an output unit 150 for outputting the dangerous operation state information through the output unit 150, (P1 to P4) representing a predetermined area, and outputs them.

At this time, only the display part of the vehicle control device corresponding to the driver's seat and the passenger seat corresponding to S1 and S2 selected by the user input and corresponding to S1 and S2, or the driver and the passenger on the driver's seat and passenger seat, May transmit the dangerous driving condition information.

The output unit 150 may output the graphic object to the passenger seat when the passenger is on the passenger or the occupant's terminal. For example, as shown in Fig. 16, when a passenger seat is displayed as a predetermined area, the area for the driver's seat, the front passenger's seat and the rear-right seat recognized as being occupied by a passenger or occupants wearing a terminal, Or a predetermined graphic object may be added to an area recognized as the presence of the occupant and displayed.

On the other hand, the other terminal having received the dangerous driving state information can output the dangerous driving state information, but the output unit of the other terminal determines whether the dangerous driving state, the danger level, And an in-vehicle position of the other terminal.

When the mobile terminal 100 receiving the input of the open mode (M3) from the user determines the dangerous driving state, as shown in FIG. 15C, the other terminals (the passengers P1 to P3 And transmits the dangerous driving state information to the worn mobile terminals 101 to 103. Upon receiving the dangerous driving state information, the other terminal can output the dangerous driving state through the output unit.

When the control unit 180 receiving the open mode input determines the dangerous driving state, the communication unit 110 may broadcast the dangerous driving state information to all terminals capable of transmitting the dangerous driving state information.

At this time, the counterpart terminal that transmits the dangerous driving state may transmit the dangerous operation state to the terminal list stored in the memory 160 of the memory 160 or the memory 440 of the vehicle control apparatus 400, in addition to the other terminals recognized around the mobile terminal 100 The operation state information can be transmitted.

In addition, the memory 160 of the memory 160 or the memory 440 of the vehicle control unit 400 may store a list of the terminals that have denied the dangerous driving status information. In this case, The operation state information may not be transmitted.

According to another aspect of the present invention, the other terminal may be configured not to receive the dangerous operation state information, and in this case, the other terminal may not output the dangerous operation state information even though the dangerous operation state information is received.

Meanwhile, when the mobile terminal 100 receiving the open mode input determines the dangerous driving state, the dangerous driving state information may be transmitted to the remote terminal through the communication unit 110.

Accordingly, the user, who has received the signal through the remote terminal, makes a call to the mobile terminal of the driver, the mobile terminal or the vehicle control device of the passenger who has determined the dangerous driving state, or transmits a message or a control command, It is possible to cause the driver to be notified directly or indirectly through the passenger. In addition, the other user who has received the dangerous driving condition information of the driver from the remote may connect to the vehicle control device 400 and control the vehicle to move the vehicle directly to a nearby safe place through the vehicle driving unit 420. [

According to one embodiment, the remote terminal may be limited to a terminal located within a predetermined distance range larger than the size of the vehicle around the location of the mobile terminal 100 of the driver, A driver of another vehicle located in the vicinity of the vehicle can receive the dangerous driving condition information and sound a horn outside the vehicle or blink an emergency light so that the driver in a dangerous driving state can be alerted.

When the control unit 180 receiving the input of the open mode (M3) from the user determines the dangerous driving state, the dangerous driving state information can be received from the user.

As shown in FIG. 16, the output unit 150 may include an output unit 150 for outputting the dangerous operation state information through the output unit 150, (P1 to P4) representing a predetermined area, and outputs them.

At this time, only the display part of the vehicle control device corresponding to the driver's seat and the passenger seat corresponding to S1 and S2 selected by the user input and corresponding to S1 and S2, or the driver and the passenger on the driver's seat and passenger seat, May transmit the dangerous driving condition information.

The output unit 150 may output the graphic object to the passenger seat when the passenger is on the passenger or the occupant's terminal. For example, as shown in Fig. 16, when a passenger seat is displayed as a predetermined area, the area for the driver's seat, the front passenger's seat and the rear-right seat recognized as being occupied by a passenger or occupants wearing a terminal, Or a predetermined graphic object may be added to an area recognized as the presence of the occupant and displayed.

Also, in order to select a terminal for transmitting the dangerous driving status information among the terminals located at a remote location, the terminal receives an input for selecting at least a portion of the address book stored in the memory 160, And transmit the dangerous driving condition information.

On the other hand, the other terminal having received the dangerous driving state information can output the dangerous driving state information, but the output unit of the other terminal determines whether the dangerous driving state, the danger level, And an in-vehicle position of the other terminal.

When the mobile terminal 100 receives the custom mode M4 input from the user, the mobile terminal 100 transmits the dangerous operation status information to another terminal that has been previously set using the communication unit 110, The dangerous driving state can be output to the outside through the load.

According to an embodiment, the controller 180 may receive information about an arbitrary terminal from a user and store the received information in the memory 160. Alternatively, the controller 180 may transmit dangerous operation state information over a predetermined number of times And may receive an input for selecting at least one terminal to transmit the dangerous driving status information from a list storing terminal information such as an address book, and may store it in the memory 160. [

The control unit 180 stores the information on the terminal arbitrarily inputted into the memory 160 so as to transmit the dangerous driving state information to any terminal stored in the memory 160 when the dangerous driving state is determined, So that any terminal that has received the state information can output the dangerous operation state information.

At this time, any terminal stored in the memory 160 may be irrelevant to the type, position, or whether the terminal is worn by the human body.

On the other hand, when the control unit 180 receiving the custom mode M4 input from the user determines the dangerous driving state, the dangerous driving state information can be received from the user.

As shown in FIG. 16, the output unit 150 may include an output unit 150 for outputting the dangerous operation state information through the output unit 150, (P1 to P4) representing a predetermined area, and outputs them.

At this time, only the display part of the vehicle control device corresponding to the driver's seat and the passenger seat corresponding to S1 and S2 selected by the user input and corresponding to S1 and S2, or the driver and the passenger on the driver's seat and passenger seat, May transmit the dangerous driving condition information.

The output unit 150 may output the graphic object to the passenger seat when the passenger is on the passenger or the occupant's terminal. For example, as shown in Fig. 16, when a passenger seat is displayed as a predetermined area, the area for the driver's seat, the front passenger's seat and the rear-right seat recognized as being occupied by a passenger or occupants wearing a terminal, Or a predetermined graphic object may be added to an area recognized as the presence of the occupant and displayed.

On the other hand, the other terminal having received the dangerous driving state information can output the dangerous driving state information, but the output unit of the other terminal determines whether the dangerous driving state, the danger level, And an in-vehicle position of the other terminal.

A vehicle control device for judging a driver's carelessness

Hereinafter, the vehicle control device for determining a carelessness of a driver according to an embodiment disclosed herein will be described in detail, and performs the same or similar function as the mobile terminal that determines the carelessness of the driver described above. And will not be described in detail.

The vehicle control apparatus according to one embodiment of the present invention includes a communication unit 431, an image information acquisition unit 470, and a control unit 410.

The communication unit 431 corresponds to the short-range communication module 431 described with reference to FIG. 6A, and overlaps with the description of the communication module 431 on the grounds thereof, so that a detailed description thereof will be omitted.

In addition, the image information acquisition unit 470 acquires image information about the user from the image acquisition device 900.

The control unit 410 controls the communication unit 431 to receive the biometric information for the user or acquire the image information through the image information acquisition unit 470. The control unit 410 may receive at least one of the image information and the biometric information And detects a dangerous driving state for the user based on one.

Here, the dangerous driving state for the user may include at least one of a drowsiness driving state, a driver distraction driving state, and a stress state for the user.

The control unit 410 may determine which of the drowsy driving state, the visually dispersed state, and the stress state the user is in, or may generate the danger level information for each of the dangerous driving states.

In order to calculate the dangerous driving state or the danger level information of the user, the control unit 410 applies a weight to a plurality of determination factors when determining based on the image information and / or the biometric information.

In this case, in order to improve the accuracy of the determination of the dangerous driving state or the danger level information, the weights for the plurality of determination factors may be weights determined by learning based on a large amount of databases.

At this time, the learning model can be generated based on the database of the image information and the biometric information of the general user including the user or the user by the control unit 180, and the risk driving state or the risk level is determined based on the generated learning model .

In addition, when the control unit 410 calculates the risk level information for the dangerous driving state or the dangerous driving state for the user, the control unit 410 uses the image information and the biometric information, and applies a weight to each of the image information and the biometric information Thus, it is possible to improve the accuracy of calculating the dangerous driving state or the dangerous level information of the control unit 410.

At this time, different values may be applied depending on whether the weight is in a drowsy driving state, a visually dispersed driving state, or a stress state.

On the other hand, when the control unit 410 determines the drowsiness operation state (including the stress state) of the user or the drowsiness operation level (including the stress level) of the user, it is necessary to consider both the image information and the biometric information Preferably, more preferably, the drowsiness trend line calculated can be further considered by using at least one of the sleep state information and the biorhythm information for the user before boarding the vehicle (see Fig. 13).

That is, when the control unit 410 determines the sleeping operation level (including the stress level) of the user, the first sleep operation level (including the first stress level) calculated based on the image information and the biometric information, (Including the stress level) corresponding to the sleepiness level calculated on the basis of the sleepiness level of the user (including the second stress level).

By applying a predetermined weight to each of the first drowsiness operation level (including the first stress level) and the second drowsiness operation level (including the second stress level), the user's drowsiness level (including the stress level) Can be calculated.

At this time, the weight applied to the first drowsiness operation level (including the first stress level) and the second drowsiness operation level (including the second stress level) may be set in advance by user input, It can be adaptively changed according to the confirmation input.

Meanwhile, the output unit 450 may output the dangerous driving state determined by the controller 410 or the danger level information thereof in various forms (various types that can stimulate the user's five senses).

The output unit 450 may output the danger level information in the form of image or voice and the output unit 450 of the vehicle control apparatus changes the output type or output intensity as the danger level information changes .

When the output unit 450 of the vehicle control apparatus outputs sound or sound as described above, the dangerous driving state or the danger level information is changed to output different images, , The amplitude and / or the output period, and the like.

Also, the control unit 410 may output the danger level information through a vehicle output unit (not shown) installed in the vehicle. Similarly to the output unit 450 of the vehicle control device, the vehicle output unit (not shown) may output the output mode or output intensity to the outside as the dangerous driving state or the dangerous level information changes.

At this time, the vehicle output portion may be at least one of an emergency light, an instrument panel, a perimeter of the vehicle, or at least one LED installed for each seat, a vibrating seat of the vehicle, and a vibration steerable device of the vehicle. The output period of the emergency light, the color of the LED, the output period of the LED, the vibration frequency of the vibration sheet or the vibration-capable steering apparatus, the vibration amplitude of the vibration sheet or the vibration-capable steering apparatus, At least one of the oscillation output periods of the apparatus may be outputted in different manners.

In addition, the vehicle control apparatus 400 can directly open at least one window installed in the vehicle for ventilation inside the vehicle, or operate the air conditioner of the vehicle to lower the dangerous driving level of the user.

Further, when the danger level is higher than the predetermined level, it is preferable that the vehicle control device 400 directly controls the vehicle operation.

After at least one of the mobile terminal 100, the vehicle control device 400, and the vehicle output unit has performed the above-described action with respect to the dangerous level with respect to the dangerous driving condition, or after a predetermined time has elapsed, , It is desirable to continuously repeat the determination of the risk level and the action accordingly.

The dangerous driving state and the danger level information of the dangerous driving state determined by the controller 410 may be transmitted to the mobile terminal 100 so that the mobile terminal 100 can transmit the dangerous driving state and the danger level information to the mobile terminal 100 through the output unit 150, The output intensity can be changed as the danger level information changes.

12, the mobile terminal 100 is provided with an alarm AS20 that increases the frequency of the vibration as the danger level increases, based on the vibration type alarm AS10 having the basic amplitude and the fundamental frequency, , An alarm AS30 with an increased amplitude of vibration, or a shorter output period of the vibration can be output.

The mobile terminal 100 further includes an alarm AS40 in which the frequency or amplitude of the vibration varies with time based on the vibration level alarm AS10 having the basic amplitude and the fundamental frequency And output it to the outside.

As described above, according to another embodiment, it is possible to output a sound (for example, "current drowsiness driving state is level 5") or a sound according to frequency, The amplitude and / or the output period, and the like.

The mobile terminal 100 outputs sounds according to the danger level information, but the types of sounds output according to the dangerous driving states may be different.

Meanwhile, when the dangerous driving state information determined by the vehicle control device 400 is output to the outside via the output unit 150 of the mobile terminal 100, And can receive a mode selection input for the terminal 100. FIG.

The dangerous driving state information may be transmitted to another terminal according to the automatic mode M1, the personal mode M2, the open mode M3, or the custom mode M4 according to an embodiment disclosed herein. Since the detailed description of each mode is the same as that described above, the redundant contents are replaced with the detailed description, and a detailed description will be omitted.

A control method of a mobile terminal or a vehicle control device

Hereinafter, a method of controlling a mobile terminal according to an embodiment of the present invention will be described in detail. Hereinafter, the mobile terminal will be mainly described, but it goes without saying that the present invention can be similarly applied to a vehicle control apparatus.

The contents overlapping with those of the above-described mobile terminal or the vehicle control device are replaced with the detailed description.

As shown in FIG. 18, the control method of a mobile terminal according to an exemplary embodiment of the present invention includes steps of receiving a mode selection input (S10), determining a dangerous driving state (S30) (S50) of transmitting the driving state information to the other terminal (S60) and outputting the dangerous driving state information by the other terminal (S60).

The step S10 of receiving the mode selection input may receive the mode selection input so that the mobile terminal 100 selects an object to which the dangerous driving state for the user (preferably the driver) is to be transmitted.

At this time, the mobile terminal 100 receives, from the user through the user input unit 130, an automatic mode M1, a personal mode M2, a public mode M3, and a custom mode M4 At least one can receive all of the selection inputs.

Then, the mobile terminal 100 acquires biometric information from a specific body part of a user (preferably a driver) (S21), and the mobile terminal 100 or the vehicle control device 400 acquires biometric information based on the biometric information (S30).

At this time, the dangerous driving state can be determined based on the image information acquired through the image acquisition device 900 installed toward the user (preferably the driver) inside and outside the vehicle and the biometric information.

The mobile terminal 100 or the vehicle control apparatus 400 may control at least one of a drowsiness operation state, a driver distraction operation state and a stress state for the driver based on the image information and the biometric information It is possible to accurately determine the dangerous driving condition and the level of the risk level.

The mobile terminal 100 or the vehicle control device 400 that has determined the dangerous driving state may transmit the dangerous driving state information to another terminal corresponding to the mode selection input received through the user input unit 130 (S50).

For example, the control unit 180 of the mobile terminal 100 determines whether there is an automatic mode selection input through the user input unit 130 (S41), whether there is a personal mode selection input (S42) Or whether there is a custom mode selection input (S44), and transmit the dangerous driving state information to the corresponding other terminal accordingly.

Accordingly, the other terminal having received the dangerous operation state information may output the dangerous operation state information (S60).

According to an embodiment of the present invention, the mobile terminal 100 or the vehicle control device 400 may receive a mode selection input from a user, .

At this time, the mobile terminal 100 or the vehicle control device 400 may output whether the passenger is boarding or wearing the occupant's terminal, so that the driver can intuitively select the other terminal.

The techniques disclosed herein may be applied to vehicles

Hereinafter, a vehicle to which the technique disclosed in this specification is applicable will be described with reference to Fig.

Although the electric vehicle is described in Fig. 17, the techniques disclosed herein may be applied to other types of vehicles (for example, a vehicle having a gasoline engine, a diesel engine, or an LPG engine) without departing from the technical idea disclosed in this specification Applicable to those skilled in the art.

17 is a schematic configuration diagram of an electric vehicle having a battery charging device according to an embodiment disclosed herein.

17, an electric vehicle 600 having a battery charging device according to an embodiment of the present invention includes a vehicle body 610, a battery 625 provided in the vehicle body 610, And a battery charging device 640 provided in the vehicle body 610 and connected to an external power source to charge the battery 625.

In the upper region of the vehicle body 610, although not shown, a passenger space on which the driver or the passenger can ride may be provided. For example, the vehicle body 610 may include a cabin (not shown) in which a boarding space is formed.

The vehicle body 610 may include a plurality of wheels 615 so that the vehicle body 610 can travel. A suspension 620 may be provided between the vehicle body 610 and the wheels 615, respectively. Accordingly, vibrations and shocks generated when the vehicle body 610 travels on the road surface can be alleviated.

The wheels 615 may be disposed on both the front and rear sides of the vehicle body 610, respectively.

The vehicle body 610 may be provided with a battery 625 for supplying power.

The battery 625 may be a rechargeable battery.

An electric motor 630 may be provided on one side of the vehicle body 610 to provide a driving force to the wheel 615.

The vehicle body 610 may be provided with an inverter device 635 for providing driving power to the electric motor 630. The inverter device 635 may be connected to the battery 625 and the motor 630, respectively.

The inverter unit 635 is connected to the battery 125 and receives DC power and converts the DC power to a power suitable for driving the motor 630 and provides the power to the motor 130.

The vehicle body 610 may be provided with a battery charging device 640 to charge the battery 625. The battery charging device 640 includes a charging circuit 660 connected to an external commercial power supply AC to convert commercial power into a power suitable for charging the battery 625 and providing the power to the battery 625, . The charging circuit 660 includes a commercial power input unit connected to a commercial power source and receiving commercial power, a rectifying unit converting a commercial power input through the commercial power input unit to DC, And a power conversion unit converting the converted DC power into a power suitable for charging the battery 625 and outputting the converted power.

The scope of the present invention is not limited to the embodiments disclosed herein, and the present invention can be modified, changed, or improved in various forms within the scope of the present invention and the claims.

100: mobile terminal 110:
130: user input unit 145: biometric information obtaining unit
150: output unit 160: memory
180: control unit 400: vehicle control device
410: control unit 431:
420: vehicle drive unit 450: output unit
470: Image information acquisition unit 900: Image acquisition device

Claims (20)

A body formed to be wearable on a specific body part of the user;
A communication unit configured to enable wireless communication with another terminal;
A user input unit for receiving a selection input of a mode for determining a transmission target of the dangerous driving state of the user among the other terminals;
A biometric information obtaining unit for obtaining biometric information for the user from the specific body part; And
And a control unit for detecting a dangerous driving state for the user based on the biometric information and forming a control command for outputting the dangerous driving state information of the user from the other terminal determined based on the selected mode through the user input unit and,
Wherein,
And a controller for recognizing the passenger of the user from the image information of the occupant in the vehicle obtained by the image capturing apparatus provided in the vehicle on which the user is boarding when the automatic mode is selected by the user input unit, And controls the communication unit such that the control command is transmitted to another terminal corresponding to the passenger among other terminals available. The method according to claim 1,
Wherein the type of the other terminal capable of outputting the dangerous driving condition information of the user is at least one of a wearable type device, a bar type mobile terminal, a vehicle control device, The mobile terminal comprising: The method according to claim 1,
In the dangerous driving state,
Wherein the user is at least one of a drowsy driving state and a stress state of the user. The method according to claim 1,
Wherein,
Receiving the image information from the image acquiring device,
Wherein,
And detects the dangerous driving state based on at least one of the image information and the biometric information. 5. The method of claim 4,
Wherein,
Wherein the mobile terminal determines whether the dangerous driving state is a drowsy driving state, a visual line dispersion driving state, or a stressed state based on at least one of the image information and the biometric information. The method according to claim 1,
Wherein,
And generates risk level information indicating a risk level with respect to the dangerous driving condition. delete delete The method according to claim 1,
The other terminal corresponding to the passenger,
And the at least one terminal is selected through the user input unit of the other terminal. The method according to claim 1,
Wherein,
Wherein the control unit controls the communication unit such that the control command is transmitted to another terminal of the other terminal corresponding to the passenger within a predetermined distance from the mobile terminal. A body formed to be wearable on a specific body part of the user;
A communication unit configured to be capable of wireless communication with a vehicle control apparatus having a plurality of display units;
A biometric information obtaining unit for obtaining biometric information for the user from the specific body part; And
A control unit for detecting a dangerous driving state for the user based on the biometric information and transmitting a control command for outputting dangerous driving state information of the user from at least one of the plurality of display units to the vehicle control device, And a control unit for controlling the control unit,
Wherein,
The control unit recognizes the position of the boarding seat in which the occupant is present among the boarding seats provided in the vehicle from the image information of the occupant in the vehicle obtained by the image capturing apparatus provided in the vehicle, Wherein the controller forms the control command so that the dangerous driving state information of the user is output from the display unit corresponding to the position of the recognized boarding seat. The method according to claim 1,
Further comprising an output section,
Wherein,
When the personal mode is selected by the user input unit,
Wherein the control unit controls the output unit so that the dangerous operation state information of the user is output to the outside of the mobile terminal. The method according to claim 1,
Wherein,
If the disclosure mode is selected by the user input,
The dangerous operation state information of the user is output from the control device of the vehicle, the other terminal located within a predetermined distance from the mobile terminal, and the other terminal registered in the mobile terminal or the control device of the vehicle, And transmits the dangerous driving condition information of the user to the control device of the vehicle, another terminal located within a predetermined distance from the mobile terminal, and the other terminal registered in advance, using the communication unit. 14. The method of claim 13,
The other terminal located within the predetermined distance and the other terminal registered in advance,
Wherein the at least one terminal is selected through the user input unit. The method according to claim 1,
Wherein,
When the custom mode is selected by the user input unit,
And the dangerous driving state information of the user is transmitted to another terminal set in the mobile terminal by using the communication unit so that the dangerous driving state information of the user is outputted to another terminal set in the mobile terminal among the other terminals . 16. The method of claim 15,
The other terminal, which is preset in the mobile terminal,
Wherein the at least one terminal is selected through the user input unit. 17. The method according to any one of claims 9 to 16,
And an output unit.
Wherein,
When receiving a selection input for the at least one terminal through the user input unit,
Wherein the control unit controls the output unit so that information on at least one of whether a passenger is boarding the passenger and whether the passenger is wearing the terminal is output together with the graphical object for the boarding seat provided in the vehicle. 16. A method according to any one of claims 12, 13 and 15,
The dangerous driving condition information of the user may include:
Wherein the mobile terminal is differently output according to at least one of the type of the dangerous driving state, the level of the danger level, whether the mobile terminal and the other terminal are located in the vehicle, and the vehicle position of the mobile terminal and the other terminal. . 19. The method of claim 18,
The vibration frequency, the amplitude, the cycle, the type of the voice or sound, the content, the frequency, the amplitude, the frequency, and the like of the main body according to the type of the dangerous driving state or the level of the dangerous level when the dangerous driving state information of the user is outputted from the mobile terminal, At least one of the periods is changed and output,
When the dangerous driving condition information of the user is outputted from the control device of the vehicle,
An output period of the LED, an oscillation frequency of the vehicle vibration sheet, a vibration amplitude of the vehicle vibration sheet, and an output period of the vehicle vibration sheet, wherein the output period of the vehicle emergency light, the LEDs of the plurality of LEDs arranged inside and outside the vehicle, Wherein at least one of a vibration period, a vibration frequency of the vibration-steerable device of the vehicle, a vibration amplitude of the steering device, and a vibration period of the steering device is changed and output. A method of controlling a mobile terminal that is worn on a specific body part of a user,
Receiving a selection input of a mode for determining a transmission target of a dangerous driving state for the user among other terminals;
Obtaining biometric information for the user from the specific body part;
Detecting the dangerous driving condition based on the biometric information;
If the mode is selected as the automatic mode, recognizing the passenger of the user from the image information of the occupant in the vehicle obtained by the image acquisition device provided in the vehicle; And
And transmitting a control command for outputting the dangerous driving state information of the user to another terminal corresponding to the passenger of the other terminal.

Images