KR102043145B1 - Mobile terminal and control method for the mobile terminal - Google Patents

Abstract

The present invention relates to a mobile terminal including a flexible display unit and a control method thereof. A mobile terminal according to an embodiment of the present invention is a flexible display that can be bent to cover at least one region of the wrist, disposed on one surface of the flexible display, a sensing unit for sensing the movement of the tendons passing through the wrist and the tendon And a control unit for generating a control signal for executing a pre-matched function according to the movement characteristics of the children.

Description

MOBILE TERMINAL AND CONTROL METHOD FOR THE MOBILE TERMINAL}

The present invention relates to a mobile terminal including a flexible display unit and a control method thereof.

Terminals may be divided into mobile / portable terminals and stationary terminals according to their mobility. The mobile terminal may be further classified into a handheld terminal and a vehicle mount terminal according to whether a user can directly carry it.

As the functions are diversified, for example, the terminal is implemented in the form of a multimedia player having complex functions such as taking a picture or a video, playing a music or a video file, playing a game, and receiving a broadcast. . In order to support and increase the function of such a terminal, it may be considered to improve the structural part and / or the software part of the terminal.

In general, terminals have evolved into various types of designs, and accordingly, flexible displays have attracted attention due to their light and unbreakable characteristics. Flexible displays may allow for the creation of new user interface areas that are limited or impossible to apply with existing glass substrate based displays.

In addition, as the flexible display attracts attention, the necessity of a user-friendly user interface using the characteristics of the flexible display is emerging.

An object of the present invention relates to a mobile terminal capable of generating a control command based on the movement of tendons across a user's wrist using the characteristics of a flexible display capable of bending or bending, and a control method thereof.

A mobile terminal according to an embodiment of the present invention is a flexible display that can be bent to cover at least one region of the wrist, disposed on one surface of the flexible display, a sensing unit for sensing the movement of the tendons passing through the wrist and the tendon And a control unit for generating a control signal for executing a pre-matched function according to the movement characteristics of the children.

In one embodiment, the control unit generates a control signal for executing different functions corresponding to different movement characteristics of the tendons, respectively, wherein the movement characteristics of the tendons are the shape of the finger, the speed of movement of the finger, the wrist And at least one of a movement speed and a grip force change.

According to an embodiment of the present disclosure, the control unit executes a function corresponding to a movement characteristic of tendons sensed through the sensing unit only when a preset starting gesture is sensed.

According to an embodiment of the present disclosure, based on the sensing of the preset starting gesture, guide information related to a function matched with movement characteristics of different tendons is displayed on the flexible display unit.

In an embodiment, the controller executes a first function when the shape of the finger corresponds to the first gesture, and executes a second function when the shape of the finger corresponds to the second gesture. The first function is a camera function, and the second function is a music execution function.

According to an embodiment of the present disclosure, when the camera function is executed based on the sensing of the first gesture, the controller may provide a control signal for controlling a function related to the camera based on a shape change of the finger. It is characterized by generating.

In an embodiment, the controller estimates a distance between the first finger and the second finger based on the movement of the tendons, and zooms in or zooms out according to the estimated distance. -out) to control the function.

In one embodiment, the camera includes a plurality of cameras disposed at different positions, and the controller controls at least one of the plurality of cameras so that an activated camera among the plurality of cameras is changed according to a direction that the wrist faces. It is characterized by.

The display device may include a camera sensor, and the controller may control a position of a display area in which information is displayed on the flexible display unit based on a user's gaze direction sensed by the camera sensor. do.

The control unit may move the information displayed on the display area based on the movement of the wrist sensed by the sensing unit.

The control unit may select at least one of at least one selection target item included in the information displayed on the display area, based on the movement of the tendon that changes in response to the movement of the finger. .

In an exemplary embodiment, a graphic object whose position is changed according to the movement of the finger is displayed on the display area.

In an embodiment, at least one of the selection target items may be selected based on sensing movement of tendons corresponding to a collision between at least two fingers of the plurality of fingers.

The control unit may determine a position at which the flexible display unit is wrapped on the wrist based on an area where the sensing unit and the wrist contact each other, and the size of the display area in which information is displayed according to the determined position. It characterized in that to control.

 In one embodiment, the sensitivity of the sensing unit for sensing the movement of the tendon is characterized in that it is controlled differently depending on the position in which the flexible display unit is wrapped.

According to another embodiment of the present invention, a mobile terminal and a control method thereof may sense movement of a tendon passing through a wrist by using a flexible display unit surrounding the wrist. Therefore, the user can execute a matched function according to the movement characteristics of the tendon just by moving a wrist or a finger even without applying a separate control command through a touch input or the like on the flexible display unit. That is, the mobile terminal and its control method according to an embodiment of the present invention can increase user convenience by executing various functions using only one hand instead of two hands.

1 is a block diagram illustrating a mobile terminal according to an embodiment of the present invention.
2A, 2B, and 2C are conceptual views illustrating a flexible display unit included in a mobile terminal according to an embodiment of the present invention.
3 is a flowchart illustrating a control method of generating a control signal based on movement characteristic information of tendons in a mobile terminal according to an embodiment of the present invention.
4A and 4B are conceptual views illustrating the control method of FIG. 3 in a mobile terminal according to an embodiment of the present invention.
5A and 5B are conceptual views illustrating a method of receiving a preparation operation in a mobile terminal according to an embodiment of the present invention.
6A, 6B, 6C, 7A, 7B, 8A, and 8B are conceptual views illustrating a method of generating various control signals according to different movement characteristics in a mobile terminal according to an embodiment of the present invention. to be.
9A and 9B are conceptual views illustrating a method of performing a security-related function based on a movement characteristic of a tendon in a mobile terminal according to an embodiment of the present invention.
10A and 10B are conceptual views illustrating a method of displaying information on a flexible display unit in a mobile terminal according to an embodiment of the present invention.
11 is a conceptual diagram illustrating a method of setting a function corresponding to a movement characteristic in a mobile terminal according to an embodiment of the present invention.
12 is a conceptual view illustrating a method of performing a function using a motion characteristic in a mobile terminal according to an embodiment of the present invention.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, and the same or similar components are denoted by the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used in consideration of ease of specification, and do not have distinct meanings or roles from each other. In addition, in describing the embodiments disclosed herein, when it is determined that the detailed description of the related known technology may obscure the gist of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easily understanding the embodiments disclosed in the present specification and are not to be construed as limiting the technical spirit disclosed in the present specification by the accompanying drawings.

The mobile terminal described herein includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, and a slate PC. , Tablet PC, Ultra Book, and so on. However, it will be readily apparent to those skilled in the art that the configuration according to the embodiments described herein may also be applied to fixed terminals such as digital TVs, desktop computers, etc., except when applicable only to mobile terminals.

1 is a block diagram illustrating a mobile terminal according to an exemplary embodiment disclosed herein.

The mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, and an interface. The unit 170, the controller 180, and the power supply unit 190 may be included. The components shown in FIG. 1 are not essential, so that a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and a network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless internet module 113, a short range communication module 114, a location information module 115, and the like. .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 may include, for example, Digital Multimedia Broadcasting-Terrestrial (DMB-T), Digital Multimedia Broadcasting-Satellite (DMB-S), Media Forward Link Only (MediaFLO), and Digital Video Broadcast (DVB-H). Digital broadcast signals can be received using digital broadcasting systems such as Handheld and Integrated Services Digital Broadcast-Terrestrial (ISDB-T). Of course, the broadcast receiving module 111 may be configured to be suitable for not only the above-described digital broadcasting system but also other broadcasting systems.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives a wireless signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The mobile communication module 112 is configured to implement a video call mode and a voice call mode. The video call mode refers to a state of making a call while viewing the other party's video, and the voice call mode refers to a state of making a call without viewing the other party's image. In order to implement the video call mode and the voice call mode, the mobile communication module 112 is configured to transmit and receive at least one of audio and video.

The wireless internet module 113 refers to a module for wireless internet access and may be embedded or external to the mobile terminal 100. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication module 114 refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, etc. may be used.

The location information module 115 is a module for obtaining a location of a mobile terminal, and a representative example thereof is a GPS (Global Position System) module.

Referring to FIG. 1, the A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in a video call mode or a photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras may be provided according to the use environment.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. The processed voice data may be converted into a form transmittable to the mobile communication base station through the mobile communication module 112 and output in the call mode. The microphone 122 may implement various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for the user to control the operation of the terminal. The user input unit 130 may include a key pad, a dome switch, a touch pad (constant voltage / capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 140 detects a current state of the mobile terminal 100 such as an open / closed state of the mobile terminal 100, a location of the mobile terminal 100, presence or absence of a user contact, orientation of the mobile terminal, acceleration / deceleration of the mobile terminal, and the like. To generate a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to the external device may be sensed.

The output unit 150 is used to generate an output related to visual, auditory, or tactile, and may include a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154. have.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal is in a call mode, the mobile terminal displays a user interface (UI) or a graphic user interface (GUI) related to the call. When the mobile terminal 100 is in a video call mode or a photographing mode, the mobile terminal 100 displays a photographed and / or received image, UI, or GUI.

The display unit 151 may include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (flexible display). display, a 3D display, or an e-ink display.

Some of these displays can be configured to be transparent or light transmissive so that they can be seen from the outside. This may be referred to as a transparent display. A representative example of the transparent display is TOLED (Transparant OLED). The rear structure of the display unit 151 may also be configured as a light transmissive structure. With this structure, the user can see the object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

According to an implementation form of the mobile terminal 100, two or more display units 151 may exist. For example, a plurality of display units may be spaced apart or integrally disposed on one surface of the mobile terminal 100, or may be disposed on different surfaces, respectively.

When the display unit 151 and a sensor for detecting a touch operation (hereinafter, referred to as a touch sensor) form a mutual layer structure (hereinafter referred to as a touch screen), the display unit 151 may be used in addition to an output device. Can also be used as an input device. The touch sensor may have, for example, a form of a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display unit 151 or capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor may be configured to detect not only the position and area of the touch but also the pressure at the touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and then transmits the corresponding data to the controller 180. As a result, the controller 180 can know which area of the display unit 151 is touched.

Referring to FIG. 1, a proximity sensor may be disposed in an inner region of a mobile terminal surrounded by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without using a mechanical contact by using an electromagnetic force or infrared rays. Proximity sensors have a longer life and higher utilization than touch sensors.

Examples of the proximity sensor include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. When the touch screen is capacitive, the touch screen is configured to detect the proximity of the pointer by the change of the electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching the pointer on the screen is called "contact touch." The position where the proximity touch is performed by the pointer on the touch screen refers to a position where the pointer is perpendicular to the touch screen when the pointer is in proximity proximity.

The proximity sensor detects a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 152 may also output a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The sound output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying occurrence of an event of the mobile terminal 100. Examples of events occurring in the mobile terminal include call signal reception, message reception, key signal input, and touch input. The alarm unit 153 may output a signal for notifying occurrence of an event in a form other than a video signal or an audio signal, for example, vibration. The video signal or the audio signal may also be output through the display unit 151 or the sound output module 152, so that they 151 and 152 may be classified as part of the alarm unit 153.

The haptic module 154 generates various haptic effects that a user can feel. Vibration is a representative example of the haptic effect generated by the haptic module 154. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or may be sequentially output.

In addition to vibration, the haptic module 154 may be configured to provide a pin array that vertically moves with respect to the contact skin surface, a jetting force or suction force of air through the jetting or suction port, grazing to the skin surface, contact of the electrode, electrostatic force, and the like. Various tactile effects can be generated, such as effects by the endothermic and the reproduction of a sense of cold using the elements capable of endotherm or heat generation.

The haptic module 154 may not only deliver the haptic effect through direct contact, but also may implement the user to feel the haptic effect through a muscle sense such as a finger or an arm. Two or more haptic modules 154 may be provided according to configuration aspects of the mobile terminal 100.

The memory 160 may store a program for the operation of the controller 180 and may temporarily store input / output data (for example, a phone book, a message, a still image, a video, etc.). The memory 160 may store data regarding vibration and sound of various patterns output when a touch input on the touch screen is performed.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), RAM random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic It may include a storage medium of at least one type of disk, optical disk. The mobile terminal 100 may operate in connection with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device, receives power, transfers the power to each component inside the mobile terminal 100, or transmits data inside the mobile terminal 100 to an external device. For example, wired / wireless headset ports, external charger ports, wired / wireless data ports, memory card ports, ports for connecting devices with identification modules, audio input / output (I / O) ports, The video input / output (I / O) port, the earphone port, and the like may be included in the interface unit 170.

The identification module is a chip that stores various types of information for authenticating the use authority of the mobile terminal 100. The identification module includes a user identify module (UIM), a subscriber identify module (SIM), and a universal user authentication module ( universal subscriber identity module (USIM), and the like. A device equipped with an identification module (hereinafter referred to as an 'identification device') may be manufactured in the form of a smart card. Therefore, the identification device may be connected to the terminal 100 through a port.

The interface unit 170 may be a passage for supplying power from the cradle to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, or various commands input from the cradle by a user. It may be a passage through which a signal is transmitted to the mobile terminal. Various command signals or power input from the cradle may be operated as signals for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, perform related control and processing for voice calls, data communications, video calls, and the like. The controller 180 may include a multimedia module 182 for playing multimedia. The multimedia module 182 may be implemented in the controller 180 or may be implemented separately from the controller 180.

 In addition, the controller 180 may perform a pattern recognition process for recognizing a writing input or a drawing input performed on the touch screen as text and an image, respectively.

In addition, if the state of the mobile terminal satisfies a set condition, the controller 180 may execute a lock state for restricting input of a user's control command to applications. The controller 180 may control the lock screen displayed in the locked state based on a touch input detected through the display unit (hereinafter, referred to as a “touch screen” 151) in the locked state.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

Various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions. In some cases, the embodiments described herein may be implemented by the controller 180 itself.

According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

The software code may be implemented as a software application written in a suitable programming language. The software code may be stored in the memory 160 and executed by the controller 180.

Meanwhile, in the present invention, information processed by the mobile terminal 100 may be displayed by using a flexible display. Hereinafter, the flexible display unit will be described in more detail with reference to the accompanying drawings. 2A and 2B are conceptual views illustrating a flexible display unit included in a mobile terminal according to an embodiment of the present invention.

As shown in (a) and (b) of FIG. 2A, the flexible display unit 151 includes a bent, collapsible, rollable display that can be bent by an externally applied physical force. do. Here, the flexible display unit 151 may include both a general flexible display and an electronic paper.

Here, the general flexible display means a light and unbreakable, durable display that is fabricated on a thin flexible substrate that can be bent, bent, folded or rolled like a paper while maintaining the display characteristics of a conventional flat panel display.

In addition, electronic paper is a display technology to which the characteristics of general ink are applied, and the use of reflected light may be different from that of a general flat panel display. Electronic paper can change pictures or text using twisted balls or electrophoresis using capsules.

Meanwhile, a sensing unit 181 (see FIG. 1) is included on the flexible display unit 151 to detect bending information of the flexible display unit 151. In the present specification, the word of “bent” may include all of the meanings of “bent”, “curling”, “folding” and “bending”.

The sensing unit 181 may be disposed on the flexible display unit 151 as a whole or partially disposed, and may detect bending information of the flexible display unit 151. Here, the bending information of the flexible display unit 151 may include a bending direction, a bending degree, a bending position, a bending time, and an acceleration in which the flexible display unit returns to its original state. Due to the bending of the flexible display unit, various information may be sensed.

In addition, the controller 180 may change the information displayed on the flexible display unit 151 or control a function of the terminal based on the bending information of the flexible display unit detected by the sensing unit 181. Can be generated.

For example, as illustrated in FIGS. 2A and 2B, the controller 180 may bend the flexible display unit 151 when the flexible display unit 151 bends in response to an external physical force. According to the bending angle and the return acceleration to return to the original state, the screen image previously displayed on the flexible display unit 151 may be rearranged, separated, synthesized, or changed.

For example, as illustrated in FIGS. 2A and 2B, when the flexible display unit 151 is bent inward by an external physical force, the controller 180 may be disposed on the flexible display unit 151. The screen images displayed on the screen may be displayed to be close to each other. In addition, on the contrary, when the flexible display unit 151 is bent outward by an external physical force, the screen images displayed on the flexible display unit 151 may be displayed to be spaced apart from each other.

Also, in addition to the methods discussed above. In response to the bending of the flexible display unit, the controller 180 can variously control a method of displaying information on the flexible display unit 151 so that a user can accurately recognize the information displayed on the flexible display unit 151. have.

Meanwhile, the mobile terminal according to the present invention including the flexible display unit 151 may include a case 200 surrounding the flexible display unit 151 as illustrated in FIG. 2B. In addition, the case 200 may be configured to be bent together with the flexible display unit 151 by an external physical force in consideration of characteristics of the flexible display unit 151.

Meanwhile, as mentioned above, the controller 180 may generate a control signal related to a function of the mobile terminal according to the bending information of the flexible display unit.

In addition, in the mobile terminal according to the present invention, as shown in (a) of FIG. 2C, it is possible to implement the mobile terminal in the form of being wrapped around the wrist by using the flexible property of the flexible display unit 151. .

In addition, as illustrated in (b) of FIG. 2C, in the mobile terminal according to an embodiment of the present invention, the flexible display unit 151 places the sensing unit 140 on a surface in contact with the wrist, thereby passing the wrist. Movement of a plurality of tendons may be sensed through the sensing unit 140. In this case, the sensing unit 140 may serve as a finger motion detector. That is, the sensing unit 140 may sense the movement of tendons according to the movement of the finger, and transmit the sensed information to the controller 180. In addition, the controller 180 may process a function matched to the movement of the finger based on the information sensed by the sensing unit 140.

On the other hand, when looking at the movement of the tendon according to the movement of the finger, the carpal tunnel (or carpal tunnel, 410) inside the wrist 410, there is a tendon bundle of tendons (finger flexor tendons) that govern each finger movement do. There are nine tendons and one nerve in the tendon bundle, and when the finger moves, the nine tendons included in the tendon bundle move in various combinations. The sensing unit 140 senses the shape of the tendons deformed according to the movement of the finger or the wrist, and the controller 180 may determine what gestures the fingers are making based on the sensed information.

In this case, shape information of tendons matched with different finger gestures may be stored in the memory 160.

Meanwhile, the sensing unit 180 may sense not only the shape of tendons deformed according to the movement of the finger, but also the change of grip force and the speed of movement of the finger.

That is, the sensing unit 180 may sense the movement characteristics of the tendons, where the movement characteristics include a finger shape (or gesture), a finger movement speed, a wrist movement direction, a wrist movement speed, and a grip force change. It may be for at least one.

On the other hand, as described above, in the mobile terminal according to the present invention, by wrapping the wrist through the flexible display unit 151, it is possible to sense the movement and shape of the tendons through the sensing unit 140.

Hereinafter, a method of executing a function previously matched to the sensed movement characteristic based on the movement characteristic of the tendons sensed through the above-described method will be described in more detail with the accompanying drawings.

3 is a flowchart illustrating a control method of generating a control signal using a flexible display unit in a mobile terminal according to an embodiment of the present invention, and FIGS. 4A and 4B are conceptual views illustrating the control method of FIG. 3. .

First, the sensing unit 140 (see (b) of FIG. 1 and FIG. 2C) senses movement of tendons passing through the wrist (S310). The sensing unit 140 is disposed at a portion where the flexible display unit 151 is in contact with the wrist. In addition, the tendons that are the sensing targets of the sensing unit 140 are tendons included in the wrist oyster 410 described with reference to FIG. 2C (b).

As in the step S310, when the movement of the tendons by the sensing unit 310 is sensed, the controller 180 generates a control signal for executing a function previously matched to the movement characteristics of the sensed tendons (S320) .

As described above, the movement characteristics of the tendons may be at least one of a shape (or a gesture) of a finger, a movement speed of a finger, a movement direction of a wrist, a movement speed of a wrist, and a grip force change. In addition, the movement characteristic may be variously changed according to the movement of the finger or the wrist. For example, the movement characteristics when all five fingers are extended and when all five fingers are pinched are different. That is, this is because the movement and the shape of the tendons vary according to the movement of the finger.

That is, the controller 180 generates a control signal for executing different functions corresponding to different movement characteristics of the tendons, respectively. For example, when the shape (or gesture) of the finger corresponds to the preset first shape, the controller 180 generates a first control signal corresponding to the first function matched to the first shape, and the finger When the shape corresponds to the preset second shape, a second control signal corresponding to the second function matched with the second shape may be generated.

Here, the control signal for executing the 'pre-matched function' is a control signal for controlling the mobile terminal according to the present invention. Therefore, in the mobile terminal according to the present invention, a function according to a control signal generated based on the movement characteristics of the sensed tendons may be executed. Therefore, in the mobile terminal according to the present invention, only various motions of the tendons or the shape of the tendon may be detected. Also, a function matching each of the different finger movement characteristics may be set based on a user's selection or initial setting.

Looking at an example in which different functions are executed according to different finger shapes, as shown in FIGS. 4A and 4B, when the fingers take different shapes, FIG. 4A (b). And as illustrated in (b) of FIG. 4B, different functions are executed in the mobile terminal.

For example, as shown in (a) of FIG. 4A, when the first finger 421 and the fifth finger 425 are extended and the remaining second, third and fourth fingers are pinched, the controller 180 A control signal corresponding to the "call" (or call) related function matched to the illustrated finger shape (or gesture) is generated. As a result, as illustrated in (b) of FIG. 4A, an execution screen on which the “call” related function is executed may be output on the flexible display unit 151.

As another example, as shown in (a) of FIG. 4B, when the second finger 422 and the third finger 423 are extended and the remaining first, fourth and fifth fingers are pinched, the controller 180 ) Generates a control signal corresponding to the "camera" function matched to the illustrated finger shape (or gesture). As a result, as shown in (b) of FIG. 4B, the camera 121 or 121 ′ (see FIGS. 2A and 2B) may be activated, and the image input through the camera may be displayed on the flexible display unit 151. Can be output.

Meanwhile, in addition to the examples described with reference to FIGS. 4A and 4B, the controller 180 may execute a predetermined function based on a dynamic movement and a speed of a finger, a change in grip force, or a movement and a speed of a wrist.

As described above, the mobile terminal according to the present invention can execute different functions according to different finger gestures. Therefore, the user can execute a desired function only by the movement of a finger or a wrist even without applying a control command through a touch or the like on the flexible display unit.

Hereinafter, a method of executing a function corresponding to the movement of a finger or a wrist only when the starting gesture is sensed will be described in more detail with reference to the accompanying drawings. 5A and 5B are conceptual views illustrating a method of receiving a preparation operation in a mobile terminal according to an embodiment of the present invention.

In the mobile terminal according to the present invention, it is possible to prevent the control signal from being incorrectly generated when the user merely moves the finger or the hand, rather than the user moving the finger or the wrist to generate the control signal. That is, in the mobile terminal according to the present invention, the controller 180 may provide a function corresponding to the movement characteristic of the tendons detected by the sensing unit 180 only when a preset starting gesture is sensed. Can be run.

Here, the preset starting gesture may be based on the user's setting or correspond to initial setting information.

For example, the preset starting gesture is a gesture of moving the second finger 422 once in an arbitrary direction, and as shown in FIG. 5A (a), the movement of the second finger 422 is set in the preset direction. When moving to correspond to the starting gesture, the controller 180 may generate a control signal related to a function corresponding to the movement characteristic of the tendons after the starting gesture is sensed.

In addition, the controller 180 may output guide information on the flexible display unit 151 as illustrated in FIG. 5A (b) to inform the user that the starting gesture has been sensed. In addition, the guide information may be output through an auditory or tactile method such as voice or vibration, in addition to the visual method illustrated in FIG. 5A (b).

In addition, when a preset time elapses after the starting gesture is sensed or when a preset finish gesture is sensed, the controller 180 senses the movement of the tendons by the sensing unit 140. It may not be possible to implement a function corresponding to the movement of the tendons.

In addition, as shown in FIG. 5B, the controller 180 may be configured to match functions of different movement characteristics according to different movements of the tendons on the flexible display unit 151 based on the sensing of the starting gesture. Related guide information 210 may be output.

As described above, in the mobile terminal according to the present invention, only when the starting gesture is sensed, by executing a function corresponding to the movement characteristics of the tendons, it is possible to prevent unnecessary functions from being executed if the user does not intend it. It is possible.

Hereinafter, the method of controlling various functions according to the movement characteristics of the tendons and the direction of the wrist through the specific embodiments will be described in more detail. 6A, 6B, 6C, 7A, 7B, 8A, and 8B are conceptual views illustrating a method of generating various control signals according to different movement characteristics in a mobile terminal according to an embodiment of the present invention. to be.

As an example, a method of controlling a camera related function according to a movement characteristic of tendons while a camera function is activated will be described. When the movement of the tendons according to the movement of the finger is sensed in the mobile terminal according to the present invention, the controller 180 can estimate the distance between at least two fingers based on the sensed information.

As such, when the interval between the fingers is estimated, the controller 180 may generate a control signal based on the estimated interval information between the fingers.

For example, as illustrated in FIGS. 6A and 6B, the controller 180 may adjust the magnification of the camera based on different finger spacing. That is, the controller 180 may control a zoom-in or zoom-out function based on different finger spacing.

If the estimated distance between the first and second fingers 421 and 422 is less than or equal to the reference length, the controller 180 zooms out as shown in FIGS. 6A and 6B. -out) function can be executed. The controller 180 zooms in as shown in (a) and (b) of FIG. 6B when the estimated distance a between the first and second fingers 421 and 422 is greater than or equal to the reference length. (zoom-in) function can be executed. In addition, when the zoom-in or zoom-out function is executed, the controller 180 may control the magnification differently according to the distance between the fingers.

Meanwhile, the mobile terminal according to the present invention includes a plurality of cameras disposed at different positions, and the controller 180 includes at least one of the plurality of cameras so that the activated camera among the plurality of cameras varies according to a direction that the wrist faces. You can control one.

For example, a case in which the first camera 121 of the plurality of cameras is disposed on a surface corresponding to the back of the hand and the second camera is disposed on the surface corresponding to the palm of the hand will be described. At this time, the controller 180 may sense the direction of the back of the hand (or the palm) through a motion sensor (not shown) included in the sensor unit 140. Here, the motion sensor 141 is mounted inside the mobile terminal 100, and may recognize the movement or position of the mobile terminal 100. The motion sensor may include a geomagnetic sensor, an acceleration sensor, a gyro sensor, and the like.

When the back of the hand faces the user, the controller 180 activates the second camera 121 'when the back of the hand faces the user, and when the palm faces the user, the first camera 121. By activating, it is possible to receive different images through the camera.

In this case, the controller 180 may control an area in which information (eg, an image input through a camera) is displayed on the flexible display unit 151 according to the direction of the back of the hand or the palm of the user. . That is, the controller 180 may change an area where information is displayed on the flexible display unit 151 according to the user's gaze direction.

In addition, although not shown, the controller 180 may control various functions related to the camera based on the movement characteristics of the tendons according to the movement of the wrist or the finger. For example, various functions related to the camera, such as the number of shots continuously shot, a photographing command, brightness control, and contrast control, can be variously controlled according to a user's selection.

As another example, a method of selecting at least one of selection items displayed on the flexible display unit 151 based on the movement characteristics of the tendons will be described. For example, as illustrated in FIG. 7A, the controller 180 may display a graphic object (or a touch pointer 300) displayed on the flexible display unit 151 as the finger 422 moves 422a-> 422b. Can be moved.

And, as shown in (a) of FIG. 7B, the controller 180 selects an item 210 displayed where the touch pointer 300 is located based on the meeting of at least two fingers 241 and 242. The flexible display unit 151 may be controlled to be able to. As shown in (a) of FIG. 7B, when an item displayed where the touch pointer 300 is located is selected, as shown in (b) of FIG. 7B, the selected item is displayed on the flexible display unit 151. Corresponding information may be output.

As another example, a method of controlling a function in a music or video playback related application based on the movement of tendons according to the movement of a wrist or a finger is described.

For example, as shown in (a), (b) and (c) of FIG. 8A, when the wrist rotates in the first direction (a) and then returns to the original direction (b), the controller 180 ) May play a second song, the second song after the first song currently being played in the playlist.

In addition, the controller 180 may determine whether to play the next song or the previous song, according to the direction in which the wrist rotates.

In addition, as shown in (a) of FIG. 8B, in an audio or video related application, when the wrist is moved in a predetermined direction as shown in (b) of FIG. 8B while searching for a file currently being played at 1x speed, The search speed can be increased by three times.

As another example, as shown in FIG. 9A, when a lock device such as a car or a door lock is released through NFC (Near Field Communication) communication performed through the wireless communication unit 110, the controller 180 may be preset. Only when the movement characteristic of the tendons is sensed, a signal for releasing the locking device may be transmitted to the locking device.

Therefore, in such a case, the control unit may release the lock only when the arrangement of the user's own tendons is sensed or when the tendon movement of the preset pattern is sensed, so that the user may enhance the security of the lock.

As another example, as shown in (a), (b) and (c) of FIG. 9B, in the locked state limiting the control command of the application, the movement characteristic of the tendons preset through the sensing unit 140 is sensed. In this case, the controller 180 can switch the locked state to the released state.

Here, the locked state is a state that limits the input of the user's control command to the applications included in the mobile terminal. This prevents the user from inputting a control command that is not intended by a user in a mobile terminal having a touch screen (or a display unit or a flexible display unit 151) mainly made of a touch screen to activate or deactivate the functions and applications of the mobile terminal. It is to. Therefore, in the locked state, the input of the control command of the user input through the touch screen (or the display unit 151 and the other user input unit 130) may be limited within the set range.

On the other hand, in the locked state, the input of the control command of the user is limited, but the functions of the mobile terminal and the operations of the applications that were operating before the locked state can be continuously executed.

The release state is a state in which a user's input of a control command to the mobile terminal is not restricted. Therefore, in the released state, the functions and applications of the mobile terminal are activated or deactivated according to the control command input by the user through the touch screen (or the display unit 151 or the user input unit 130).

Here, the locked state may be executed when the user's input is not sensed for the time set for the touch screen (or display unit 151) and the other user input unit 130 provided in the mobile terminal. The set time may be changed according to a user's setting. In addition, the locked state may be executed when a user presses a special key (for example, a hold key) provided in the mobile terminal that is set in advance for the locked state.

As described above, in the mobile terminal according to the present invention, in addition to the method of switching the locked state to the released state through a touch on the flexible display unit, the user may change the locked state to the released state only by the movement of a wrist or a finger. Therefore, the user can release the locked state even when the use of both hands is impossible.

Meanwhile, in the mobile terminal according to the present invention, the controller 180 may control the size of the region where the information is displayed differently according to which point of the wrist of the flexible display unit 151 is located.

As described above, the flexible display unit 180 that bends to surround the wrist may display information on the entire area surrounding the wrist. However, in practice, the user may view only information displayed on a portion of the flexible display unit 180 according to the direction of the back of the hand (or the palm).

Accordingly, the controller 180 according to the present invention may recognize the direction in which the back of the hand is facing through the camera sensor or the above-described motion sensor, and control the area where the information is displayed according to the recognized direction. For example, when the back of the user's hand faces the user's eye, the controller 180 may display information on a region corresponding to the back of the hand on the flexible display unit 151.

Also, as illustrated in FIG. 10A, the controller 180 may control the size of an area where information is displayed according to the area where the flexible display unit 151 contacts the wrist. For example, the sensing unit 140 senses an area where the sensing unit 140 is in contact with the wrist. The controller 180 determines a position where the flexible display unit 151 is wrapped around the wrist based on the sensed information as described above. Then, the size of the area where the information is displayed is controlled according to the determined position.

Therefore, as shown in the figure, when the flexible display unit 151 is located closer to the user's body than the palm of the hand, the area where the flexible display unit 151 is in contact with the user's wrist increases, so that the controller 180 displays the flexible display unit 151. ) Controls the size of the information display area at position (A) to be larger than the size of the information display area at position (B).

Meanwhile, the controller 180 may control the sensing unit 140 so that the tendons at the position (A) are present inside the wrist than at the position (B), so that the sensitivity of the sensing unit 140 may be more sensitive. have.

Also, as illustrated in (a) and (b) of FIG. 10B, the controller 180 may change the information display area of the flexible display unit 151 when the information display area of the flexible display unit 151 is changed based on the user's gaze direction. Based on the size, the flexible display unit 151 may be controlled to vary the amount of information displayed.

As described above, in the mobile terminal according to the present invention, it is possible to efficiently provide information to the user by changing the location where the information is displayed according to the position of the flexible display unit and the user's gaze direction.

Meanwhile, in the mobile terminal according to the present invention, as illustrated in FIG. 11, a user may directly set a function corresponding to a specific gesture. For example, in order to match a gesture corresponding to a navigation function as illustrated in (b) of FIG. 11, when the navigation function is selected by the user, the controller 180 may control (c) of FIG. As shown in), the guide information for receiving a gesture to be matched with the navigation function is output. As shown in (c) and (d) of FIG. 11, when a specific gesture is recognized through the sensing unit 140, it may be matched with the navigation function.

In addition, although not shown, in the mobile terminal according to the present invention, in addition to a method of recognizing a specific gesture, it is possible to receive any one of pre-stored gesture items from the user based on a gesture directly taken by the user.

As described above, in the mobile terminal according to the present invention, the user may directly select or input a gesture corresponding to a specific function in the terminal, thereby establishing a user-specific setting environment.

On the other hand, in the mobile terminal according to the present invention, by moving the finger or wrist, the user can activate a specific mode, such as 'health mode' to help the user's health. The health mode may be activated based on a user's selection. Accordingly, as shown in FIG. 12, when the health mode is activated, information about calories to be consumed and gesture information corresponding to specific functions may be output, thereby inducing a user to move a finger or a wrist. .

As described above, the mobile terminal according to the present invention not only executes the function of the terminal through a gesture according to the movement of the user, but also improves the user's health.

In addition, according to an embodiment of the present invention, a mobile terminal and a control method thereof may sense movement of a tendon passing through a wrist by using a flexible display unit surrounding the wrist. Therefore, the user can execute a matched function according to the movement characteristics of the tendon just by moving a wrist or a finger even without applying a separate control command through a touch input or the like on the flexible display unit. That is, the mobile terminal and its control method according to an embodiment of the present invention can increase user convenience by executing various functions using only one hand instead of two hands.

In addition, according to one embodiment disclosed herein, the above-described method may be implemented as code that can be read by a processor in a medium in which a program is recorded. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and may be implemented in the form of a carrier wave (for example, transmission over the Internet). Include.

The above-described mobile terminal is not limited to the configuration and method of the above-described embodiments, but the embodiments may be configured by selectively combining all or some of the embodiments so that various modifications can be made. It may be.

Claims (15)

A flexible display unit capable of bending to surround at least one region of the wrist;
A sensing unit disposed on one surface of the flexible display unit to sense movement of tendons passing through the wrist;
Camera sensor; And
And a controller configured to generate a control signal for executing a function previously matched to the sensed movement characteristics of the tendons,
The control unit,
Determine a position at which the flexible display unit is wrapped on the wrist based on an area where the sensing unit is in contact with the wrist,
Controls the size of the display area where information is displayed according to the determined position,
Controlling the position of the display area on which the information is displayed on the flexible display unit based on the user's gaze direction sensed by the camera sensor,
Move the displayed information on the display area based on the movement of the wrist sensed by the sensing unit;
Selecting at least one of at least one selection item included in the information displayed on the display area based on the movement of the tendon that changes in response to the movement of the finger,
And a graphic object whose position is changed according to the movement of the finger on the display area. The method of claim 1, wherein the control unit
Generating control signals for executing different functions respectively corresponding to different movement characteristics of the tendons;
The movement characteristic of the tendons is related to at least one of the shape of the finger, the movement speed of the finger, the movement speed of the wrist and the grip force change. The method of claim 2, wherein the control unit
Only when a preset starting gesture is sensed, the mobile terminal executes a function corresponding to a movement characteristic of tendons sensed through the sensing unit. The method of claim 3,
And based on the sensing of the preset starting gesture, guide information related to a function matched to different movement characteristics according to different movements of the tendons is displayed on the flexible display unit. The method of claim 2, wherein the control unit
If the shape of the finger corresponds to the first gesture, the first function is executed; if the shape of the finger corresponds to the second gesture, the second function is executed;
And wherein the first function is a camera function and the second function is a music execution function. The method of claim 5,
When the camera function is executed based on the sensing of the first gesture,
The control unit generates a control signal for controlling a function related to the camera based on the shape change of the finger. The method of claim 6, wherein the control unit
Estimate a distance between a first finger and a second finger based on the movement of the tendons;
And a zoom-in or zoom-out function according to the estimated interval. The method of claim 6,
It includes a plurality of cameras disposed at different positions,
The control unit
And controlling at least one of the plurality of cameras so that an activated camera among the plurality of cameras is changed according to a direction that the wrist is facing. delete delete delete delete The method of claim 1,
And at least one of the selection target items is selected based on sensing of movement of tendons corresponding to the collision of at least two fingers of the plurality of fingers. delete The method of claim 1,
The sensitivity of the sensing unit for sensing the movement of the tendon is controlled differently according to the position in which the flexible display is wrapped.

Images