KR101974431B1 - Mobile terminal and method for controlling of the same - Google Patents

Abstract

A control method for a mobile terminal and a mobile terminal is disclosed. When a predetermined input is received through the touch screen, the mobile terminal divides the touch screen into at least one screen area based on two or more gaze directions acquired through the camera, The control unit can select an operation mode of the divided at least one screen area according to a predetermined input and control display of at least one screen area divided according to the selected operation mode and two or more sight directions.

Description

TECHNICAL FIELD [0001] The present invention relates to a mobile terminal and a control method of the mobile terminal,

The present invention relates to a mobile terminal for dividing a touch screen into at least one screen area based on a viewing direction and controlling display of the divided screen area, and a control method for the mobile terminal.

In recent years, various multimedia functions such as a camera, a digital multimedia broadcasting (DMB), and a photo gallery have been installed in a mobile terminal, so that a mobile terminal having a large screen with a high picture quality has been introduced.

Such a mobile terminal is used for various purposes such as a video call, a game, a memo, a moving picture reproduction, etc., as well as a voice call. Multitasking is supported and a plurality of functions can be performed simultaneously by dividing pollen.

Currently, when multitasking is performed using a mobile terminal, a service is provided on the assumption that a user uses the mobile terminal. In order to provide a service in which a plurality of users simultaneously use one mobile terminal, , A screen division method and a driving method of a divided screen are not determined, which causes inconvenience to a plurality of users.

According to the present invention, a touch screen is divided into at least one screen area based on a viewing direction, an operation mode of a divided screen area is selected according to a predetermined input, and display of each screen area is controlled according to a viewing direction and an operation mode And a control method of the mobile terminal and the mobile terminal.

According to an embodiment of the present invention, when a predetermined input is received through a touch screen, a camera, and a touch screen, the mobile terminal receives the input in two or more gaze directions Wherein the touch screen is divided into at least one screen area based on a predetermined input, and the at least one screen area is divided into at least one screen area divided according to a selected operation mode and two or more viewing directions, And a control unit for controlling the display of the display unit.

The control unit controls the display of at least one screen region divided through the main control region or the first mode for controlling display of each screen region through each control region of at least one screen region divided according to a predetermined input And the second mode can be selected.

When the second mode is selected, the control unit may set the first screen region among the at least one divided screen region and display the main control region in the first screen region.

The control unit may set the screen area corresponding to the first obtained sight direction among the two or more sight line directions acquired through the camera as the first screen area.

The control unit may set a screen area corresponding to the viewing direction in which the face information coincident with the stored face information among the two or more gaze directions acquired through the camera is detected as the first screen area.

The controller may further display a user interface for inputting a control signal to the first screen area when the touch input is received in the main control area.

The control unit may display at least one divided screen area in a combined manner and display the user interface in the first screen area when the display directions of at least one screen area divided based on the viewing direction are the same.

The control unit can confirm the change of the visual line direction acquired from the camera at each set cycle, and re-divide the screen of the touch screen based on the change of the confirmed visual line direction.

According to another aspect of the present invention, there is provided a method of controlling a mobile terminal, the method including sensing at least two gaze directions through a camera, receiving a predetermined input through a touch screen, Dividing the touch screen into at least one screen area based on two or more gaze directions according to a predetermined input, selecting an operation mode of at least one of the divided screen areas, And displaying at least one screen area divided according to two or more gaze directions.

At this time, the step of selecting the operation mode of the divided two or more screen areas and displaying the selected operation mode and the two or more screen areas divided according to the two or more sight line directions may include displaying the divided at least one A first mode for controlling the display of each screen region through each control region of the screen region, or a second mode for simultaneously controlling the display of the divided at least one screen region through the main control region.

The details of other embodiments are included in the detailed description and drawings.

The control method of the mobile terminal and the mobile terminal according to the present invention has the following effects.

According to an embodiment of the present invention, a screen division method of a touch screen can be determined according to two or more eye directions acquired through a camera, and a content displayed on a divided screen can be determined. Therefore, The user can conveniently use the terminal according to his or her own gaze direction.

According to an embodiment of the present invention, when at least one divided screen region operates in the first mode, each screen region can be controlled independently, and when operating in the second mode, It is possible to select the operation mode of the divided screen area so as to control the screen.

According to an embodiment of the present invention, when operating in the second mode, a control signal can be input based on the gaze direction of the main user while determining the display direction of the divided regions according to the gaze direction of a plurality of users By arranging the user interface, it is possible to improve convenience of use for all of the multiple users.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.
2 is a front perspective view of a mobile terminal according to an embodiment of the present invention.
3 and 4 are flowcharts of a method of controlling a mobile terminal according to an embodiment of the present invention.
5 is a diagram illustrating a method of acquiring a user's gaze direction of a mobile terminal according to an embodiment of the present invention.
6A to 9B are diagrams for explaining a control method of a mobile terminal according to an embodiment of the present invention.

The above objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. It is to be understood, however, that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and similarities. Like reference numerals designate like elements throughout the specification. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. In addition, numerals (e.g., first, second, etc.) used in the description of the present invention are merely an identifier for distinguishing one component from another.

Hereinafter, a mobile terminal according to the present invention will be described in detail with reference to the drawings. The suffix " module " and " part " for components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The mobile terminal described in this specification may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), navigation and the like.

However, it will be understood by those skilled in the art that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless the configuration is applicable only to a mobile terminal.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory unit 160, An interface unit 170, a control unit 180, a power supply unit 190, and the like. The components shown in Fig. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of a DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of a DVBH (Digital Video BroadcastHandheld).

The broadcast receiving module 111 receives a broadcast signal using various broadcasting systems. In particular, the broadcast receiving module 111 may be a digital multimedia broadcasting broadcasting (DMBT), a digital multimedia broadcasting satellite (DMBS), a media forward link only (MediaFLO), a digital video broadcasting ), And ISDBT (Integrated Services Digital Broadcast Terrestrial). Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems that provide broadcast signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory unit 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 can be embedded in the mobile terminal 100 or externally. WLAN (WiFi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. A typical example of the location information module is a GPS (Global Position System) module. According to the current technology, the GPS module 115 calculates information on a distance (distance) from three or more satellites to one point (entity), information on the time when the distance information is measured, , Three-dimensional position information according to latitude, longitude, and altitude of one point (individual) in one hour can be calculated. Further, a method of calculating position and time information using three satellites and correcting the error of the calculated position and time information using another satellite is also used. The GPS module 115 continues to calculate the current position in real time and uses it to calculate speed information.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display module 151. [

The image frame processed by the camera 121 may be stored in the memory unit 160 or may be transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. In addition, it may be responsible for a sensing function related to whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include an orientation sensor 141 and / or a proximity sensor.

The output unit 150 is for generating output related to visual, auditory or tactile sense and includes a display module 151, an acoustic output module 152, an alarm unit 153, and a haptic module 154 .

The display module 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display module 151 may be a liquid crystal display, a thin film transistor liquid crystal display, an organic light emitting diode, a flexible display, a 3D display And may include at least one.

Some of these displays may be transparent or light transmissive so that they can be seen through. This may be referred to as a transparent display. A typical example of the transparent display is a transparent LCD or the like. The rear structure of the display module 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display module 151 of the terminal body.

There may be two or more display modules 151 according to the embodiment of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another or may be disposed integrally with each other, or may be disposed on different surfaces.

In a case where the display module 151 and the sensor for sensing the touch operation (hereinafter, referred to as 'touch sensor') have a mutual layer structure (hereinafter referred to as 'touch screen' It can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display module 151 or a capacitance generated in a specific portion of the display module 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display module 151 is touched or the like.

Referring to FIG. 1, a proximity sensor may be disposed in an inner region of the mobile terminal or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as " proximity touch & The act of actually touching the pointer on the screen is called " contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory unit 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 outputs an acoustic signal related to a function (e.g., a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or audio signal may also be output through the display module 151 or the audio output module 152.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may be configured to perform various functions such as an effect of stimulation by a pin arrangement vertically moving with respect to a contact skin surface, an effect of stimulation by air spraying force or suction force through a jet opening or a suction opening, A variety of tactile effects such as an effect of stimulation through contact of an electrode, an effect of stimulation by an electrostatic force, and an effect of reproducing a cold sensation using a heat absorbing or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to feel the tactile effect through the muscles of the user's finger or arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the portable terminal 100.

The memory unit 160 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory unit 160 may store data related to vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory unit 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory) At least one of a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read- Lt; / RTI > type of storage medium. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory unit 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to each component in the mobile terminal 100 or to transmit data in the mobile terminal 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

When the mobile terminal 100 is connected to an external cradle, the interface unit may be a path through which power from the cradle is supplied to the mobile terminal 100, or various command signals input by the user to the cradle may be transmitted It can be a passage to be transmitted to the terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The control unit 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, microcontrollers, microprocessors, and electrical units for performing functions. In some cases such embodiments may be implemented using a controller 180 < / RTI >

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. Also, the software codes may be stored in the memory unit 160 and executed by the control unit 180. [

2 is a perspective view of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 has a bar-shaped terminal body. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body includes a case (a casing, a housing, a cover, and the like) which forms an appearance. In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injection molding a synthetic resin, or may be formed to have a metal material such as stainless steel (STS) or titanium (Ti) or the like.

A display module 151, an audio output module 152, a camera 121, user input units 130/131 and 132, a microphone 122, an interface unit 170, and the like are disposed in the front body 101 .

The display module 151 occupies most of the main surface of the front case 101.

An audio output module 152 and a camera 121 are disposed in an area adjacent to one end of both ends of the display module 151 and a user input part 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132, the interface unit 170, and the like are disposed on the side surfaces of the front case 101 and the rear case 102.

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100 and may include a plurality of operation units 131 and 132.

The operation units 131 and 132 may be collectively referred to as a manipulating portion and may be employed in any manner as long as the user operates in a tactile manner.

The contents input by the operation units 131 and 132 can be variously set. For example, the first operation unit 131 receives commands such as start, end, scroll, and the like, and the second operation unit 132 controls the size of the sound outputted from the sound output module 152, To the touch recognition mode of the touch screen.

Hereinafter, embodiments of the present invention will be described.

FIG. 3 and FIG. 4 are flowcharts of a method of controlling a mobile terminal according to an embodiment of the present invention, and FIG. 5 is a diagram illustrating a method of acquiring a direction of a user's gaze of a mobile terminal according to an embodiment of the present invention.

3, the control unit 180 of the mobile terminal 100 may sense the gaze direction of the user through the camera 121 (S110).

At this time, the controller 180 acquires an image of at least one user staring at the display module 151 through the camera 121 disposed on the front surface of the mobile terminal 100, It is possible to detect the gaze direction in consideration of the positional relationship.

The gaze direction may indicate a direction and / or a point that the user of the mobile terminal 100 is watching. For example, it may include which portion of the display module 151 the user is staring at, how much he or she is staring at the point, which point is following the point, and the like. A method of detecting the gaze direction will be described in detail with reference to FIG.

Referring to FIG. 5, the controller 180 of the mobile terminal 100 may acquire the direction of the user's gaze from the image photographed through the camera 121.

Step S110 of sensing the gaze direction may include a process of acquiring an image of the eye, and a process of calculating a center point of the pupil.

An image of the eye can be obtained through the camera 121 of the mobile terminal 100. [

For example, an image of the user's eyes operating the mobile terminal 100 may be obtained through the camera 121 located at one point on the front surface of the mobile terminal 100.

A user operating the mobile terminal 100 typically looks at the touch screen of the display module 151 located on the front of the mobile terminal 100. Therefore, the face image of the user can be photographed through the camera 121 on the front side, and the control unit 180 can extract the image of the eye part from the photographed face image.

The pupil is the central part of the eye, a round part surrounded by an iris, darker than other parts of the eye, and usually black for Asian.

The user's gaze is closely related to the pupil. For example, the point that the user looks at with interest is matched with the direction of the center point of the pupil, and the center point of the pupil is located at the center from the image of the acquired eye and can be extracted as a feature of relatively dark shape and color . For example, the center point of the pupil can be determined as the center point of the round shape composed of the dark color of the image of the eye.

In order to determine the gaze direction, the motion of the face must be considered.

Facial movements are an element for calculating the user's gaze direction along with the pupil center point. For example, even if the pupil is gazing at the front without motion, moving the face to the left or right will change the direction of the line of sight.

The movement of the face can be detected through the face image of the user photographed through the camera 121. The control unit 180 can extract the direction of the face based on characteristic elements such as forehead, eyes, nose, mouth, ear, and the like in the face image.

Based on the motion of the face and the calculated center point of the pupil, the control unit 180 can calculate the gaze direction.

Specifically, the viewpoint of the eye direction may be the center point EP of the eyeball, and the eye vectors GD1 and GD2 starting at the center point EP of the eyeball and passing through the center point P of the pupil may be the eye direction do.

When the user moves his or her gaze, the center point P of the pupil can move. In this case, if there is no movement of the user's face, the center point EP of the eyeball is fixed and only the center point P of the pupil can move. When the center point P of the pupil moves, the line-of-sight vector can move from the first line-of-sight vector GD1 to the second line-of-sight vector GD2.

The control unit 180 can obtain the gaze direction based on the position of the face of the user acquired through the camera 121 and the position of the center point P of the pupil.

In addition, if there is movement of the user's face, the controller 180 can move the center point EP of the eyeball and the center point P of the pupil simultaneously so that the movement of the user's face acquired through the camera 121 and / The eye direction can be obtained based on the movement of the center point P of the image.

After detecting the gazing direction (S110), the controller 180 receives a predetermined input through the touch screen (S120).

The predetermined input is an input for changing the operation mode of the touch screen to the split operation mode. The input is a touch input of a specific soft key (Soft_key) including an icon displayed on the touch screen, an interface The input unit 170 and the operation unit 132 may be input.

The control unit 180 divides the touch screen into at least one area based on two or more gaze directions obtained through the camera 121 (S130). The control unit 180 may divide the screen area of the touch screen into at least one area according to two or more viewing directions and a set screen division rule.

6A to 6E are diagrams for explaining an embodiment in which the controller 180 divides the touch screen screen into at least one area according to two or more eye directions and a set screen division rule.

6A, the controller 180 acquires a face image of the user A and the user B through the front camera 121, and obtains the face image of the user A and the user B from the face image by moving the face of the user and / It is possible to acquire the gaze direction on the basis of this.

The control unit 180 projects all the user's gaze direction obtained through the camera 121 on a plane parallel to the mobile terminal 100 and expresses the gaze vector as a gaze vector. If the gaze vector of the user A is a y-vector, The line-of-sight vector can be represented by -x vector.

Hereinafter, the 'line of sight' is used to refer to a vector obtained by projecting the line of sight of the user obtained through the camera 121 onto a plane parallel to the mobile terminal 100.

The control unit 180 may detect the position of the user based on the starting point of the projected line of sight vector and may divide the touch screen screen area according to the screen division rule set according to the user's position.

The memory 160 may store a set screen division rule. The screen division rule may be set and programmed at the time of manufacture of the mobile terminal 100, or may be changed after manufacture.

For example, when a screen division rule set as shown in the following [Table 1] is stored in the memory 160, the control unit 180 displays the screen division rule shown in FIG. 6A to FIG. 6E according to the viewing direction of the user A, The screen area can be divided into a, b, and c, respectively.

Number of Users Eye vector Split rule 2

The two line-of-sight vectors are parallel

The touch screen is divided into two screen areas in a direction parallel to the line-of-sight vector (Figs. 6B, 6C, and 6D) 2 If two eye vectors are vertical

The touch screen is divided into two screen areas in the vertical direction (Fig. 6A) 3 The two line-of-sight vectors are parallel, and the remaining line-of-sight vectors are the two line-

The touch screen is divided into two screen regions in a direction parallel to the vertical line of sight vector and the region where the starting point of two parallel line-of-sight vectors of the divided touch screen is divided into two screen regions again, Split into three screen areas (Fig. 6E) 3 All three eye vectors are parallel

The touch screen is divided into three screen areas in a direction parallel to the line-of-sight vector (not shown)

At this time, the controller 180 detects that the two line-of-sight vectors are parallel when the two line-of-sight vectors are equal to or more than 0 and equal to or less than 45 degrees, and detects that the two line-

The control unit 180 can detect only the set number of users as the number of users acquired through the camera 121 is changed. That is, when the image of the user whose number of users is greater than the number of the set users is acquired, the controller 180 processes the image of the user who has exceeded the number of set users as noise and does not divide the screen area of the touch screen any more.

In the above embodiment, the control unit 180 detects two or three users, but it can be configured to detect four or more users.

In addition, the controller 180 may divide the screen area of the touch screen differently based on the changed viewing direction when the number of the viewing directions and the viewing directions sensed through the camera 121 at a specific time vary with time.

When the touch screen is divided into at least one screen area, the controller 180 may select an operation mode of the divided screen area, and control the display of the divided screen areas according to the operation mode and two or more eye directions (S140).

Referring to FIG. 4, the controller 180 of the mobile terminal 100 determines an operation mode according to a predetermined input (S141), and can select whether the operation mode is the first mode or the second mode (S142).

The controller 180 can select one of a screen division operation mode in which the touch screen is divided into at least one screen area and a general screen operation mode in which the touch screen is driven in one screen area according to a predetermined input S141).

In addition, when the multi-region operation mode is selected, the controller 180 divides the touch screen into at least one screen region and drives the multi-region operation mode. In accordance with a predetermined input, the multi-region operation mode is switched to either the first mode or the second mode The operation mode can be determined to be selected (S142).

If the first mode is selected, the control unit 180 may control the display of each screen area through each control area of at least one divided screen area (S143). That is, the control unit 180 can process control signals independently of each of the divided screen areas.

If the second mode is selected, the control unit 180 can simultaneously control display of at least one screen area divided through the main control area (S144). That is, the same control signal can be used for each divided screen area in accordance with the control signal of the main control area.

The control unit 180 may display each screen area according to the control signals of the first mode or the second mode (S145).

6A to 9B are diagrams for explaining a control method of a mobile terminal according to an embodiment of the present invention.

Hereinafter, a method of dividing a screen area of a touch screen of a mobile terminal and a control method of displaying a divided screen area according to an embodiment of the present invention will be described in detail with reference to FIGS. 6A to 8E.

6A to 6E are views for explaining a method of dividing a screen area of a touch screen of the mobile terminal 100. FIG.

As described above, the screen area dividing method of the touch screen can be determined according to the viewing direction obtained through the camera 121 and the set screen division rule.

Referring to FIG. 6A, when two line-of-sight vectors of the user, which are obtained through the camera 121 and are vertical such as a y-vector and a -x vector, are detected, the control unit 180 displays the screen area of the touch- Into two screen areas (a area and b area).

6B to 6D, the control unit 180 determines whether the user's gaze vector acquired through the camera 121 is (y vector, -y vector), (x vector, -x vector), (y vector, y The screen region of the touch screen can be divided into two screen regions (a region and b region) in a direction parallel to the line-of-sight vector, if the two line-of-sight vectors are detected to be parallel to each other.

At this time, the controller 180 can estimate the position of the user in consideration of the start position of the line-of-sight vector and determine the screen area ratio to be divided accordingly. If the screen area ratio to be divided is not set, the control unit 180 can divide the screen area so that the touch screen is bisected.

That is, the control unit 180 divides the screen into two screen areas divided by the center of the camera 121 located at the center of the horizontal side of the mobile terminal 100, or bisects the touch screen to connect the points bisecting the vertical side It can be divided into two screen areas.

Referring to FIG. 6E, the control unit 180 determines whether the user's gaze vector obtained through the camera 121 is two vertical line vectors such as a y vector, a y vector, and a -x vector, If one line of sight vector is detected, the screen area of the touch screen can be divided into three screen areas.

Specifically, the control unit 180 divides the touch screen into two screen regions (c region, a + b region) in a direction parallel to the vertical line of sight vector, and generates two line-of-sight vectors The screen area of the touch screen on the side where the starting point of the touch screen is located can be divided into two screen areas (area a and area b).

As a result, when three line-of-sight vectors are detected, the controller 180 can divide the touch screen into three screen areas of the area a, the area b, and the area c.

7A to 7E are diagrams for explaining a control method of displaying a screen area of a divided touch screen of the mobile terminal 100 in the first mode, FIG. 2 is a diagram illustrating a control method of displaying a screen area of a divided touch screen of FIG.

7A to 7E, when the first mode is selected, the controller 180 can control the display of each screen area through each of the control areas of at least one divided screen area. That is, the control unit 180 can process control signals independently of each of the divided screen areas.

Since the control unit 180 processes the input signals received from the divided two screen areas or the three screen areas, the a area, the b area or the c area as independent control signals, An application execution screen can be displayed.

When the input signals simultaneously input to the touch screen of one mobile terminal 100 are input signals received in different screen areas (a area, b area, c area), the controller 180 controls the operation of two or more independent inputs Signal, so that one mobile terminal 100 can divide the screen area and freely use multitasking.

Therefore, two, three, or four or more applications can be independently executed at the same time according to the number of divided screen areas.

8A to 8E, when the second mode is selected, the controller 180 can simultaneously control display of at least one screen area divided through the main control area. That is, the control unit 180 can control each divided screen area with the same control signal according to the control signal of the main control area.

The control unit 180 sets the first screen area of the divided two screen areas or the three screen areas, the area a and the area b, or the areas a, b, and c, Area can be set.

In the second mode, the control unit 180 simultaneously controls other screen areas divided through the main control area disposed in the first screen area, so that the same application is executed in all the divided screen areas.

The control unit 180 may set a screen area corresponding to the first sight line direction obtained in the time among the two or more sight line directions obtained through the camera 180 as a first screen area.

In addition, the controller 180 may set a screen area corresponding to the viewing direction in which face information coinciding with the stored face information among the two or more gaze directions acquired through the camera 180 is detected, as the first screen area.

8A to 8E, the controller 180 is an embodiment in a case where the area a is set as the first screen area and the area a1 in the area a is set as the main control area.

Since the control unit 180 processes the input signal received in the main control area a1 arranged in the first screen area (area a) as an independent control signal, Display the application.

The control unit 180 determines that only the input signal received in the a1 region as the main control region is a valid input signal and processes the input signal as a corresponding control signal. The control unit 180 controls the b1 region, which is a control region of the b region or the c region, The input signal received in the c1 region can be processed as noise.

The controller 180 may additionally display a user interface a1 'capable of inputting a control signal to the area a, which is the first screen area, when the touch input is received in the area a1, which is the main control area. At this time, the user interface may be a keypad, an additional search window, or the like.

When the divided screen area of the touch screen is operated in the second mode, the controller 180 determines that only the input signal received in the first screen area is valid, and when the user uses the first screen area, A control area or a user interface.

Therefore, according to one embodiment of the present invention, when the mobile terminal 100 operates in the second mode, only the main user controls the mobile terminal 100 and other users only view the result screen, Can be improved.

In the second mode, when at least one screen area divided on the basis of the viewing direction is the same, the control unit 180 integrally displays the divided screen areas, and displays the user interface a1 ' 1 screen area.

Referring to FIG. 8D, since the gaze vectors of the two users are the same in the y vector direction, the controller 180 can integrate and display two divided screen areas a and b, When the control signal is input to the control area a1, the user interface a1 'can be additionally displayed in the first screen area a.

Referring to FIG. 8E, since the gaze vectors of two users of the three users are the same in the y vector direction, the controller 180 determines that the user has two screen regions a, b, Area (a, b) can be integrated and displayed. In this case, the main control area a1 means a screen area including the main control area a1 before the divided screen areas are integrated. When a control signal is input to the main control area a1, the control unit 180 displays the user interface a1 'in the first screen area a, which is a screen area including the main control area a1, can do.

At this time, the controller 180 may set the first screen area a as a screen area corresponding to the gaze direction of the first user obtained through the camera 121, The first screen region (a) can be set to a screen region corresponding to the viewing direction in which the face information is obtained.

The security method of the mobile terminal according to the viewing direction will be described in detail with reference to FIGS. 9A and 9B.

Referring to FIG. 9A, when a predetermined input is received, the controller 180 detects a first user through the camera 121 and detects a second user, performs a mosaic process on a specific area displayed on the touch screen, , And the like.

9B, when a predetermined input is received, the controller 180 detects the first user through the camera 121 and then turns off the screen of the touch screen when the second user is sensed. And a warning message such as " others are recognized " on the touch screen can be displayed in the form of a pop-up window.

In this case, the predetermined input may be a touch input of a specific soft key displayed on the touch screen or an operation signal of a function key of the mobile terminal 100, .

The control unit 180 can set the first user to be the first user to obtain the gaze direction through the camera 121 or to set the user who matches the stored user's face information as the first user.

The control unit 180 may enhance the security by screening the entire screen or a specific area displayed on the touch screen when a plurality of sight directions other than the first user are recognized through the camera 121. [

The above-described method of controlling a mobile terminal according to the present invention can be provided by being recorded in a computer-readable recording medium, which is a program for execution in a computer.

The control method of the mobile terminal according to the present invention can be executed through software. When executed in software, the constituent means of the present invention are code segments that perform the necessary tasks. The program or code segments may be stored in a processor readable medium or transmitted by a computer data signal coupled with a carrier wave in a transmission medium or a communication network.

A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording device include a ROM, a RAM, a CD-ROM, a DVD-ROM, a DVD-RAM, a magnetic tape, a floppy disk, a hard disk and an optical data storage device. The computer-readable recording medium may also be distributed to networked computer devices so that computer readable code can be stored and executed in a distributed manner.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The present invention is not limited to the drawings. In addition, the embodiments described in this document are not limited to be applied, but all or some of the embodiments may be selectively combined so that various modifications can be made.

100: mobile terminal 151: display module
180:

Claims (10)

touch screen;
camera; And
Wherein when the predetermined input is received through the touch screen, the touch screen is divided into at least two or more screen areas based on two or more gaze directions acquired through the camera, and in accordance with the predetermined input And a control unit for selecting an operation mode of at least two or more divided screen regions and controlling display of the divided at least two screen regions according to the selected operation mode and the two or more sight line directions,
Wherein,
A first mode for controlling the display of each screen through each control area of the divided at least two screen areas according to the predetermined input or a display for displaying the divided at least two screen areas simultaneously And selects one of the first mode and the second mode for controlling the mobile terminal.
delete The method according to claim 1,
Wherein the control unit sets a first screen area among the divided at least two screen areas when the second mode is selected and displays the main control area in the first screen area.
The method of claim 3,
Wherein the control unit sets the screen area corresponding to the first sight line direction obtained first among the two or more sight line directions obtained through the camera as the first screen area.
The method of claim 3,
Wherein the control unit sets the screen area corresponding to the viewing direction in which face information coinciding with the stored face information among the two or more gaze directions acquired through the camera is detected as the first screen area Mobile terminal.
The method of claim 3,
Wherein the control unit further displays a user interface capable of inputting a control signal to the first screen area when the touch input is received in the main control area.
The method according to claim 6,
Wherein the control unit integrates and displays the divided at least two screen areas when the divided display areas of the at least two screen areas are the same based on the viewing direction, To the mobile terminal.
The method according to claim 1,
Wherein the control unit confirms the change of the visual line direction obtained from the camera at each set cycle and re-divides the screen of the touch screen based on the change in the verified visual line direction.
Sensing two or more gaze directions through a camera;
Receiving a predetermined input via a touch screen;
Dividing the touch screen into at least two screen areas based on the two or more gaze directions according to the predetermined input; And
And selecting at least two divided screen regions in accordance with the selected operation mode and the two or more viewing directions,
Wherein the step of displaying the divided two or more screen areas comprises:
A first mode for controlling the display of each screen area through each control area of the divided at least two screen areas or a display for displaying the divided at least two screen areas through the main control area in accordance with the predetermined input And a second mode for simultaneously controlling the first mode and the second mode.
delete

Images