KR20110028809A - Mobile terminal and information prcessing method thereof - Google Patents

Abstract

PURPOSE: A mobile terminal and an information processing method thereof are provided to execute a function corresponding to a gesture. CONSTITUTION: A control unit(180) receives a gesture through a camera while performing video communication based on an image acquired by the camera and searches an image corresponding to the received gesture from a memory(160). The control unit transmits the searched image to another party through the video communication. The control unit transmits the image obtained by the camera and the image synthesized with the searched image. The memory stores the program for operating the control unit, and temporally stores input/output data. An interface unit(170) receives power and data from an external device to transfer the power and the data to each component or the internal data to an external device. A sound output module(152) outputs audio data stored in the memory in a communication mode, a voice recognition mode or a broadcasting reception mode.

Description

MOBILE TERMINAL AND INFORMATION PRCESSING METHOD THEREOF}

The present invention relates to a mobile terminal and an information processing method thereof, and more particularly, to a mobile terminal and an information processing method capable of executing a function corresponding thereto by performing a gesture, thereby improving usability of the mobile terminal. It is about.

A terminal such as a personal computer, a notebook computer, or a mobile phone has various functions, for example, a multimedia device having a complex function such as photographing or photographing of a moving picture, reproduction of music or a moving picture file, (Multimedia player).

The terminal can move It may be divided into a mobile terminal and a stationary terminal, depending on whether it is present. The mobile terminal may be further classified into a handheld terminal and a vehicle mount terminal according to whether a user can directly carry it.

In order to support and enhance the functionality of the terminal, it is contemplated to improve the structural and / or software portion of the terminal.

Recently, a variety of terminals including mobile terminals provide a complex and various functions, and the menu structure is also becoming complicated. In addition, a function of displaying various digital documents including a web page is added to a mobile terminal.

The present invention relates to a mobile terminal and an information processing method thereof capable of executing a function corresponding thereto by taking a gesture, thereby improving the usability of the mobile terminal.

Mobile terminal according to an embodiment of the present invention for realizing the above object, at least one camera; And a controller configured to execute a function corresponding to the received gesture when a gesture is received through the camera during a video call based on the image acquired by the camera.

The function may be to search for an image corresponding to the received gesture in a memory and to transmit the searched image to the other party of the video call.

The controller may transmit a composite image obtained by synthesizing the image acquired through the camera and the retrieved image.

The controller may display the progress of transmitting the image on a display module.

The function may be to search for an image corresponding to the received gesture in a memory and to display the retrieved image on a display module of the mobile terminal.

The controller may display at least one of the received gesture or an image corresponding to the received gesture separately from the video call screen.

The controller may display at least one gesture similar to the received gesture on the display module together with the received gesture.

The function may be to search for a function of the mobile terminal corresponding to the received gesture and to execute a function of the searched mobile terminal.

The camera may include a plurality of cameras, one of the cameras may acquire an image for the video call, and the other of the cameras may acquire an image for the gesture reception.

The other camera may be provided at the rear of the mobile terminal.

In addition, the information processing method of the mobile terminal according to an embodiment of the present invention for realizing the above object, the information processing method of the mobile terminal, comprising the steps of: starting a video call; Receiving a gesture through a camera while the video call is in progress; And executing a function corresponding to the received gesture.

The executing of the function may include retrieving an image corresponding to the received gesture and transmitting the retrieved image to the other party of the video call.

The executing of the function may include searching for an image corresponding to the received gesture and displaying the found image on the mobile terminal.

The executing of the function may include searching for a function of the mobile terminal corresponding to the received gesture, and executing the found function.

The camera may be provided in plural, and the camera for receiving the gesture may be provided at a rear side of the mobile terminal.

The mobile terminal and its information processing method according to the present invention can execute a function corresponding thereto by taking a gesture, thereby improving the usability of the mobile terminal.

The above objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. It is to be understood, however, that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and similarities. Like reference numerals designate like elements throughout the specification. In addition, when it is determined that the detailed description of the known function or configuration related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, the numbers (eg, first, second, etc.) used in the description process of the present specification are merely identification symbols for distinguishing one component from another component.

Hereinafter, a mobile terminal related to the present invention will be described in more detail with reference to the accompanying drawings. The suffixes "module" and "unit" for components used in the following description are given or used in consideration of ease of specification, and do not have distinct meanings or roles from each other.

The mobile terminal described herein may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, and the like. However, it will be readily apparent to those skilled in the art that the configuration according to the embodiments described herein may also be applied to fixed terminals such as digital TVs, desktop computers, etc., except when applicable only to mobile terminals.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory unit 160, An interface unit 170, a control unit 180, a power supply unit 190, and the like. The components shown in Fig. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and a network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless internet module 113, a short range communication module 114, a location information module 115, and the like. have.

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video BroadcastHandheld (DVBH).

The broadcast receiving module 111 receives a broadcast signal using various broadcasting systems, in particular, Digital Multimedia Broadcasting Terrestrial (DMBT), Digital Multimedia Broadcasting Satellite (DMBS), Media Forward Link Only (MediaFLO), and Digital Video BroadcastHandheld (DVBH). ), A digital broadcast signal can be received using a digital broadcast system such as Integrated Services Digital Broadcast Terrestrial (ISDBT). Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems that provide broadcast signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory unit 160.

The mobile communication module 112 transmits and receives a wireless signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 can be embedded in the mobile terminal 100 or externally. WLAN (WiFi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short range communication module 114 refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like may be used.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. A representative example of the location information module is a GPS (Global Position System) module. According to the current technology, the GPS module 115 calculates information on a distance (distance) from three or more satellites to one point (entity), information on the time when the distance information is measured, , Three-dimensional position information according to latitude, longitude, and altitude of one point (individual) in one hour can be calculated. Furthermore, a method of calculating position and time information using three satellites and correcting the error of the calculated position and time information using another satellite is also used. The GPS module 115 may continuously calculate the current position in real time and use the same to calculate speed information.

Referring to FIG. 1, the A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory unit 160 or transmitted to the outside through the wireless communication unit 110. The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. The microphone 122 may implement various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for the user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 140 detects a current state of the mobile terminal 100 such as an open / closed state of the mobile terminal 100, a location of the mobile terminal 100, presence or absence of a user contact, orientation of the mobile terminal, acceleration / deceleration of the mobile terminal, and the like. To generate a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, it may be responsible for sensing functions related to whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor.

The output unit 150 is used to generate an output related to sight, hearing, or tactile sense, and includes a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154. Can be.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in a call mode, a user interface (UI) or a graphic user interface (GUI) related to a call is displayed. When the mobile terminal 100 is in a video call mode or a photographing mode, the mobile terminal 100 displays photographed and / or received images, a UI, and a GUI.

The display unit 151 may be a liquid crystal display, a thin film transistor liquid crystal display, an organic light emitting diode, a flexible display, a 3D display And may include at least one.

Some of these displays can be configured to be transparent or light transmissive so that they can be seen from the outside. This may be referred to as a transparent display. A representative example of the transparent display is a transparent LCD. The rear structure of the display unit 151 may also be configured as a light transmissive structure. With this structure, the user can see the object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the implementation form of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another, or may be disposed integrally with one another, and may be disposed on different surfaces, respectively.

When the display unit 151 and a sensor for detecting a touch operation (hereinafter, referred to as a touch sensor) form a mutual layer structure (hereinafter, abbreviated as “touch screen”), the display unit 151 is an output device. It can also be used as an input device. The touch sensor may have, for example, a form of a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display unit 151 or capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor may be configured to detect not only the position and area of the touch but also the pressure at the touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and then transmits the corresponding data to the controller 180. As a result, the controller 180 can know which area of the display unit 151 is touched.

Referring to FIG. 1, a proximity sensor may be disposed in an inner region of the mobile terminal or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

When the touch screen is capacitive, the touch screen is configured to detect the proximity of the pointer by the change of the electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching the pointer on the screen is called "contact touch." The position where the proximity touch is performed by the pointer on the touch screen refers to a position where the pointer is perpendicular to the touch screen when the pointer is in proximity proximity.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory unit 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 152 outputs an acoustic signal related to a function (e.g., a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The sound output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying occurrence of an event of the mobile terminal 100. Examples of events occurring in the mobile terminal include call signal reception, message reception, key signal input, and touch input. The alarm unit 153 may output a signal for notifying occurrence of an event in a form other than a video signal or an audio signal, for example, vibration. The video signal or the audio signal can also be output through the display unit 151 or the audio output module 152.

The haptic module 154 generates various tactile effects that the user can feel. Vibration is a representative example of the haptic effect generated by the haptic module 154. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or may be sequentially output.

In addition to the vibration, the haptic module 154 may be used for the effects of stimulation by the arrangement of pins vertically moving with respect to the contact skin surface, the effect of the injection force of the air through the injection or inlet or the stimulation through the suction force, and the stimulation that rubs the skin surface. Various tactile effects may be generated, such as effects by stimulation through contact of electrodes, effects by stimulation using electrostatic force, and effects of reproducing a sense of warmth and heat using an endothermic or heat generating element.

The haptic module 154 may not only deliver the haptic effect through direct contact, but also implement the haptic effect through the muscle sense of the user's finger or arm. Two or more haptic modules 154 may be provided according to a configuration aspect of the mobile terminal 100.

The memory unit 160 may store a program for the operation of the controller 180 and may temporarily store input / output data (for example, a phone book, a message, a still image, a video, etc.). The memory unit 160 may store data regarding vibration and sound of various patterns output when a touch input on the touch screen is performed.

The memory unit 160 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), At least one of Random Access Memory (RAM), Static Random Access Memory (SRAM), ReadOnly Memory (ROM), Electrically Erasable Programmable ReadOnly Memory (EEPROM), Programmable ReadOnly Memory (PROM) magnetic memory, magnetic disk, optical disk It may include a storage medium of the type. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory unit 160 on the Internet.

The interface unit 170 serves as a path with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to each component in the mobile terminal 100 or to transmit data in the mobile terminal 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip that stores various types of information for authenticating the usage rights of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identify module (SIM), and a universal user authentication module ( Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit may be a passage through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, or various command signals input from the cradle by a user may be moved. It may be a passage that is delivered to the terminal. Various command signals or power input from the cradle may be operated as signals for recognizing that the mobile terminal is correctly mounted on the cradle.

The control unit 180 typically controls the overall operation of the mobile terminal. For example, perform related control and processing for voice calls, data communications, video calls, and the like. The controller 180 may include a multimedia module 181 for playing multimedia. The multimedia module 181 may be implemented in the controller 180 or may be implemented separately from the controller 180.

The controller 180 may perform a pattern recognition process for recognizing a writing input or a drawing input performed on the touch screen as text and an image, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

Various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, microcontrollers, microprocessors, and electrical units for performing the functions. 180).

In a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that allow at least one function or operation to be performed. The software code may be implemented by a software application written in a suitable programming language. Also, the software codes may be stored in the memory unit 160 and executed by the control unit 180. [

2A is a front perspective view of an example of a mobile terminal or a portable terminal according to the present invention.

The disclosed portable terminal 100 has a terminal body in the form of a bar. However, the present invention is not limited thereto and may be applied to various structures such as a slide type, a folder type, a swing type, a swivel type, and two or more bodies are coupled to be relatively movable.

The body includes a casing (casing, housing, cover, etc.) that forms an exterior. In this embodiment, the case may be divided into a front case 101 and a rear case 102. Various electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injecting synthetic resin or may be formed of a metal material, for example, a metal material such as stainless steel (STS) or titanium (Ti).

The display unit 151, the audio output unit 152, the camera 121, the user input units 130/131 and 132, the microphone 122, and the interface 170 may be disposed in the terminal body, mainly the front case 101. have.

The display unit 151 occupies most of the main surface of the front case 101. The sound output unit 152 and the camera 121 are disposed in regions adjacent to one end of both ends of the display unit 151, and the user input unit 131 and the microphone 122 are disposed in regions adjacent to the other end. The user input unit 132, the interface 170, and the like are disposed on the side surfaces of the front case 101 and the rear case 102.

The user input unit 130 is manipulated to receive a command for controlling the operation of the portable terminal 100 and may include a plurality of operation units 131 and 132.

The manipulation units 131 and 132 may also be collectively referred to as manipulating portions, and may be employed in any manner as long as the user operates the tactile manner with a tactile feeling.

The contents input by the manipulation units 131 and 132 may be variously set. For example, the first operation unit 131 receives a command such as start, end, scroll, and the like, and the second operation unit 132 adjusts the volume of the sound output from the sound output unit 152 or the display unit ( A command such as switching to the touch recognition mode of 151 may be input.

FIG. 2B is a rear perspective view of the portable terminal shown in FIG. 2A.

Referring to FIG. 2B, a camera 121 ′ may be additionally mounted on the rear of the terminal body, that is, the rear case 102. The camera 121 'may have a photographing direction substantially opposite to the camera 121 (see FIG. 2A), and may be a camera having different pixels from the camera 121.

For example, the camera 121 has a low pixel so that the user's face is photographed and transmitted to the counterpart in case of a video call, and the camera 121 'photographs a general subject and does not transmit it immediately. It is desirable to have a high pixel because there are many. The cameras 121 and 121 'may be installed in the terminal body such that they can be rotated or popped up.

A flash 123 and a mirror 124 are further disposed adjacent to the camera 121 '. The flash 123 shines light toward the subject when the subject is photographed by the camera 121 '. The mirror 124 allows the user to see his / her own face or the like when photographing (self-photographing) the user using the camera 121 '.

The sound output unit 152 'may be further disposed on the rear surface of the terminal body. The sound output unit 152 ′ may implement a stereo function together with the sound output unit 152 (see FIG. 2A), and may be used to implement a speakerphone mode during a call.

In addition to the antenna for a call or the like, a broadcast signal reception antenna 124 may be additionally disposed on the side of the terminal body. An antenna 124 forming part of the broadcast receiving module 111 (see FIG. 1) may be installed to be pulled out of the terminal body.

A power supply unit 190 for supplying power to the portable terminal 100 is mounted on the terminal body. The power supply unit 190 may be embedded in the terminal body or may be directly detachable from the outside of the terminal body.

The rear case 102 may be further equipped with a touch pad 135 for sensing a touch. Like the display unit 151, the touch pad 135 may also be configured to have a light transmission type. In this case, if the display unit 151 is configured to output visual information from both sides, the visual information may be recognized through the touch pad 135. The information output on both surfaces may be controlled by the touch pad 135. Alternatively, a display may be additionally mounted on the touch pad 135, and a touch screen may be disposed on the rear case 102.

The touch pad 135 operates in association with the display unit 151 of the front case 101. The touch pad 135 may be disposed in parallel to the rear of the display unit 151. The touch pad 135 may have the same or smaller size as the display unit 151.

2C and 2D illustrate various embodiments of the mobile terminal 100 and the display unit 151.

The display unit 151 may include a first display unit in which display panels are physically divided and a second display unit as shown in FIG. 3A. In a foldable or slide type mobile terminal in which two bodies are hinged or slided, the first display unit (or the main display unit) may be formed on the inner surface or the outer surface of one body, and the second display unit The auxiliary display unit or the external display unit may be formed on an inner surface or an outer surface of another body. The secondary display unit is a display unit separate from the mobile terminal, and is detachably connected to the mobile terminal body through an interface means to display data from the mobile terminal 100.

In addition, the display unit 151 may include a first display unit and a second display unit that are logically divided in the display panel as shown in FIG. 2C.

3 is a conceptual diagram illustrating a proximity depth of a proximity sensor.

As shown in FIG. 3, when a pointer such as a user's finger is close to the touch screen, the proximity sensor disposed in or near the touch screen detects this and outputs a proximity signal.

The proximity sensor may be configured to output different proximity signals according to a distance between the proximity touched pointer and the touch screen (hereinafter, referred to as "proximity depth").

The distance at which the proximity signal is output when the pointer approaches the touch screen is referred to as a detection distance. The distance between the proximity signals output from each of the proximity sensors by using a plurality of proximity sensors having different detection distances is the depth of proximity. It can be seen.

In FIG. 3, for example, a cross section of a touch screen on which three proximity sensors capable of sensing three proximity depths is disposed is illustrated. Of course, proximity sensors that detect less than three or more than four proximity depths are also possible.

In detail, when the pointer is completely in contact with the touch screen (d0), the pointer is recognized as a touch. When the pointer is positioned below the distance d1 on the touch screen, the pointer is recognized as a proximity touch of a first proximity depth.

When the pointer is spaced apart from the distance d1 or more and less than d2 on the touch screen, the pointer is recognized as a proximity touch of a second proximity depth. When the pointer is spaced apart from the d2 distance by more than d3 distance on the touch screen, it is recognized as a proximity touch of a third proximity depth. In addition, when the pointer is located at a distance greater than or equal to d3 on the touch screen, the proximity touch is recognized as released.

Accordingly, the controller 180 may recognize the proximity touch as various input signals according to the proximity distance and the proximity position of the pointer touch screen, and perform various operation control according to the various input signals.

Referring to FIG. 4, a CDMA wireless communication system includes a plurality of mobile terminals 100, a plurality of base stations 270, base station controllers BSCs 275, and a mobile switching center. Centers; MSCs, 280). The mobile switching center 280 is configured to be connected to a public switch telephone network (PSTN) 290. The mobile switching center 280 is also configured to be connected with the base station controllers 275. The base station controllers 275 are connected to the base stations 270 via backhaul lines. The backhaul lines may be configured according to E1 / T1, ATM, IP, PPP, frame relay, HDSL, ADSL, or xDSL known to those skilled in the art. It will be apparent to those skilled in the art that the system may include two or more base station controllers 275.

Each of the base stations 270 may include one or more sectors, each of which may include an omnidirectional antenna or an antenna tuned to a specific radial direction from the base station 270. Can be. Alternatively, each sector may include two antennas for diversity reception. Each base station 270 is configured to accommodate a plurality of frequency assignments, which may have a specific spectrum (eg, 1.25 MHz, 5 Mhz).

The intersection of sectors and frequency assignments may be referred to as a CDMA channel.

The base stations 27 may also be referred to as Base station Transceiver Subsystems (BTSs). In some examples, the term "base station" may be used to collectively refer to base station controller 275 and one or more base stations 270. The base stations may also be labeled "cell sites". Alternatively, individual sectors of a given base station 270 may be referred to as cell sites.

The terrestrial DMB transmitter 295 may broadcast to mobile terminals 100 operating in the system. The broadcast receiving module 111 of the mobile terminal 100 is generally configured to receive broadcast signals transmitted by the DMB transmitter 295. This may be similarly applied to other types of broadcast and multicast signaling as described above.

4 illustrates various Global Positioning System (GPS) satellites 300. Such satellites 300 may track the location of some or all mobile terminals 100. Although two satellites are shown, it is apparent to those skilled in the art that the location information can be obtained from more or fewer satellites. Other types of location techniques (eg, location techniques that can be used in place of or in addition to GPS technology) can be used. If desired, some or all of the GPS satellites 300 may be configured to support satellite DMB transmission separately or additionally.

During operation of the wireless communication system, the base stations 270 receive reverselink signals from several mobile terminals 100. The mobile terminals 100 may be in a call, sending a message, or performing other communication. Each reverse link signal received by the base station is processed at the base station. The processed data is transmitted to the connected base station controller 275. The base station controller 275 provides call resource allocation and includes mobility management functionality including orchestration of soft handoffs between base stations 270. ). The base station controllers 275 also send the received data to the mobile switching centers (MSCs) 280. The mobile switching center 280 provides additional routing services for interfacing with the public switch telephone network (PSTN) 290. Similarly, the public switching Ching-Telephone network 290 interfaces with the mobile switching center 280, which interfaces with the base station controllers 275. The base station controllers 275 in turn control the base stations 270 to transmit forward link signals to the mobile terminals 100.

FIG. 5 is a flow chart of a mobile terminal according to an embodiment of the present invention, and FIG. 6 is a view schematically illustrating an operation form of the mobile terminal according to FIG. 5.

As shown in these figures, the mobile terminal 100 according to an embodiment of the present invention may include a step (S10) of executing a camera 121, 121 ′ related function.

The cameras 121 and 121 'may be a first camera 121 located at the front of the mobile terminal 100 and a second camera 121' located at the rear of the mobile terminal 100. In general, the first camera 121 is used for capturing the user himself at the time of video call or photo shoot, and the second camera 121 'is generally used for capturing the surrounding subject.

A function associated with the cameras 121 and 121 'may be a video call. Conventionally, the mobile terminal 100 has been used for the transmission of voice, but as the related technology has developed, it is possible to transmit not only the voice but also the image to the mobile terminal 100 in real time. When making a video call using the mobile terminal 100, cameras 121 and 121 'mounted on the mobile terminal 100 are used. When making a video call, the first camera 121 located in front of the cameras 121 and 121 'is typically used. In other words, when the user holding the mobile terminal 100 executes the video call function after positioning the first camera 121 to face the user, the video call is transmitted to the other party.

When the camera related function is executed, a step S20 of determining whether a gesture is input may be performed.

The gesture may be various hand gestures. That is, it means that it can be a specific shape made using hands in space. As will be described in more detail below, the gesture may be set in advance, or the user may preset and store a desired gesture.

The input of the gesture is performed by the cameras 121 and 121 'being executed. In particular, the cameras 121 and 121 'may be made through the second camera 121' located at the rear of the mobile terminal 100. As described above, the video call is generally made through the first camera 121 located at the front of the mobile terminal 100, so that the second camera (located at the back of the mobile terminal 100) while the video call is being made. 121 ') is generally not used. According to the information processing method of the mobile terminal 100 according to an embodiment of the present invention, the second camera 121 'which is not in use can be utilized, thereby increasing the use efficiency of resources of the mobile terminal 100. .

The input of the gesture using the second camera 121 'may be made without interrupting the video call. As described above, the video call is made through the first camera 121 located in front of the mobile terminal 100, and the second camera 121 'is generally not directly involved in the video call. In addition, the first and second cameras 121 and 121 'are positioned at the front and the rear of the mobile terminal 100, respectively, so that the first and second cameras 121 and 121' are substantially 180 degrees. Done. In such a situation, if the user makes a specific gesture by moving his hand to the rear of the mobile terminal 100, the gesture input can be performed without disturbing the photographing of the first camera 121. Therefore, the gesture can be input while the other party does not recognize it. However, the gesture may not be input only through the second camera 121 ', and may be input through the first camera 121 as necessary.

When the gesture is input, step S30 may determine whether the input gesture corresponds to a preset one.

When the user's hand gesture is input through the second camera 121 ', it may be determined whether the input gesture corresponds to a specific gesture input to the memory (160 of FIG. 1) of the mobile terminal 100. As described above, the gesture for allowing the mobile terminal 100 to perform a specific operation is stored in the mobile terminal 100 from the production stage of the mobile terminal 100, or the user of the mobile terminal 100 You can enter a gesture. In addition, the mobile terminal 100 may be downloaded using the wireless communication unit 110 of FIG. 1.

The gesture input through the second camera 121 'may be a gesture intended by the user, or a gesture day unintentionally recognized by the second camera 121' in the process of holding the mobile terminal 100 by the user. It may be. Therefore, the gesture input through the second camera 121 'is subjected to a process of determining whether or not the gesture matches the preset gesture.

If the input gesture corresponds to the preset gesture, step S40 of executing the corresponding function may be performed.

The corresponding function means a function that a user inputs a gesture through the second camera 121 'and is intended. As a corresponding function, as will be described in more detail below, a specific function of a mobile terminal 100 such as displaying a specific image or text corresponding to a gesture on a counterpart or own screen or searching a phone number is executed. For example, if possible. The corresponding function may be one function corresponding to one gesture or may be a plurality of functions corresponding to one gesture. In addition, there may be one or several functions corresponding to several consecutive gestures. For example, when a crown, an exclamation point, and a heart image are to be transmitted to the opponent, the gesture corresponding to the crown, the gesture corresponding to the exclamation point, and the gesture corresponding to the heart may be input and transmitted to the opponent, respectively. In addition, if one gesture corresponding to a crown, an exclamation point, and a heart is previously input, a continuous image of the crown exclamation point heart may be transmitted by inputting the gesture. Furthermore, the crown image may be transmitted when the palm-shaped gesture and the palm-shaped gesture are continuously input.

6 illustrates a case where an image corresponding to a gesture is transmitted to an opponent. Hereinafter, the mobile terminal currently operated by the user will be displayed as the second mobile terminal 100b as another terminal communicating with the first mobile terminal 100a and the first mobile terminal 100a.

In the first and second mobile terminals 100a and 100b, the display module 151 may be logically partitioned. That is, one's own image captured by the first camera 121 may be displayed in the first display area 151a. The image displayed on the first display area 151a is transmitted to the other party through the wireless communication unit 110 (in FIG. 1). Meanwhile, an image received from the other party may be displayed in the second display area 151b.

As shown in FIG. 6, video communication may be performed between the first mobile terminal 100a and the second mobile terminal 100b. During the video communication process, the user of the first mobile terminal 100a may take a specific gesture G by using his or her hand within the angle of view of the second camera 121 '. Since the gesture G is taken through the second camera 121 'opposite to the first camera 121, the gesture G can be taken without interrupting the video call.

If the specific gesture G taken by the user of the first mobile terminal 100a corresponds to a specific function previously set in the first mobile terminal 100a, the first mobile terminal 100a may execute the corresponding function. have. In FIG. 6, the function corresponding to the specific gesture G is a function of transmitting a heart-shaped image S to the second mobile terminal 100b. Therefore, in the second display area 151b of the second mobile terminal 100b displaying the received image, an image of the user of the first mobile terminal 100a photographed and transmitted by the first mobile terminal 100a, The heart-shaped image S transmitted from the mobile terminal 100a may be displayed. Since the image S can be transmitted through a simple gesture G, the user of the first mobile terminal 100a can express his or her intention to the user of the second mobile terminal 100b simply and clearly. I can display it.

FIG. 7 is a view schematically showing another operation mode of the mobile terminal according to FIG. 5. FIG. 7 illustrates a case where a function corresponding to the gesture G executes a specific function of the first mobile terminal 100a which receives the gesture G input regardless of another mobile terminal 100b of FIG. 6. .

As shown in FIG. 7A, the user of the first mobile terminal 100a may take a specific gesture G within the angle of view of the second camera 121 ′. The controller (180 of FIG. 1) may determine whether the user's gesture G corresponds to a function and allow the corresponding function to be executed. In FIG. 7, the case in which the user's gesture G corresponds to the phone number search function F is described. However, the function corresponding to the gesture G is not limited thereto.

As shown in FIG. 7B, when it is recognized that the user's specific gesture G corresponds to the phone number search function F, the phone number search function F is executed in one area of the display module 151. Can be.

The phone number search function F executed through the gesture G may search for a specific phone number. That is, the movement direction D2 of the indicator I displayed on the retrieved telephone number corresponding to the movement direction D1 of the gesture G may be determined. Therefore, when the gesture G moves upward, the indicator I also moves upward, when the gesture G moves downward, the indicator I moves downward, and the user touches the display module 151. You can also search for the desired phone number without any additional operation.

8A to 8D are diagrams illustrating in detail the operation of a mobile terminal according to an embodiment of the present invention.

As illustrated in FIG. 8A, an area photographed by the second camera 121 ′ of the first mobile terminal 100a may be assumed as a virtual area VP. The virtual area VP may be a first virtual area VP1 and a second virtual area VP2 corresponding to the first display area 151a and the second display area 151b that are logically divided in the display module 151. It can be regarded as divided.

As shown in FIG. 8B, the position where the gesture G is made in the virtual region VP may be different. That is, the gesture G made in the first region VP1 of the virtual region VP is the first gesture G1, and the gesture G made in the second region VP2 of the virtual region VP is the second gesture. It can be divided into (G2). The user can appropriately select an area where the user's gesture G is to be recognized by moving the position of the hand forward, backward, left, and right. That is, when the gesture G is performed from the upper side in the virtual region VP, the gesture G may be recognized as the second gesture G2 of the second region VP2, and the position is relatively lower. In this case, it may be recognized as the first gesture G1 of the first region VP1. How different gestures G, respectively recognized in the first region VP1 and the second region VP2, will be described with reference to FIGS. 8C and 8D.

FIG. 8C illustrates a case where the second gesture G2 is recognized in the second region VP2. As shown in FIG. 2, the second gesture G2 may affect the second display area 151b. That is, when the second gesture G2 is input, it means that a specific image S may be added to the video of the other party currently performing video communication. As described above, the second display area 151b is an area where an image received from the other party performing image communication is displayed. Therefore, when the image S is displayed on the inputted second gesture G2, only the user of the first mobile terminal 100a can view the image on which the image S is displayed without transmitting the image to the counterpart. .

FIG. 8D illustrates a case where the first gesture G1 is recognized in the first region VP1. As shown in the drawing, the first gesture G1 may affect the first display area 151a. That is, when the first gesture G1 is input, it means that the image including the specific image S can be transmitted to the counterpart currently performing video communication. This will be described in detail as follows.

When the first gesture G1 is input to the second area VP2 in the first mobile terminal 100a, the image S may be displayed on an image of the first display area 151a. As described above, the image displayed on the first display area 151a is an image of itself to be transmitted to the other party.

When the image is received from the first mobile terminal 100a, the second mobile terminal 100b may display the received image on the second display area 151b. An image including the image S is displayed in the second display area 151b of the second mobile terminal 100b. As described above, the user may select his or her own appreciation or sharing with the other party by inputting the gesture G by dividing the area.

9A and 9B are diagrams illustrating in detail the operation of a mobile terminal according to an embodiment of the present invention.

As shown in FIG. 9A, according to an exemplary embodiment of the present invention, the mobile terminal 100 can display the gesture G recognized in the third display area 151c and the image S corresponding thereto. have. That is, when the gesture G is recognized through the second camera 121 ′, the screen may be displayed on the third display area 151c. The image displayed on the third display area 151c may be an image itself received through the second camera 121 'or a hand-shaped image corresponding to the recognized gesture G. FIG.

After the recognized gesture G is displayed in the third display area 151c and a predetermined time elapses, the image S corresponding to the gesture G may be displayed. The displayed image S may be displayed on the first display area 151a and transmitted to the other party, or displayed on the second display area 151b to be used only by the user.

FIG. 9B is a diagram illustrating an embodiment of mobile terminals 100a and 100b in which the above-described operation is implemented.

As shown in the first drawing of FIG. 9B, when the user of the first mobile terminal 100a performs a specific gesture G, the first mobile terminal 100a recognizes the gesture G and performs a third operation. It can be displayed on the display area 151c. Accordingly, the user of the first mobile terminal 100a may know whether the gesture G intended by the user is recognized. If the gesture G is not the desired gesture G, the user of the first mobile terminal 100a may recognize the gesture G before the related function is executed. You can enter it again.

As shown in the second drawing of FIG. 9B, when the recognized gesture G is displayed and a predetermined time elapses, an image S corresponding to the gesture G may be displayed on the third display area 151c. Can be.

As shown in the third drawing of FIG. 9B, when the image S is displayed on the third display area 151c and a predetermined time elapses, the image (S) is displayed on the first display area 151a where an image to be transmitted to the counterpart is displayed. S) can be displayed overlapping. An image of the first display area 151a overlaid with the image S may be transmitted to the second mobile terminal 100b, which is the other party of the video call, and transmitted to the third display area 151c during the transmission of the image. An indication that it is busy may be displayed.

As shown in the fourth drawing of FIG. 9B, when an image including the image S is transmitted to the second mobile terminal 100b which is the other party, the second display area 151b of the second mobile terminal 100b is transmitted. The received image including the image S may be displayed.

10A to 10D are diagrams illustrating in detail the operation of a mobile terminal according to an embodiment of the present invention.

As shown in FIG. 10A, the user's gesture G may be input through the second camera 121 ′ of the mobile terminal 100.

As shown in FIG. 10B, the gesture G may have similar first, second, and third gestures G1, G2, and G3. The input gesture G and the first, second, and third gestures G1, G2, and G3 may be determined to be similar to each other as the fingers are extended or bent. Therefore, by displaying the first, second, and third gestures G1, G2, and G3 similar to the current gesture G on the display module 151, the user can easily select the gesture G to be truly input. can do.

As shown in FIG. 10C, the gesture G currently being input may be displayed in the third display area 151c. In the fourth to sixth display areas 151d to 151f, first, second, and third gestures G1, G2, and G3 similar to the currently input gesture G are displayed. The gesture G currently taken by him and the first, second and third gestures G1, G2, and G3 that are determined to be similar to the displayed gesture are displayed on the display module 151, and the user refers to the gesture G. Can change.

As shown in FIG. 10D, the user may change the first gesture G to the first gesture G1. When the user changes the shape of the hand with the first gesture G1, the first gesture G, in which all the fingers are bent, may exclude the first gesture G from the candidate group judged to be similar. The gesture G, which is determined to be similar, may be displayed on the third to fifth display areas 151c to 151e of the display module 151 except for being excluded. Accordingly, the user may select the gesture G that he or she really intends to input or take the gesture G as a reference to allow the user to perform a desired operation.

In the above-described embodiment, the case where one gesture performs one function is mainly described, but various modifications such as one gesture performing several functions or several gestures combined to perform one function Of course this is possible.

As described above, the present invention is not limited to the described embodiments, and various modifications and changes can be made without departing from the spirit and scope of the present invention, which will be apparent to those skilled in the art. Accordingly, such modifications or variations are intended to fall within the scope of the appended claims.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

2A is a front perspective view of a mobile terminal according to an embodiment of the present invention.

2B is a rear perspective view of a mobile terminal according to an embodiment of the present invention.

2C and 2D are diagrams illustrating implementation forms and display screens of a mobile terminal according to various embodiments of the present disclosure.

3 is a conceptual diagram illustrating a proximity depth of a proximity sensor.

4 is a block diagram of a CDMA wireless communication system for communicating with the mobile terminal shown in FIG.

5 is a flow chart of a mobile terminal according to an embodiment of the present invention.

FIG. 6 is a view schematically illustrating an operation form of the mobile terminal according to FIG. 5.

FIG. 7 is a view schematically showing another operation mode of the mobile terminal according to FIG. 5.

8A to 8D are diagrams illustrating in detail the operation of a mobile terminal according to an embodiment of the present invention.

9A and 9B are diagrams illustrating in detail the operation of a mobile terminal according to an embodiment of the present invention.

10A to 10D are diagrams illustrating in detail the operation of a mobile terminal according to an embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

100: mobile terminal 110: wireless communication unit

140: sensing unit 151: display unit

180: control unit

Claims (15)

At least one camera; And And a control unit configured to execute a function corresponding to the received gesture when a gesture is received through the camera during a video call based on the image acquired by the camera. The method of claim 1, The function is And retrieving an image corresponding to the received gesture from a memory and transmitting the retrieved image to a counterpart of the video call. 3. The method of claim 2, The control unit, And a synthesized image obtained by synthesizing the image acquired through the camera and the retrieved image. 3. The method of claim 2, The control unit, And displaying the transmission progress of the image on a display module. The method of claim 1, The function is And searching for an image corresponding to the received gesture in a memory and displaying the retrieved image on a display module of the mobile terminal. The method of claim 5, The control unit, And displaying at least one of the received gesture or an image corresponding to the received gesture separately from the video call screen. The method of claim 5, The control unit, And displaying at least one gesture similar to the received gesture on the display module together with the received gesture. The method of claim 1, The function is And searching for a function of the mobile terminal corresponding to the received gesture to execute a function of the searched mobile terminal. The method of claim 1, The camera is a plurality, Any one of the cameras acquires an image for the video call, The other one of the cameras, characterized in that for obtaining the image for the gesture reception. The method of claim 9, The other camera is provided on the rear of the mobile terminal. In the information processing method of a mobile terminal, Initiating a video call; Receiving a gesture through a camera while the video call is in progress; And And executing a function corresponding to the received gesture. The method of claim 11, The step of executing the function, Searching for an image corresponding to the received gesture; And transmitting the retrieved image to the counterpart of the video call. The method of claim 11, The step of executing the function, Searching for an image corresponding to the received gesture; And displaying the retrieved image on the mobile terminal. The method of claim 11, The step of executing the function, Searching for a function of the mobile terminal corresponding to the received gesture; And executing the searched function. The method of claim 11, The camera is a plurality, And the camera receiving the gesture is provided at a rear side of the mobile terminal.

Images