KR20130071059A - Mobile terminal and method for controlling thereof - Google Patents

Abstract

PURPOSE: A portable terminal and a control method thereof are provided to enable a user to conveniently capture a three-dimensional image and to watch the three-dimensional image. CONSTITUTION: A display unit(151) displays a preview image by using an image photographed by a first and a second camera. A control unit(180) determines a first distance with a first object by using an image obtained from the first and the second camera. The control unit obtains a difference value between the first and the second distance. The control unit displays a visual effects corresponding to the difference value on the display unit. [Reference numerals] (110) Wireless communication unit; (111) Broadcasting reception module; (112) Mobile communication module; (113) Wireless internet module; (114) Local area communication module; (115) Location information module; (120) A/V input unit; (121) Camera; (122) Microphone; (130) User input unit; (140) Sensing unit; (141) Proximity sensor; (150) Output unit; (151) Display unit; (152) Sound output module; (153) Alarm unit; (154) Haptic module; (155) Projector module; (160) Memory; (170) Interface unit; (180) Control unit; (181) Multimedia module; (190) Power supply unit

Description

[0001] MOBILE TERMINAL AND METHOD FOR CONTROLLING THEREOF [0002]

The present invention relates to a mobile terminal capable of capturing 3D stereoscopic images and a control method for performing various comparative measurement functions using the images photographed through the same.

Recently, the image display device of the terminal type is increasing. These terminals are mobile And can be divided into a mobile / portable terminal and a stationary terminal depending on whether the mobile terminal is a mobile terminal or a mobile terminal. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

In order to support and enhance the functionality of such terminals, it may be considered to improve the structural and / or software parts of the terminal.

Recently, a stereoscopic 3D stereoscopic image of a stereoscopic method is implemented through a display unit of a mobile terminal, and a plurality of cameras provided therein can also directly capture content that can be output as a 3D stereoscopic image. .

Accordingly, users want to shoot 3D stereoscopic content content under optimal conditions, and demand a user interface that can use the captured image for more various purposes.

The present invention is to provide a mobile terminal and a user interface for controlling the mobile terminal capable of taking and viewing 3D stereoscopic images more conveniently.

Another object of the present invention is to provide a mobile terminal having a user interface capable of informing a user of an optimal 3D photographing condition and a control method thereof.

Another object of the present invention is to provide a mobile terminal and a method of controlling the same, which can use an image of a specific subject for various purposes.

In addition, an object of the present invention is to provide a mobile terminal capable of controlling the mobile terminal with a captured pointer image and a method of controlling the same.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

A mobile terminal according to an embodiment of the present invention for realizing the above object includes a first camera and a second camera for photographing at least one common object; A display unit configured to display a preview image by using images captured by the first camera and the second camera; And determining a first distance with a first object from among the at least one common object by using the images acquired from the first camera and the second camera, and wherein the first distance is determined with the preset second distance. The controller may be configured to obtain a difference value and to control a visual effect corresponding to the difference value to be displayed on the display unit.

In addition, a control method of a mobile terminal according to an embodiment of the present invention for realizing the above object comprises the steps of activating a first camera and a second camera for photographing at least one common object; Generating a preview image by using images captured by the first camera and the second camera, and displaying the preview image on a display unit; Determining a first distance from a first object of the at least one common object using an image obtained from the first camera and the second camera; And obtaining a difference value between the first distance and the second distance, and displaying a visual effect corresponding to the difference value on the display.

On the other hand, the mobile terminal according to another embodiment of the present invention for realizing the above object, the camera for taking an image of the pointer; And determining the distance-specific size of the pointer using a first image photographed when the pointer is spaced apart from the camera by a first distance, and a second image photographed when the pointer is spaced apart by a second distance, and inputted from the camera at predetermined intervals. And a controller configured to detect the pointer from an image and determine a position and a separation distance of the pointer.

In addition, a control method of a mobile terminal according to another embodiment of the present invention for realizing the above object comprises the steps of: photographing a first image from which the pointer is spaced apart from the camera by a camera; Photographing a second image when the pointer is spaced apart from the camera by a second distance through the camera; Determining a distance-specific size of the pointer using the first image and the second image; And detecting the pointer from an image input from the camera at predetermined intervals to determine the position and the separation distance of the pointer.

The present invention configured as described above provides the following effects.

First, the user can shoot and enjoy 3D stereoscopic images more conveniently.

Second, the user can visually feedback the optimal 3D stereoscopic image shooting conditions through various visual effects.

Third, according to the present invention, an image of a specific subject may be stored together with a distance at the time of photographing to correspond to the preview image.

Fourth, the mobile terminal can be controlled by the movement of the pointer without touching the touch screen by using the image of the pointer.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.
2A is a front perspective view of a mobile terminal according to an embodiment of the present invention.
2B is a rear perspective view of a mobile terminal according to an embodiment of the present invention.
3 is a conceptual diagram for explaining the principle of binocular disparity.
4 is a conceptual diagram for explaining a sense of distance and three-dimensional depth due to binocular parallax.
5 is a view for explaining the principle of the 3D stereoscopic image display method using binocular parallax that can be applied to embodiments of the present invention.
6 is a flowchart illustrating an example of an operation process of a mobile terminal capable of utilizing 3D stereoscopic image capturing and photographed content according to an embodiment of the present invention.
7 illustrates an example of a method of measuring a distance of a subject in a mobile terminal according to an embodiment of the present invention.
8 illustrates an example of a form in which an optimum distance from a subject is informed in a mobile terminal according to an embodiment of the present invention.
9A and 9B illustrate another example of a form informing an optimum distance to a subject from a mobile terminal according to an embodiment of the present invention.
FIG. 10 illustrates another example of a form of informing an optimum distance from a subject in a mobile terminal according to an embodiment of the present invention.
11 is a diagram for one example of notifying an optimum distance from a background in a mobile terminal according to one embodiment of the present invention;
12 illustrates an example of a process of storing information including selection of a specific subject and distance information in a mobile terminal according to an embodiment of the present invention.
13 illustrates an example of a process in which a matching mode is performed in a mobile terminal according to an embodiment of the present invention.
14A to 14C are views for explaining an example of a process of performing a length measurement mode in a mobile terminal according to another embodiment of the present invention.
15 is a flowchart illustrating an example of an operation process of a mobile terminal for driving a user interface according to another embodiment of the present invention.
16 illustrates an example of a pointer recognition procedure according to another embodiment of the present invention.
17 illustrates an example of a selection target object implemented as a 3D stereoscopic image in a mobile terminal according to another embodiment of the present invention.
FIG. 18 illustrates an example of a visual effect given to a selection target object implemented as a 3D stereoscopic image in a mobile terminal according to a position of a pointer according to another embodiment of the present invention.
19 illustrates an example in which the shape of a cursor is changed according to a distance between a selection target object and a pointer according to another embodiment of the present invention.

Hereinafter, an image display device according to the present invention will be described in more detail with reference to the accompanying drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

It is assumed that the image display device described herein is a mobile terminal. The mobile terminal may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, and the like. However, it will be readily apparent to those skilled in the art that the configuration according to the embodiments described herein may also be applied to fixed terminals such as digital TVs, desktop computers, digital media players, etc., except when applicable only to mobile terminals. will be.

Overall configuration

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, A controller 170, a controller 180, a power supply 190, and the like. The components shown in FIG. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

For example, the broadcast receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S), a Media Forward Link Only And a Digital Broadcasting System (ISDB-T) (Integrated Services Digital Broadcast-Terrestrial). Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication module 114 refers to a module for short range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

The position information module 115 is a module for obtaining the position of the mobile terminal, and a representative example thereof is a Global Position System (GPS) module.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ Two or more cameras 121 may be provided depending on the use environment.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to the external device may be sensed. Meanwhile, the sensing unit 140 may include a proximity sensor 141.

The output unit 150 is for generating an output relating to visual, auditory or tactile sense and includes a display unit 151, an acoustic output module 152, an alarm unit 153, a haptic module 154, 155, and the like.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, and a 3D display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another, or may be disposed integrally with one another, and may be disposed on different surfaces, respectively.

When the display unit 151 and a sensor for detecting a touch operation (hereinafter, referred to as a touch sensor) form a mutual layer structure (hereinafter referred to as a touch screen), the display unit 151 may be configured in addition to an output device. Can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. As a result, the controller 180 can know which area of the display unit 151 is touched.

The proximity sensor 141 may be disposed in an inner region of the mobile terminal or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching the pointer on the screen is called "contact touch." The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or the audio signal may be output through the display unit 151 or the audio output module 152 so that they may be classified as a part of the alarm unit 153.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to vibration, the haptic module 154 may be configured to provide a pin array that vertically moves with respect to the contact skin surface, a jetting force or suction force of air through an injection or inlet port, grazing to the skin surface, contact of an electrode, electrostatic force, and the like. Various tactile effects can be generated, such as effects by the endothermic and the reproduction of a sense of cold using the elements capable of endotherm or heat generation.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the portable terminal 100.

The projector module 155 is a component for performing an image project function using the mobile terminal 100 and is similar to the image displayed on the display unit 151 in accordance with a control signal of the controller 180 Or at least partly display another image on an external screen or wall.

Specifically, the projector module 155 includes a light source (not shown) that generates light (for example, laser light) for outputting an image to the outside, a light source And a lens (not shown) for enlarging and outputting the image at a predetermined focal distance to the outside. Further, the projector module 155 may include a device (not shown) capable of mechanically moving the lens or the entire module to adjust the image projection direction.

The projector module 155 can be divided into a CRT (Cathode Ray Tube) module, an LCD (Liquid Crystal Display) module and a DLP (Digital Light Processing) module according to the type of the display means. In particular, the DLP module may be advantageous for miniaturization of the projector module 151 by enlarging and projecting an image generated by reflecting light generated from a light source on a DMD (Digital Micromirror Device) chip.

Preferably, the projector module 155 may be provided longitudinally on the side, front or back side of the mobile terminal 100. It goes without saying that the projector module 155 may be provided at any position of the mobile terminal 100 as occasion demands.

The memory unit 160 may store a program for processing and controlling the control unit 180 and temporarily store the input / output data (e.g., telephone directory, message, audio, For example. The memory unit 160 may also store the frequency of use of each of the data (for example, each telephone number, each message, and frequency of use for each multimedia). In addition, the memory unit 160 may store data on vibration and sound of various patterns output when the touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data to the external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit may be a passage through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, or various command signals input from the cradle by a user may be transferred. It may be a passage that is delivered to the terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions. The described embodiments may be implemented by the controller 180 itself.

According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code can be implemented in a software application written in a suitable programming language. The software code is stored in the memory 160 and can be executed by the control unit 180. [

Organization description

2A is a perspective view of an example of a mobile terminal or a mobile terminal according to the present invention.

The disclosed mobile terminal 100 has a bar-shaped terminal body. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body includes a case (a casing, a housing, a cover, and the like) which forms an appearance. In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injecting synthetic resin or may be formed of a metal material, for example, a metal material such as stainless steel (STS) or titanium (Ti).

The display unit 151, the sound output unit 152, the camera 121, the user input units 130/131 and 132, the microphone 122, the interface 170, and the like may be disposed in the front body 101 have.

The display unit 151 occupies most of the main surface of the front case 101. A sound output unit 151 and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151 and a user input unit 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132 and the interface 170 may be disposed on the side surfaces of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the portable terminal 100 and may include a plurality of operation units 131 and 132. The operation units 131 and 132 may be collectively referred to as a manipulating portion and may be employed in any manner as long as the user operates in a tactile manner.

The contents inputted by the first or second operation unit 131 or 132 may be variously set. For example, the first operation unit 131 receives commands such as start, end, scroll, and the like, and the second operation unit 132 controls the size of the sound output from the sound output unit 152 or the size of the sound output from the display unit 151 To the touch recognition mode of the touch screen.

FIG. 2B is a rear perspective view of the portable terminal shown in FIG. 2A.

Referring to FIG. 2, two cameras 121-1 and 121-2 may be additionally mounted on the rear of the terminal body, that is, the rear case 102. The two cameras 121-1 and 121-2 may be spaced apart from each other by a predetermined interval, and may be cameras having different pixels from the camera 121 provided in the front portion of FIG. 1. However, the two rear cameras 121-1 and 121-2 preferably have the same number of pixels and angle of view. The two cameras on the rear side are arranged to be spaced apart by a predetermined interval, so that the same subjects can be photographed at different times using the two cameras at the same time. That is, one camera captures an image for the left eye, and the other camera captures an image for the right eye, and the source image may be generated as the captured images are processed in a method to be described later.

The flash 123 may be further disposed adjacent to the two cameras 121-1 and 121-2. The flash 123 shines light toward the subject when the subject is photographed through at least one of the two cameras 121-1 and 121-2.

An acoustic output 152 'may be additionally disposed on the rear surface of the terminal body. The sound output unit 152 'may implement the stereo function together with the sound output unit 152 (see FIG. 2A), and may be used for the implementation of the speakerphone mode during a call.

Implementation method of 3D stereoscopic image

Hereinafter, a method of expressing a 3D image of a mobile terminal and a display unit therefor which can be applied in embodiments of the present invention will be described.

Stereoscopic images implemented on the display unit 151 of the mobile terminal may be classified into two categories. The criterion for this classification is whether or not different images are provided in both eyes.

First, the first stereoscopic image category will be described.

The first category is a method in which the same image is provided in both eyes (monoscopic), which can be implemented as a general display unit. In more detail, the controller 180 arranges (or renders) at least one polyhedron generated through one or more points, lines, planes, or a combination thereof in a virtual three-dimensional space, and displays an image viewed from a specific viewpoint. The display unit 151 may be displayed on the display unit 151. Therefore, the substantial part of such a stereoscopic image can be referred to as a planar image.

The second category is a stereo scopic method in which different images are provided in both eyes. It is a method using a principle of feeling a three-dimensional feeling when a human is looking at an object with the naked eye. In other words, the two eyes of a person see different plane images when they see the same thing by the distance between them. These different planar images are transmitted to the brain through the retina, and the brain fuses them to feel the depth and reality of the stereoscopic image. Therefore, although there are some differences between people, binocular disparity due to the distance between the two eyes makes a sense of three-dimensional feeling, and such binocular disparity becomes the most important element of the stereoscopic method. This binocular disparity will be described in more detail with reference to Fig.

3 is a conceptual diagram for explaining the principle of binocular disparity.

In FIG. 3, it is assumed that the hexahedron 310 is viewed with the naked eye in front of the eye level. In this case, the left eye planar image 320 in which only three surfaces of the top, front, and left sides of the hexahedron 310 are visible is seen as the left eye. In addition, as the right eye, the right eye plane image 330 showing only three surfaces of the top, front, and right sides of the cube 310 is visible.

If the object is in front of the eye, even if the left eye planar image 320 is reached in the left eye and the right eye planar image 330 is reached in the right eye, the person may feel as if the user actually sees a three-dimensional cube 310.

As a result, in order to implement the stereoscopic image of the second category in the mobile terminal, the left eye image and the right eye image, which have been viewed with the same object with a predetermined parallax, must be separated and reached through the display unit.

In the present specification, in order to distinguish the two categories described above, a stereoscopic image of the first category is referred to as a "2D stereoscopic image" or a "planar 3D rendered image", and a stereoscopic image of the second category is referred to as a "3D stereoscopic image".

Next, with reference to FIG. 4, the three-dimensional depth by binocular parallax is demonstrated.

4 is a conceptual diagram for explaining a sense of distance and three-dimensional depth due to binocular parallax.

Referring to FIG. 4, when looking at the cube 400 at a distance d1 through both eyes, the proportion of the lateral surface of the image coming into each eye is relatively higher than that at the distance d2 at the distance from the cube 400. The difference is also great. Also, the degree of three-dimensional sensation experienced by a person when viewing the hexahedron 400 at a distance d1 is greater than when viewing the hexahedron 400 at a distance d2. In other words, when a person looks at an object through both eyes, the closer to the object, the greater the three-dimensional feeling, and the farther the object, the less the three-dimensional feeling.

This difference in three-dimensionality can be digitized to a 3D depth or 3D level. Hereinafter, in the present specification, the 3D depth protruding from the display unit is referred to as the 3D depth in the + direction, and the 3D depth that is recessed from the display unit is referred to as the 3D depth in the − direction, and is viewed as the same depth as the display unit. "0") is called depth. The definition of the three-dimensional depth or three-dimensional level is relative, and the classification criteria and the direction of increase / decrease may be changed.

Next, an implementation method of the 3D stereoscopic image will be described.

As described above, in order to realize a 3D stereoscopic image, it is necessary that the right eye image and the left eye image are divided into two and reach the binocular. Various methods for this are described below.

1) parallax barrier method

The parallax barrier method is a method of electrically controlling a blocking device provided between a general display unit and both eyes so as to control the direction of light to reach different images in both eyes.

This will be described with reference to FIG.

5 is a view for explaining the principle of a three-dimensional image display method using binocular parallax according to an embodiment of the present invention.

As shown, in order to display a 3D stereoscopic image in FIG. 5, the display unit 151 includes a display panel and a switching panel attached to an upper surface thereof. The switching panel may be electrically controlled to block or pass some of the light reaching both eyes from the display panel. In this case, the display panel may be configured with a general display device such as LCD, LED, AMOLED.

In FIG. 5, b denotes a barrier spacing of the switching panel, g denotes a gap between the switching panel and the display panel, and z denotes a distance from the position where a person sees the display panel. As shown in FIG. 5, when the two images are synthesized pixel by pixel (L, R), the time of the right eye corresponds to the pixel included in the right image, and the time of the left eye corresponds to the pixel included in the left image. The incident switching panel may operate.

 When the switching panel is to display a 3D stereoscopic image, the switching panel may be turned on to separate the incident time. In addition, when the 2D image is to be displayed, the switching panel may be turned off and passed as it is without separating the incident time. Therefore, binocular disparity is not separated when the switching panel is turned off. The switching panel method is easy to switch between 2D and 3D, so that a user can view a 3D stereoscopic image without wearing glasses that are separately polarized or operated with an active shutter.

5 illustrates that the parallax barrier operates in one axial direction, the present invention is not limited thereto, and a parallax barrier that may operate in two or more axial directions according to a control signal of the controller 180 may be used.

2) Lens refraction method

The lens refraction method (lenticular) is a method using a lenticular screen provided between a display part and a binocular eye, and a method of refracting a traveling direction of light through lenses on a lenticular screen so that different images are reached in both eyes.

3) polarized glasses system

Polarized light is polarized so that the directions of polarization are orthogonal to each other to provide different images in both directions or in the case of the circular polarized light so that the directions of rotation are different from each other.

4) Active shutter method

Eye image is alternately displayed at a predetermined cycle through the display unit and the user's eyeglasses are arranged such that when the image in the corresponding direction is displayed, the shutter in the opposite direction is closed, To reach the eye. That is, during the time when the left eye image is displayed, the shutter of the right eye is closed to allow the left eye image to be reached only in the left eye, and the shutter of the left eye is closed during the time when the right eye image is displayed.

For convenience of explanation, it is assumed that the mobile terminal referred to below includes at least one of the components shown in FIG. In particular, the mobile terminal to which the present invention is applicable includes a display unit capable of providing a 3D stereoscopic image to a user through at least one of the aforementioned 3D stereoscopic image implementation methods.

Meanwhile, an arrow or finger graphic for pointing a specific object (object) or selecting a menu on the display unit is called a pointer or a cursor. However, in the case of a pointer, it often means a finger or a stylus pen for a touch operation or the like. Therefore, in the present specification, the graphic image displayed on the display unit is referred to as a cursor to clearly distinguish the two, and a physical means capable of performing touch, proximity touch, and gesture such as a finger or a stylus pen is called a pointer.

Optimal 3D stereoscopic shooting and additional features

According to an embodiment of the present invention, the user visually informs the user of the distance of a subject or background for capturing a 3D stereoscopic image that may exhibit an optimal 3D effect, and compares the size in the preview image using the captured image. A method that can be performed is provided. The subject may mean an object to be focused on the camera, and the background may mean an object farther from the camera than the subject.

The operation of the mobile terminal for this purpose will be described with reference to FIG.

6 is a flowchart illustrating an example of an operation process of a mobile terminal capable of utilizing 3D stereoscopic image capturing and photographed content according to an embodiment of the present invention.

Referring to FIG. 6, at least two cameras for capturing 3D stereoscopic images are activated by a predetermined menu operation or by executing a camera application, and a preview image captured by the cameras may be displayed on the display. . Accordingly, the controller 180 may determine the distance from the subject and / or the background (S610).

The method of determining the distance from the subject / background by the controller will be described later in more detail with reference to FIG. 7.

The controller 180 may output the visual effect according to the difference between the distance for displaying the optimal 3D effect through the output unit 150 according to the determined distance to the subject / background (S620). In this case, the distance for the optimal 3D effect may be considered in consideration of the distance from the camera to the background and the distance from the camera to the subject, which may vary depending on the characteristics of the camera itself provided in the mobile terminal, and each value may be a memory ( 160 may be stored in advance. Specific visual effects will be described later in more detail with reference to FIGS. 8 to 11.

The user may change the photographing target or the environment in order to obtain an optimal 3D stereoscopic effect according to the visual effect output, or may photograph as it is. When photographing, 3D stereoscopic content may be generated (S660). Instead of taking a picture immediately, the controller 180 receives a command for selecting a specific subject from the user (S630), and the controller 180 captures an image including the subject (S640). In this case, the specific subject may be selected within the preview image displayed on the touch screen 151. In addition, the image including the subject may be generated by cropping an image of the subject from general 3D stereoscopic content. In this case, together with the image including the subject, the distance information of the subject at the time of shooting may be stored in the memory 160.

The image and the shooting distance information of the specific subject may be used in the contrast mode (S650). In this case, the contrast mode is a mode in which a preview image and a pre-stored image of a specific subject are synthesized to compare their sizes. That is, when the contrast mode is executed, the controller 180 may load an image of a specific subject stored in advance, and then place the image at a specific point of the preview image displayed on the touch screen 151. In this case, the controller 160 may obtain a distance from a specific point of the preview image and resize the image of the specific subject so as to correspond to the distance to the specific point by using distance information when the specific object is captured. . For example, when a specific subject is located at a distance of 1 m and a specific point of the preview image is also located at a distance of 1 m, the controller 180 can display an image of the subject at a corresponding point without resizing. As another example, when a particular subject is located 2m away and a specific point of the preview image is located 1m away, the controller 180 enlarges the image of the subject at a magnification converted into size when it is photographed at 1m away. I can display it.

Distance measuring method

Hereinafter, methods for measuring distance to a subject applicable to embodiments of the present invention will be described.

1. Contrast Detection Method

By moving the camera lens back and forth, it is determined that the moment where the contrast of the subject has the greatest contrast is in focus.

2. Distance sensor method

The method of measuring distance by irradiating infrared rays from at least one point to a subject and calculating triangulation or reflection arrival time.

3. Measurement method of congestion angle using two or more cameras

The constriction angle refers to the point where the eyes of the two eyes meet with respect to the human body and the angle formed by the macula of the eyes. When the eyes are replaced with a camera, the two eyes are connected from the two cameras to the subject. The angle formed by the line. If this angle is large, the subject is near, and if it is small, the subject is far. As shown in FIG. 4, in the case of d1, the congestion angle is large, and in the case of d2, the congestion angle is small. In the case of using images captured by two or more cameras, distances may be calculated by comparing the positions of the subjects in the images captured by the two cameras, or without necessarily calculating the congestion angles. This will be described with reference to FIG.

7 illustrates an example of a method of measuring a distance of a subject in a mobile terminal according to an embodiment of the present invention.

In FIG. 7, it is assumed that a spherical object is 3D photographed through two cameras provided in a mobile terminal. When the spherical object is taken at the center in the preview image, the spherical object 715 is biased on the left side from the center, and the image is captured by the left camera in the image 710 captured by the right camera as shown in FIG. In the image 720, the spherical object 725 is biased to the right. When the controller 180 overlaps the two images as shown in FIG. 7B, the controller 180 can obtain the distance d between the spherical object images 715 and 725. If the distance is far, the controller 180 may determine that the spherical object is located near, and if it is close, the controller 180 may determine that the spherical object is located far. To this end, it is preferable that the memory 160 has preliminary conversion distance information for each d according to camera characteristics (eg, focal length, angle of view, etc.).

Visual information to inform the best 3D effect

Next, a form in which the user is informed of the distance to the subject / background for the optimal 3D effect will be described with reference to FIGS. 8 to 11.

8 illustrates an example of a form in which an optimum distance from a subject is informed in a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 8, the left side shows the distance of the subject 810 to the two cameras 121-1 and 121-2 arranged in parallel in the same direction in the mobile terminal 100, and the right side has a 3D photographing function. As the is activated, a preview image of the subject 810 ′ shown on the touch screen 151 of the mobile terminal 100 is illustrated. In the left figure, it is assumed that D1 is a distance for the optimal 3D effect of the subject and the camera, and D2 is a distance for the optimal 3D effect of the background and the camera, respectively. In the case of FIG. 8, the subject 810 is disposed at a distance for an optimal 3D effect. In this case, a figure 820 indicating that the subject is located at an optimal distance is displayed on the preview image.

When the subject is not located at the optimum distance, the shape of the preview image will be described with reference to FIGS. 9A and 9B. 9A and 9B illustrate another example of a form informing an optimum distance to a subject from a mobile terminal according to an embodiment of the present invention.

Referring to the left side of FIG. 9A, the subject 810 is located closer than the optimum distance D1. In this case, in the preview image displayed on the touch screen 151, the figure 820 is smaller than the existing position 820 ', that is, the figure position in the case of a distance for an optimal 3D effect. In this case, another visual effect (for example, a dotted line as shown) may be displayed at the existing position 820 ′ of the figure for comparison of the user.

In contrast, when the subject 810 is located farther than the optimum distance D1 as shown in the left side of FIG. 9B, the figure 820 is enlarged than the existing position 820 ′ in the preview image displayed on the touch screen 151. In this case, another visual effect (for example, a dotted line as shown) may be displayed at the existing position 820 ′ of the figure for comparison of the user.

9A and 9B, the degree to which the figure is enlarged / reduced may be increased as the distance between the object and the optimum distance increases, and the direction in which the figure increases or decreases according to the distance may be set to the opposite direction. have.

Of course, the figure for showing the difference with the optimum distance described above may be replaced with other forms of visual effects that can indicate the + /-direction in a plurality of steps,

Meanwhile, instead of deformation of the figure, the optimal 3D distance may be informed in the form of a rangefinder. This will be described with reference to FIG.

FIG. 10 illustrates another example of a form of informing an optimum distance from a subject in a mobile terminal according to an embodiment of the present invention.

In FIG. 10, only the preview image is shown. Referring to FIG. 10, the distance meter 830 is displayed on the lower right side of the touch screen. On the rangefinder 830, an indicator 833 indicating a desirable distance of the subject for an optimal 3D effect and an indicator 831 indicating a point corresponding to the distance of the currently determined subject may be displayed.

Next, a method for informing a user of a background distance for an optimal 3D effect will be described with reference to FIG. 11.

11 is a diagram for one example of notifying an optimum distance from a background in a mobile terminal according to one embodiment of the present invention;

The left side of FIG. 11 shows the distance of the background 920 with respect to the two cameras 121-1 and 121-2 disposed in parallel in the same direction in the mobile terminal 100 similarly to FIG. As the photographing function is activated, a preview image of the subject 910 'and the background 930' displayed on the touch screen 151 of the mobile terminal 100 is shown. In the case of FIG. 11, the background 920 is located farther than the distance D2 for the optimal 3D effect. In this case, the frame 940 of the preview image may be reduced as shown in FIG. 11 to inform the user that the background is farther than the optimal 3D distance.

On the contrary, when the background is closer than the optimum distance, the entire preview image may flicker. The above-described visual effects of frame reduction or flickering may be applied in opposite situations, or may be replaced with text indicating that the background is far or near.

8 to 11, only the visual effects of any one of the subject and the background are displayed at a time. However, this is exemplary and the present invention is not limited thereto, and the visual effect of each of the optimum distances to the subject and the background may be simultaneously displayed. In this case, the subject and the background may need to be specified sequentially. To this end, the controller 180 may recognize points corresponding to touch inputs to two points sequentially input on the preview image as positions of a subject and a background, respectively. When determining only the optimal distance with respect to the subject, the controller 180 may determine an object located at the center of the preview image as the subject.

Select a specific subject and save the crop

Hereinafter, the selection and cropping of the subject will be described with reference to FIG. 12. 12 illustrates an example of a process of storing information including selection of a specific subject and distance information in a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 12A, a preview image in which the 3D photographing function is activated is shown. A person 1210 is displayed on the left side of the preview image, and a table 1220 is displayed on the right side farther than the person 1210. On the right side of the area where the preview image is displayed, a menu icon 1230 for executing the contrast mode, a shutter icon 1240, and a selection icon 1250 for selecting a specific subject from the preview image are displayed.

In this case, when the selection icon 1250 is touched by the user and then the portion in which the person 1210 is displayed in the preview image is touched, a border (eg, visual effect) indicating that the person 1210 is selected around the person 1210 as shown in FIG. 1260 may be displayed. In this case, the controller 180 may draw a closed curve or polygon to include an area corresponding to a portion where the 3D depth is within a predetermined range from the subject located at the selected point in setting the border, and the change point of the contrast value is a predetermined value or more. You can also draw closed curves or polygons by connecting them. Alternatively, the controller 180 may set an edge along a trajectory to which the pointer is dragged after the selection icon 1250 is touched.

When the frame is displayed, the distance 1270 of the selected subject is displayed on the right side of the area where the preview image is displayed, and the storage menu 1280 and the cancel menu 1290 are displayed in the order below. If the cancel menu is selected, it may return to the state as shown in FIG. 12 (a) or only the edge may be released. On the contrary, if only the storage menu is selected, the controller 180 can store the image inside the frame together with the distance information in the memory 160. Hereinafter, for convenience, information including distance information and a cropped image at the time of shooting is referred to as "crop data".

Contrast Mode

Hereinafter, the matching mode will be described with reference to FIG. 13. 13 illustrates an example of a process in which a matching mode is performed in a mobile terminal according to an embodiment of the present invention.

In FIG. 13, it is assumed that a plurality of crop data including a chair, a car, and a flower pot are stored in the memory 160 in advance. FIG. 13A may be displayed when an icon 1230 for executing the matching mode is selected in FIG. 12A. In detail, a preview image may be continuously displayed on the left side of the touch screen, and thumbnails 1310 to 1330 of pre-stored crop data may be displayed on the right side.

In this case, it is assumed that a user wants to place a chair at a table side point 1340 and a person side point 1350 in order to compare the sizes of the chair 1210 and the table 1220 and the chair included in the preview image, respectively. To this end, when the user selects the chair thumbnail 1310 among the thumbnails on the right side of the touch screen and touches the person side seat area 1350, the controller 180 measures the distance from the touched point 1350. The controller 180 that obtains the distance loads crop data corresponding to the chair thumbnail 1310 and retrieves the distance information and the chair image from the corresponding data. Thereafter, the controller 180 resizes the chair image to a size corresponding to a distance from the touched point 1350, and then touches the resized image 1360 on the preview image as shown in FIG. (1350).

If the user selects the table side point 1340, the controller resizes the chair image to a size corresponding to the distance to the table side point 1340 through the same process as described above, and resizes the image 1360 '. ) May be displayed at the corresponding point as shown in (c) of FIG. 13. At this time, since the table 1220 is farther than the person 1210, the resized chair image 1360 ′ is smaller in FIG. 13C than in the case of FIG. 13B. In this way, the user may resize the cropped image prepared in advance to a size corresponding to a specific position of the currently displayed preview image. As a specific application example of the present embodiment, the user may store the clothes of the department store as crop data and check whether the clothes are suitable for the family in the above-mentioned contrast mode. As another example, you can save your furniture as crop data and come home to see if the furniture can fit in a specific space or fit well with existing furniture.

On the other hand, according to another aspect of the present embodiment, the length of a specific subject may be re-viewed in the preview image without the pre-photographed image. This will be described with reference to FIG. 14A to 14C are views for explaining an example of a process of performing a length measurement mode in a mobile terminal according to another embodiment of the present invention.

The principle is first described with reference to FIG. 14A. Referring to FIG. 14A, two cameras 121-1 and 121-2 are disposed on the same plane to face the same direction in the mobile terminal 100. In this case, the angle of view A that may be captured by the two cameras is a characteristic of the camera and may be determined according to the focal length. When the angle of view is A degree, an image corresponding to the W length may be photographed at the D distance. At this time, the W value can be simply calculated if there are D and A values. The A value is a predetermined value for each focal length or for each camera characteristic, and the D value may be obtained by the controller through the above-described distance measuring method. Accordingly, the controller can calculate the W value. When the user selects the length of L in the preview image, the controller 180 calculates the L value when calculating the ratio of L to W in the corresponding image. For example, assuming that the W value is obtained by 5 meters, and L corresponds to half of the width of the preview image, L becomes 2.5 meters.

Referring to FIG. 14B, when the user wants to reconsider the length of the table 1220 in the preview image, when the user drags the pointer 1420 in the direction crossing the table top, the visual effect 1430 corresponding to the drag trajectory is displayed. do. Thereafter, the controller 180 may display the length 1440 of the trajectory 1430 calculated by the above-described calculation method on the preview image.

Meanwhile, in FIG. 6, it has been described that distance display function-> crop data generation-> contrast mode is performed in order for an optimal 3D effect, but the present invention is not limited thereto and each of the three functions may be independently performed. It may be. For example, the optimum distance display function may not necessarily be executed before generating crop data (for example, when selecting a specific subject).

Pointer recognition interface

According to another embodiment of the present invention, a mobile terminal having a user interface capable of detecting a position and distance of a user's pointer on a user interface displayed in 3D stereoscopic image and performing a function corresponding to the detected position / distance And a control method thereof.

An operation of the mobile terminal for this purpose will be described with reference to FIG. 15.

15 is a flowchart illustrating an example of an operation process of a mobile terminal for driving a user interface according to another embodiment of the present invention.

First, the controller 180 may perform a pointer recognition procedure through a predetermined menu operation or execution of an application for pointer recognition from a user (S1510). Here, the pointer recognition procedure refers to a procedure of recognizing the shape of the pointer and the size for each distance through an image captured by a camera. This will be described later in more detail with reference to FIG. 16.

When the recognition procedure is completed, the pointer detection mode may be activated (S1520). In the pointer detection mode, a pointer is extracted from an image captured by at least one camera provided in the mobile terminal, the distance from the mobile terminal is determined based on the size of the extracted pointer, and the plane coordinates of the pointer are determined by the location. Can be. The pointer detection mode may be activated when a predetermined menu manipulation from a user or an application supporting the pointer detection mode is executed or an event (for example, a message reception or a call occurrence) occurs according to a predetermined condition.

As the pointer sensing mode is activated, the controller 180 may drive the at least one camera 121 to photograph an image including the pointer continuously or at a predetermined cycle, and determine the position and distance of the pointer (S1530).

If the distance of the pointer is less than the threshold (S1540), the controller executes a function corresponding to the detected position of the pointer (S1550). As an example of the corresponding function, when there is a selection target object corresponding to the position of the detected pointer, execution of a function assigned to the object is included. That is, the control displays the selection target object on the display unit 151, and when the distance between the pointer and the mobile terminal main body and / or the selection target object that can be selected by the pointer approaches a threshold value or less, the position of the pointer is determined. A gesture for selecting a selection target object corresponding to (ie, selected through a pointer) may be determined, and a function set in the selection target object may be executed.

Here, the object to be selected is executed when a function is selected through a pointer (for example, when the pointer approaches a specific distance within a certain distance or stays at the position for a certain time). An object that can Two or more selection target objects may be provided on the display unit 151 by being given a predetermined 3D depth at different locations at the same time.

In addition, in the present specification, a point placed on the three-dimensional space of the pointer will be described by dividing the position and the distance. Here, "distance" means a degree vertically spaced apart from a reference point (display unit or object to be selected), that is, coordinates on the Z axis, and "position" means X and Y coordinates on a plane parallel to the display unit. For example, the Y-axis value increases when the pointer moves vertically upward while maintaining the contact at a point on the display unit, and the X-axis value increases when the pointer moves parallel to the right. In addition, when the pointer moves away from the plane of the display unit without parallel movement, the Z-axis value may increase.

In addition, the distance from the selection target object means a distance considering the 3D depth at which the selection target object is displayed in the 3D stereoscopic image from the user's viewpoint. For example, when the 3D depth is given so that the selection target object appears to protrude from the mobile terminal body (ie, the display unit), the user may feel that the selection target object is located closer than the mobile terminal body. Therefore, in the present specification, the distance between the main body of the mobile terminal and the object to be selected is described separately. To this end, the controller 180 may calculate the 3D depth value of the selection target object in real time, or the 3D depth value preset for each type of the selection target object may be stored in the memory 160.

Hereinafter, a pointer recognition procedure will be described with reference to FIG. 16.

16 illustrates an example of a pointer recognition procedure according to another embodiment of the present invention.

In FIG. 16, the mobile terminal 100 assumes that the display unit 151 is disposed upward, and assumes that the camera 121 is disposed to face the same direction as the display unit as shown in FIG. 2A. Referring to FIG. 16A, a pointer (here, a finger) may be photographed by a camera when positioned above the mobile terminal 100. At this time, as the distance from the distance d3 to the distance d1 from the distance d3, the size will be gradually increased as shown in the right image 1630 through the center image 1620 from the left 1610 image in Figure 16 (b). . Using this principle, when the pointer recognition procedure is activated, the controller 180 outputs a visual effect (eg, text) requesting the user to position the pointer at a predetermined distance on the display unit. Through this method, a controller that acquires an image including pointers sequentially positioned at different distances (for example, 2 cm, 5 cm, and 10 cm from the camera, respectively) may have different sizes from the corresponding images, but use a common shape as a pointer. Recognize and determine the size of each distance.

Next, an example of a method of selecting a selection target object will be described with reference to FIG. 17. 17 illustrates an example of a selection target object implemented as a 3D stereoscopic image in a mobile terminal according to another embodiment of the present invention.

Referring to FIG. 17, the controller 180 displays a layer 1710 including a plurality of icon selection objects in a 3D stereoscopic image on the display unit of the mobile terminal 100 according to the present embodiment. In this case, it is assumed that the layer 1710 is provided with a 3D effect to appear to protrude from the display unit in the eyes of the user. Here, the controller 180 can continuously acquire an image including the pointer 1720 through the camera, and determine the position and distance of the pointer using the image.

However, in this case, since the user moves the pointer in the air, it may be difficult to know where the pointer is pointing until the pointer is moved within a critical distance and the function assigned to the specific selection target object is performed. Accordingly, another aspect of the present embodiment proposes to provide a predetermined visual effect to the selection target object corresponding to the position of the pointer. This will be described with reference to FIGS. 18 and 19.

FIG. 18 illustrates an example of a visual effect given to a selection target object implemented as a 3D stereoscopic image in a mobile terminal according to a position of a pointer according to another embodiment of the present invention.

In FIG. 18, similar to the assumption of FIG. 9, it is assumed that a layer including a selection object is provided with a 3D effect to appear to protrude from the display unit in the eyes of the user. However, illustration of the main body of the mobile terminal will be omitted.

Referring to FIG. 18A, the controller 180 selects the selection target object 1811 corresponding to the position of the currently detected pointer 1820 with respect to the layer 1810 in which the plurality of selection target objects are arranged. The ratio can be enlarged and displayed. In this case, when the pointer moves to the right as shown in FIG. 18B, the controller 180 detects this and enlarges the selection target 1818 existing on the right side of the previously expanded selection target 1811. have. In this case, instead of the magnification effect, the 3D depth applied to the selection target object 1812 corresponding to the detected position of the pointer as shown in FIG. 10 (c) may be changed so as to protrude more than the layer 1810 itself. 1820 ').

Meanwhile, instead of providing a visual effect to the object to be selected, the shape of the cursor may be changed according to the distance of the pointer. This will be described with reference to FIG. 19.

19 illustrates an example in which the shape of a cursor is changed according to a distance between a selection target object and a pointer according to another embodiment of the present invention.

In FIGS. 19A and 19B, the controller 180 may move a finger-shaped cursor 1910 on a layer on which a selection target object is disposed with respect to the vertical movement of the pointer. In this case, the shape of the finger-shaped cursor may change as the pointer becomes closer to the layer including the mobile terminal body or the object to be selected. For example, in FIG. 19A, the finger changes from 10 o'clock to 12 o'clock as the pointer approaches, and in FIG. 19B, the index finger bent as the pointer approaches is gradually extended. You can visually feed back the distance of the pointer. In addition, as shown in FIG. 19C, when the pointer 1930 is within a predetermined distance of the layer including the main body of the mobile terminal or the object to be selected, the controller 180 corresponds to a position of the translucent film ( 1920 3D stereoscopic image, and when the pointer gets closer, the pointer bends as if the membrane is pressed (1920 '), and if it gets closer, the film tearing effect (1920 ") can be displayed in 3D stereoscopic image. In each of the examples of FIG. 19, the rightmost visual effect may be used as a visual effect to indicate that a selection target object is selected, that is, a function assigned to the selection target object is executed.

17 to 19 illustrate that a predetermined 3D depth is provided to a layer including the selection target object. However, the present disclosure is not limited thereto and may be applied to the selection target object displayed as a planar image.

Further, according to an embodiment of the present invention, the above-described method can be implemented as a code that can be read by a processor on a medium on which the program is recorded. Examples of the medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of a carrier wave (e.g., transmission over the Internet) .

The above-described mobile terminal and its control method are not limited to the configuration and method of the above-described embodiments, but the embodiments may be modified such that all or some of the embodiments are selectively And may be configured in combination.

Claims (20)

A first camera and a second camera for photographing at least one common object;
A display unit configured to display a preview image by using images captured by the first camera and the second camera; And
The first distance is determined from a first object of the at least one common object by using the images acquired from the first camera and the second camera, and the first distance is different from the preset second distance. And a controller for obtaining a value and controlling a visual effect corresponding to the difference value to be displayed on the display unit. The method of claim 1,
The control unit,
And displaying the visual effect in a plurality of stages according to the magnitude and direction of the difference value. The method of claim 1,
The first object comprising:
The mobile terminal, characterized in that the object located in the center of the preview image. The method of claim 1,
Further comprising a user input unit for receiving a command from the user,
And the first object is an object selected according to a first command input through the user input unit. 5. The method of claim 4,
Include more memory,
The control unit,
And when the second command is input through the user input unit, an image corresponding to the first object and the first distance from the preview image are stored together in the memory. The method of claim 1,
Further comprising a memory unit for storing at least one object information including the image and the distance information when the image was taken,
The control unit,
When a third command is input through the user input unit, a thumbnail of an image included in each of the at least one object information is displayed on the display unit. When a fourth command is input through the user input unit, the at least one object information is displayed. And controlling to display an image corresponding to specific object information on the preview image. The method according to claim 6,
The control unit,
And when the second object is determined through the fifth command through the user input unit in the preview image, controlling to display an image corresponding to the specific object information at the position of the second object. 8. The method of claim 7,
The control unit,
And determining a third distance from the second object, and adjusting the size of an image corresponding to the specific object information by comparing the third distance with distance information at the time of capturing the specific object information. The method of claim 1,
The control unit,
The distance is determined by using a distance between a first coordinate at which the first object is located in the first image photographed by the first camera and a second coordinate at which the first object is located in the second image photographed by the second camera. And control to determine the first distance. The method of claim 1,
The first camera and the second camera,
The mobile terminal, characterized in that arranged in the same direction on one plane of the main body of the mobile terminal. 5. The method of claim 4,
And the display unit and the user input unit integrally configure a touch screen. The method of claim 1,
The second distance may be a distance that maximizes the stereoscopic feeling of the first object when implementing a binocular parallax type 3D stereoscopic image using the images captured by the first camera and the second camera as a source image. Mobile terminal. Activating a first camera and a second camera for photographing at least one common object;
Generating a preview image by using images captured by the first camera and the second camera, and displaying the preview image on a display unit;
Determining a first distance from a first object of the at least one common object using an image obtained from the first camera and the second camera; And
And obtaining a difference value between the first distance and the second distance, and displaying a visual effect corresponding to the difference value on the display unit. A camera for taking an image of a pointer; And
The size of each pointer is determined by using a first image photographed when the pointer is spaced apart from the camera by a first distance and a second image photographed when the second distance is separated by a distance, and the image is input from the camera at predetermined intervals. And a controller configured to detect the pointer to determine the position and the separation distance of the pointer. The method of claim 14,
Including a parallax generating means, and further comprising a display unit for displaying at least one selected target object given a predetermined three-dimensional depth,
The control unit,
And when the separation distance is equal to or less than a first threshold value and a specific selection target object exists at the location, controlling a function corresponding to the first selection target object to be executed. 16. The method of claim 15,
The separation distance is a vertical distance between the pointer and the mobile terminal, wherein the position is a coordinate on a plane parallel to the touch screen. 17. The method of claim 16,
The control unit,
And a predetermined visual effect is displayed on the touch screen in response to the detected position and distance of the pointer. 18. The method of claim 17,
The predetermined visual effect is a cursor of a predetermined shape,
The control unit,
A mobile terminal, characterized in that for rotating the cursor or deform the cursor in accordance with the distance of the pointer. Capturing a first image through the camera, the pointer having been separated from the camera by a first distance;
Photographing a second image when the pointer is spaced apart from the camera by a second distance through the camera;
Determining a distance-specific size of the pointer using the first image and the second image; And
And detecting the pointer from an image input at a predetermined period from the camera to determine a position and a separation distance of the pointer. 20. The method of claim 19,
Displaying at least one selection target object given a predetermined three-dimensional depth through a display unit including parallax generating means;
And executing the function corresponding to the first selection target object when the separation distance is less than or equal to a first threshold value and a specific selection target object exists in the location.

Images