KR20130084768A - Terminal and method for controlling the same - Google Patents

Abstract

PURPOSE: A terminal and a control method thereof are provided to convert an image into a 3D image and to reproduce the 3D image in reference to script information without recognizing the 3D effect characteristic of an image. CONSTITUTION: A control unit (180) converts an image into a 3D image by analyzing the characteristic of 3D effect which is obtained in each frame of the image when the image is initially converted into the 3D image. The control unit generates script information including the analyzed characteristic of the 3D effect in each frame. The control unit stores the generated script information in a memory. The control unit regenerates the image after converting the image into the 3D image based on the characteristic of the 3D effect for each frame within the stored script information. [Reference numerals] (110) Power supplying unit; (111) Broadcasting reception module; (112) Mobile communication module; (113) Wireless Internet module; (114) Local area communication module; (115) Location information module; (120) A/V input unit; (121) Camera; (122) Microphone; (130) User input unit; (140) Sensing unit; (141) Proximity sensor; (150) Output unit; (151) Display unit; (152) Sound output module; (153) Alarm unit; (154) Haptic module; (155) Projector module; (160) Memory; (170) Interface unit; (180) Control unit; (181) Multimedia module; (190) Power supplying unit

Description

[0001] TERMINAL AND METHOD FOR CONTROLLING THE SAME [0002]

The present invention relates to a terminal and a method of controlling the same so that the use of the terminal can be implemented in consideration of user convenience.

Terminals such as digital TVs, personal computers, laptops, mobile phones, etc. may be configured to perform various functions. Examples of such various functions include a data and voice communication function, a function of photographing a video or a moving image through a camera, a voice storage function, a music file playback function through a speaker system, and an image or video display function. Some terminals include additional functions to execute games, and some other terminals are also implemented as multimedia devices. Moreover, recent terminals can receive a broadcast or multicast signal to view a video or television program.

In general, the terminal is movable The mobile terminal can be divided into a mobile terminal and a stationary terminal depending on whether the mobile terminal is portable or not, and the mobile terminal can be divided into a handheld terminal and a vehicle mount terminal.

As described above, terminals such as a digital TV, a notebook computer, a PC, a mobile phone, and a smart phone have a function of playing 3D content recently in response to a request for providing various multimedia functions by a user.

That is, a terminal having a 3D playback function, when converting and playing back a 3D convertible image stored by an external 3D content providing server or a user to 3D, provides a characteristic of a 3D effect to be applied to each frame of the image in real time. After the analysis, each frame is sequentially 3D converted and reproduced based on the characteristics of the analyzed 3D effect.

That is, the conventional terminal analyzes and converts 3D effect characteristics in real time each time for each frame of the image whenever the image is converted to 3D and reproduced.

Therefore, the conventional terminal requires a separate 3D conversion chipset for analyzing the characteristics of the 3D effect of the image, and every time the image is converted to 3D and played back, the characteristic of the 3D effect of the image is analyzed every time. Since the conversion is performed, the loading time is long when the image is converted to 3D.

An object of the present invention is to determine the 3D effect characteristics to be applied to each frame of the image in real time after converting the 3D switchable image to the first 3D, and then the script information including the 3D effect characteristics for each frame identified. The present invention provides a terminal and a control method of converting and reproducing the image in 3D by referring to the script information only without grasping the 3D effect characteristic of the image when the image is converted back to 3D. have.

According to an aspect of the present invention, there is provided a terminal including: a memory including at least one image capable of 2D (Dimensional) and 3D conversion;

A display unit which displays a 2D playback screen or a 3D playback screen of the video; By converting the image to 3D by real-time analysis of the characteristics of the 3D effect to be given for each frame of the image when the image is first converted to 3D, by generating script information including the characteristics of the 3D effect of each analyzed frame And a controller configured to store the image in the memory and convert the image into 3D based on the characteristics of the 3D effect for each frame in the stored script information when the image is converted back to 3D.

In addition, the control method of the terminal according to the present invention comprises the steps of: real-time analysis of the characteristics of the 3D effect to be given to each frame of the image, when the first 3D conversion command of the image capable of 2D (Dimensional) and 3D conversion; Converting the image to 3D based on a characteristic of the 3D effect of each frame analyzed in real time; Generating script information including a characteristic of a 3D effect of each frame analyzed in real time; Storing the generated script information in a memory; And converting the image to 3D based on a characteristic of a 3D effect for each frame in the stored script information when the image is converted back to 3D.

The terminal and its control method according to the present invention, after converting the 3D effect characteristics of the image in real time when converting the image to the first 3D, generates and stores script information including the analyzed 3D effect characteristics, the image When playing back in 3D, by switching only by referring to the script information without the process of analyzing the real-time 3D effect characteristics of the image, it provides an effect that can reduce the loading time when the image is converted back to 3D.

1 is a block diagram showing a terminal related to the present invention.
Figure 2a is a front perspective view of an example of a terminal related to the present invention.
FIG. 2B is a rear perspective view of the terminal shown in FIG. 2A.
3 is a conceptual diagram for explaining the principle of binocular disparity.
4 is a conceptual diagram for explaining a sense of distance and 3D depth due to binocular parallax.
FIG. 5 is a conceptual diagram illustrating a method of implementing 3D stereoscopic images in a display barrier type display unit applicable to the present invention.
6 is a flowchart illustrating a process of generating script information used for 3D conversion of an image according to the present invention.
7 is a diagram for explaining script information according to the present invention.
8 is a flowchart illustrating a process of converting a corresponding video into 3D using script information according to the present invention.
9 to 11 are diagrams for explaining a process of converting a corresponding video into 3D using script information according to the present invention.

Hereinafter, a terminal related to the present invention will be described in detail with reference to the accompanying drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The terminal described herein includes a mobile phone, a smart phone, a laptop computer, a desktop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, a digital TV, and the like. May be included.

Hereinafter, a terminal according to the present specification will be described as one of a smart phone, a mobile phone, and a digital TV having a 3D function, but the type of the terminal according to the present specification is not limited thereto. In other words, any device having a 3D function may be a terminal according to the present specification.

1 is a block diagram of a terminal according to an embodiment of the present invention.

The terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory unit 160, and an interface unit. 170, the controller 180, the battery 190, and the like. The components shown in Fig. 1 are not essential, and a portable terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the terminal 100 and the wireless communication system or between the terminal 100 and the network in which the terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

For example, the broadcast receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S), a Media Forward Link Only A digital broadcasting system such as DVB-CB, OMA-BCAST, or Integrated Services Digital Broadcast-Terrestrial (ISDB-T). Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the terminal 100. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication module 114 refers to a module for short range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

The location information module 115 is a module for obtaining a location of a terminal, and a representative example thereof is a GPS (Global Position System) module. According to the current technology, the GPS module 115 calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to obtain a three-dimensional string of latitude, longitude, The location information can be accurately calculated. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module 115 can calculate speed information by continuously calculating the current position in real time. Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [

At this time, two or more cameras 121 may be provided depending on the use environment.

For example, the camera 121 may be provided with first and second cameras 121a and 121b for capturing 3D images on the opposite side of the display unit 151 of the terminal 100 and the terminal ( The third camera 121c may be provided at a portion of the surface of the display unit 151 of the 100 for self photographing by the user.

In this case, the first camera 121a is for capturing the left eye image, which is the source image of the 3D image, and the second camera 121b, for the right eye image capturing.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal.

The user input unit 130 may receive a signal from a user that designates two or more objects among the objects displayed on the display unit 151. The signal specifying two or more objects may be received through a touch input or may be received through a hard key and a soft key input.

The user input unit 130 may receive an input for selecting one or more objects from a user. In addition, an input for generating an icon related to a function that the terminal 100 can perform may be received from a user.

The user input unit 130 may include a directional keypad, a keypad, a dome switch, a touchpad (static / static), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the terminal 100 such as the open / close state of the terminal 100, the position of the terminal 100, the presence of the user, the orientation of the terminal, And generates a sensing signal for controlling the operation of the sensor. For example, when the terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. It is also possible to sense whether the battery 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor 141. The proximity sensor 141 will be described later in relation to the touch screen.

The output unit 150 is for generating an output relating to visual, auditory or tactile sense and includes a display unit 151, an acoustic output module 152, an alarm unit 153, a haptic module 154, 155, and the like.

The display unit 151 displays (outputs) information to be processed by the terminal 100. For example, when the terminal is in the call mode, a UI (User Interface) or GUI (Graphic User Interface) associated with the call is displayed. When the terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

In addition, the display unit 151 according to the present invention supports 2D and 3D display modes.

That is, the display unit 151 according to the present invention may have a configuration in which the switch liquid crystal 151b is combined with a general display device 151a as shown in FIG. 5 below. 5 (a), the optical parallax barrier 50 is operated by using the switch liquid crystal 151b to control the traveling direction of the light, so that different lights can be separated from the left and right eyes. Therefore, when a combined image of the right eye image and the left eye image is displayed on the display device 151a, the user can see the image corresponding to each eye and feel as if it is displayed in three-dimensional form.

That is, under the control of the control unit 180, the display unit 151 drives only the display device 151a without driving the switch liquid crystal 151b and the optical parallax barrier 50 in the 2D display mode Performs normal 2D display operation.

The display unit 151 drives the switch liquid crystal 151b and the optical parallax barrier 50 and the display device 151a under the control of the controller 180 to display the display liquid crystal 151b, 151a to perform the 3D display operation.

The 3D display process of the display unit 151 will be described later in detail with reference to FIGS. 3 to 5.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) A flexible display, and a three-dimensional display (3D display).

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the terminal 100. For example, in the terminal 100, a plurality of display portions may be spaced apart from one another or may be disposed integrally with each other, or may be disposed on different surfaces.

When the display unit 151 and a sensor for detecting a touch operation (hereinafter, referred to as a touch sensor) form a mutual layer structure (hereinafter referred to as a touch screen), the display unit 151 may be configured in addition to an output device. Can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller (not shown). The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. As a result, the controller 180 can know which area of the display unit 151 is touched.

The proximity sensor 141 may be disposed in an inner area of the portable terminal or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching the pointer on the screen is called "contact touch." The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the terminal 100. [ The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the terminal 100. Examples of events that occur in a terminal include receiving a call signal, receiving a message, inputting a key signal, and touch input. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. In this case, the display unit 151 and the audio output module 152 may be a type of the alarm unit 153. The display unit 151 and the audio output module 152 may be connected to the display unit 151 or the audio output module 152, .

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to vibration, the haptic module 154 may be configured to provide a pin array that vertically moves with respect to the contact skin surface, a jetting force or suction force of air through an injection or inlet port, grazing to the skin surface, contact of an electrode, electrostatic force, and the like. Various tactile effects can be generated, such as effects by the endothermic and the reproduction of a sense of cold using the elements capable of endotherm or heat generation.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. Two or more haptic modules 154 may be provided according to a configuration aspect of the terminal 100.

The projector module 155 is a component for performing an image project function using the terminal 100 and is identical to an image displayed on the display unit 151 according to a control signal of the controller 180. At least some of the other images may be displayed on an external screen or wall.

Specifically, the projector module 155 includes a light source (not shown) that generates light (for example, laser light) for outputting an image to the outside, a light source And a lens (not shown) for enlarging and outputting the image at a predetermined focal distance to the outside. Further, the projector module 155 may include a device (not shown) capable of mechanically moving the lens or the entire module to adjust the image projection direction.

The projector module 155 can be divided into a CRT (Cathode Ray Tube) module, an LCD (Liquid Crystal Display) module and a DLP (Digital Light Processing) module according to the type of the display means. In particular, the DLP module may be advantageous for miniaturization of the projector module 151 by enlarging and projecting an image generated by reflecting light generated from a light source on a DMD (Digital Micromirror Device) chip.

Preferably, the projector module 155 may be provided in the longitudinal direction on the side, front or back of the terminal 100. Of course, the projector module 155 may be provided at any position of the terminal 100 as necessary.

The memory 160 may store a program for processing and controlling the controller 180 and may store the input / output data (e.g., a telephone directory, a message, an audio, a still image, an electronic book, History, and the like). The memory 160 may also store the frequency of use of each of the data (for example, each telephone number, each message, and frequency of use for each multimedia). In addition, the memory unit 160 may store data on vibration and sound of various patterns output when the touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.) ), A random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read- A magnetic disk, an optical disk, a memory, a magnetic disk, or an optical disk. The terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the terminal 100 or transmits data to the external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip that stores various information for authenticating the use authority of the terminal 100, and includes a user identification module (UIM), a subscriber identify module (SIM), and a universal user authentication module (Universal). Subscriber Identity Module, USIM) and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit may be a passage through which power from the cradle is supplied to the portable terminal 100 when the terminal 100 is connected to an external cradle, or various command signals input from the cradle by a user to the portable terminal. It can be a passage to be delivered. Various command signals input from the cradle or the power source may be operated as a signal for recognizing that the terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the terminal 100. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions. The described embodiments may be implemented by the controller 180 itself.

According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code can be implemented in a software application written in a suitable programming language. The software code is stored in the memory 160 and can be executed by the control unit 180. [

Figure 2a is a front perspective view of an example of a terminal related to the present invention.

The disclosed terminal 100 has a bar-shaped body. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body includes a case (a casing, a housing, a cover, and the like) which forms an appearance. In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injecting synthetic resin or may be formed of a metal material, for example, a metal material such as stainless steel (STS) or titanium (Ti).

The mobile terminal body, mainly the front case 101, includes a display unit 151, an audio output unit 152, a third camera 121c, a user input unit 130/131, 132, a microphone 122, an interface 170, and the like. Can be arranged.

The display unit 151 occupies most of the main surface of the front case 101. A sound output unit 151 and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151 and a user input unit 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132 and the interface 170 may be disposed on the side surfaces of the front case 101 and the rear case 102. [

The user input unit 130 is manipulated to receive a command for controlling the operation of the terminal 100 and may include a plurality of manipulation units 131 and 132. The operation units 131 and 132 may be collectively referred to as a manipulating portion.

The content input by the first or second manipulation units 131 and 132 may be variously set. For example, the first operation unit 131 receives commands such as start, end, scroll, and the like, and the second operation unit 132 controls the size of the sound output from the sound output unit 152 or the size of the sound output from the display unit 151 The touch recognition mode can be activated or deactivated.

The third camera 121c may be a camera having a photographing direction substantially opposite to the first and second cameras 121a and 121b and having the same or different pixels as the first and second cameras 121a and 121b. have.

The flash 123 and the mirror 124 may be further disposed adjacent to the third camera 121c. The flash 123 shines light toward the subject when the subject is photographed by the third camera 121c. The mirror 124 allows the user to see his / her own face or the like when photographing (self-photographing) the user using the third camera 121c.

FIG. 2B is a rear perspective view of the terminal shown in FIG. 2A.

Referring to FIG. 2B, first and second cameras 121a and 121b for capturing 3D images may be additionally mounted on the rear of the terminal body, that is, the rear case 102.

In this case, the first and second cameras 121a and 121b may capture a general 2D image in addition to the 3D image function.

The first and second cameras 121a and 121b have a photographing direction substantially opposite to the third camera 121c and may be cameras having the same or different pixels as the third camera 121c.

For example, the third camera 121c has a low pixel so that the user's face is photographed and transmitted to the counterpart in a video call or the like, and the first and second cameras 121a and 121b may use a general subject. It is desirable to have a high pixel because it is often photographed and not immediately transmitted.

The sound output module 152 'may be further disposed on the rear side of the portable terminal body. The sound output unit 152 ′ may implement a stereo function together with the sound output module 152 (see FIG. 2A), and may be used to implement a speakerphone mode during a call.

In addition to the antenna for a call or the like, an antenna 116 for receiving a broadcast signal may be additionally disposed on the side of the portable terminal body. The antenna 116 constituting a part of the broadcast receiving module 111 (refer to FIG. 1) may be installed to be pulled out from the terminal body.

The terminal body is equipped with a power supply unit 190 for supplying power to the terminal 100. The power supply unit 190 may be built in the terminal body or may be directly detachable from the outside of the terminal body.

The rear case 102 may further include a touch pad 135 for sensing a touch. The touch pad 135 may be of a light transmission type for the display unit 151. [ In this case, if the display unit 151 is configured to output the time information on both sides (i.e., in the directions of both the front and back sides of the mobile terminal), the time information can be recognized through the touch pad 135 do. The information output on both sides may be all controlled by the touch pad 135. [

Meanwhile, the display for exclusive use of the touch pad 135 may be separately installed, so that the touch screen may be disposed in the rear case 102 as well.

The touch pad 135 operates in association with the display unit 151 of the front case 101. The touch pad 135 may be disposed parallel to the rear of the display unit 151. The touch pad 135 may have a size equal to or smaller than that of the display unit 151.

Hereinafter, a control process related to 3D conversion of an image of a terminal applicable to embodiments of the present invention will be described.

Stereoscopic images that may be implemented on the display unit 151 of the terminal 100 may be largely classified into two categories. The criterion for this classification is whether or not different images are provided in both eyes.

First, the first stereoscopic image category will be described.

The first category is a method in which the same image is provided in both eyes (monoscopic), which can be implemented as a general display unit. More specifically, the controller 180 arranges a polyhedron generated through one or more points, lines, planes, or a combination thereof in a virtual three-dimensional space, and displays an image viewed from a specific viewpoint on the display unit 151. That's how. Therefore, the substantial part of such a stereoscopic image can be referred to as a planar image.

The second category is a stereo scopic method in which different images are provided in both eyes. It is a method using a principle of feeling a three-dimensional feeling when a human is looking at an object with the naked eye. In other words, the two eyes of a person see different plane images when they see the same thing by the distance between them. These different planar images are transmitted to the brain through the retina, and the brain fuses them to feel the depth and reality of the stereoscopic image. Therefore, binocular disparity due to the distance between the two eyes makes a sense of stereoscopic effect, and this binocular disparity becomes the most important element of the second category. This binocular disparity will be described in more detail with reference to Fig.

3 is a conceptual diagram for explaining the principle of binocular disparity.

In FIG. 3, it is assumed that the cube 31 is viewed with the naked eye in front of the eye level. In this case, the left eye planar image 32 is seen in which only three surfaces of the top, front, and left sides of the hexahedron 31 are visible. In addition, as the right eye, the right eye planar image 33 showing only three surfaces of the top, front, and right sides of the cube 31 is visible.

If the left eye plane image 32 is reached in the left eye and the right eye plane image 33 is reached in the right eye, even if the object is not actually an object in front of the eye, the person may feel as if the person actually sees the cube 31.

As a result, in order to implement the stereoscopic image of the second category in the terminal 100, the left eye image and the right eye image, which have been seen with the same object with a predetermined time difference, must be reached through the display unit. Next, the 3D depth due to binocular disparity will be described with reference to FIG. 4.

4 is a conceptual diagram for explaining a sense of distance and 3D depth due to binocular parallax.

Referring to FIG. 4, when the hexahedron 40 is viewed at a distance d1 through both eyes, the lateral gravity of an image that enters each eye is relatively higher than that of the hexahedron 40 at a d2 distance, The difference is also great. In addition, the degree of three-dimensional feeling that a person feels when viewing the cube 40 at a distance d1 is greater than when the cube 40 is viewed at a distance d2. In other words, when a person looks at an object through both eyes, the closer to the object, the greater the three-dimensional feeling, and the farther the object, the less the three-dimensional feeling.

This difference in stereoscopic dimensions can be quantified to 3D depth or 3D level.

Next, an implementation method of the 3D stereoscopic image will be described.

As described above, in order to realize a 3D stereoscopic image, it is necessary that the right eye image and the left eye image are divided into two and reach the binocular. Various methods for this are described below.

1) Parallax barrier method

The parallax barrier method is a method of controlling a light propagation direction by electronically driving a blocking device provided between a general display unit and both eyes so that different images may be reached in both eyes.

This will be described with reference to FIG.

FIG. 5 is a conceptual diagram illustrating a method of implementing 3D stereoscopic images in a view barrier type display unit applicable to embodiments of the present invention.

The structure of the display barrier type display unit 151 for displaying a 3D stereoscopic image may have a configuration in which the switch liquid crystal 151b is combined with a general display device 151a. Using the switch liquid crystal 151b, the optical parallax barrier 50 may be operated as shown in FIG. 5 (a) to control the traveling direction of the light to separate the light to reach the left and right eyes. Therefore, when an image in which a right eye image and a left eye image are combined is displayed on the display device 151a, the image corresponding to each eye is seen from the user's point of view as if it is displayed in three dimensions.

In addition, as shown in FIG. 5B, the parallax barrier 50 by the switch liquid crystal is electrically controlled so that all of the light is transmitted, thereby eliminating separation of the light by the parallax barrier so that the same image can be seen by the left and right eyes. It may be. In this case, the same function as that of the general display unit may be performed.

5 illustrates that the parallax barrier moves in parallel in one axial direction, but the present invention is not limited thereto, and a parallax barrier capable of moving in parallel in two or more axial directions according to a control signal of the controller 180 may be used. have.

2) Lens refraction method

The lens refraction method (lenticular) is a method using a lenticular screen provided between a display part and a binocular eye, and a method of refracting a traveling direction of light through lenses on a lenticular screen so that different images are reached in both eyes.

3) polarized glasses system

Polarized light is polarized so that the directions of polarization are orthogonal to each other to provide different images in both directions or in the case of the circular polarized light so that the directions of rotation are different from each other.

4) Active shutter method

Eye image is alternately displayed at a predetermined cycle through the display unit and the user's eyeglasses are arranged such that when the image in the corresponding direction is displayed, the shutter in the opposite direction is closed, To reach the eye. That is, during the time when the left eye image is displayed, the shutter of the right eye is closed to allow the left eye image to be reached only in the left eye, and the shutter of the left eye is closed during the time when the right eye image is displayed.

It is assumed that the portable terminal according to an embodiment of the present invention described below can provide a 3D image to a user through the display unit 151 through any one of the above-described methods.

However, since the principle of the 3D image described above with reference to FIGS. 4 and 5 is a situation that assumes a three-dimensional object, the shape of the object is different in the left eye image and the right eye image. However, in the case of a planar object instead of a three-dimensional object, the shape of the object is the same in the left eye image and the right eye image. However, if the position where the object is arranged in each image is different, the user may feel the perspective of the object. In the present specification, for the sake of understanding, it is assumed that the stereoscopic image appearing below is a plane object. Of course, it is apparent that the present invention can be applied to a three-dimensional object.

6 to 11, a process of converting an image to 3D using script information according to the present invention will be described in detail.

First, referring to FIGS. 6 and 7, a process of generating script information used for 3D conversion of an image will be described in detail.

6 is a flowchart illustrating a process of generating script information used for 3D conversion of an image according to the present invention.

7 is a diagram for explaining script information according to the present invention.

6 and 7, when the first 3D playback command is input to an image capable of 2D and 3D switching, the controller 180 of the terminal 100 starts a 3D switching operation of the image [S110].

In this case, the left eye and right eye images for each frame are included in the image, and the controller 180 starts 3D conversion for each frame using the left eye and right eye images for each frame.

The detailed 3D switching operation of the image will be described in detail in the following S120 and S130 processes.

The image may be generated using left and right eye images input through the first and second cameras 121a and 121b, may be an image provided by default in the memory 160, or may be an interface unit ( 170 may be downloaded from an external device / server connected via wired / wireless connection through the wireless communication unit 110.

When the 3D switching operation of the image is started, the controller 180 sequentially analyzes the characteristics of the 3D effect to be applied to each frame of the image in real time [S120], and when the 3D effect characteristic is analyzed for each frame in real time. By sequentially giving the analyzed 3D effect characteristics to each frame, the image is converted to 3D and played back [S130].

That is, when the 3D conversion operation of the image is started, the controller 180 recognizes an object and a background included in each frame of the image, and the 3D conversion rule defined in the 3D conversion program provided in the memory 160 ( According to the rule), the 3D effect is applied to the object and the background included in each frame to convert and play the image in 3D.

When the 3D conversion of the image is completed, the controller 180 determines a characteristic of the 3D effect applied to each frame of the image converted to 3D according to the present invention, and is applied to each frame of the identified image. The script information including the characteristics of the 3D effect is generated [S140], and the generated script information is stored in the memory 160 [S150].

Subsequently, according to the present invention, when the controller 180 converts the image back to 3D and plays back the image, the controller 180 processes steps S140 and S150 without analyzing a 3D effect characteristic of each frame of the image in steps S120 and S130. The image is converted into 3D and reproduced by referring to the script information stored by the.

That is, the script information is annotation information indicating the characteristic of the 3D effect applied to each frame of the image, and the controller 180 already has the 3D effect characteristic applied to each frame of the image in the script information. Therefore, when the image is converted back to 3D, the image may be converted back to 3D using only the script information without analyzing the 3D effect characteristic of each frame of the image.

In addition, the controller 180 generates the script information in the form of a standardized XML document, compresses the generated script information into a Binary Code for Metadata (BiM) stream, and stores the additional script information in an MP4 file container according to the MPEG-A standard. Can be.

The script information may include information of the image, frame information of the image, object information of each frame, background image information of each frame, and the like.

The information of the image includes a category (movie, drama, sports, etc.) of the image and a genre of the category of the image (for example, if the category is a movie, the genre is an action, drama, etc.), and the controller 180 When the image is later converted back to 3D, the image may be assigned a 3D effect optimized for the category and genre of the image described in the script information. In this case, a 3D effect value optimized for each category and genre of the image is provided in the memory 160, and the controller 180 controls the category and genre of the image described in the script information in the memory 160. The 3D effect value is searched for and the searched 3D effect value is assigned to the image.

In addition, each frame information of the image includes a number for each frame of the image, and a 3D effect (including a period during which the level / 3D effect of the 3D effect type / 3D effect is maintained, etc.) assigned to each frame, and the like. The controller 180 provides a 3D effect to each frame of the image by referring to the frame information when the image is later converted to 3D.

In addition, the object information for each frame is information about an object included in each frame, and includes IDs of objects included in each frame, display positions of objects in each frame, and categories of objects in each frame. (A person, an animal, an object, and the like) and 3D effects applied to objects in each frame, and the controller 180 refers to the object information for each frame information when converting the image to 3D later. The object in each frame of the image is converted into 3D.

In addition, the background image information for each frame is information on a background image included in each frame, the ID of the background image included in each frame, the display position of the background images in each frame, A category of background images in each frame, and a 3D effect applied to the background images in each frame, and the controller 180 displays background image information for each frame information when converting the image into 3D later. For reference, the background images in each frame of the image are converted into 3D.

The controller 180 may store the generated script information in the memory 160 in operation S150 and include the script information in the attribute information of the image. When the script information is included and stored in the attribute information of the video, the controller 180 displays information representing the script information in an icon or a file name of the video, so that the script information used for 3D conversion of the video to the user is displayed. To inform the fact that it exists. When the script information is stored in the attribute information of the image, the controller 180 converts the image into 3D based on the script information included in the attribute information of the image when the image is converted into 3D again. can do.

In addition, the controller 180 stores the script information in the form of a file in the memory 160 when the generated script information is stored in the memory 160 in operation S150, and the image and the script information are mapped and stored. do. Subsequently, when the image is converted back to 3D, the controller 180 searches for script information mapped to the image in the memory 160 and converts the image to 3D based on the found script information.

FIG. 7A illustrates an identifier indicating the script information 320 in an icon of the image 310 when the controller 180 stores the script information 320 in the attribute information of the image 310. It is shown as including.

FIG. 7B illustrates the script information 320 in the file name of the image 310 when the controller 180 stores the script information 320 in the attribute information of the image 310. Indicates that it contains an identifier.

FIG. 7C illustrates that when the controller 180 stores the script information 320 in a file form separately from the image 310, the generated script file 320 includes the image 310. It shows what was displayed in the list.

6 and 7 have been described in detail with respect to the script information generation process according to the present invention.

Hereinafter, a process of converting the corresponding image back to 3D using the generated script information will be described in detail with reference to FIGS. 8 to 11.

8 is a flowchart illustrating a process of converting a corresponding video into 3D using script information according to the present invention.

9 to 11 are diagrams for explaining a process of converting a corresponding video into 3D using script information according to the present invention.

First, referring to FIG. 8, when a 3D switching command of the image is input, the controller 180 starts a 3D switching operation of the image [S210], and a script to be referred to the 3D switching of the image to the memory 160. Determine whether information exists [S220].

If the script information exists [S230], the controller 180 determines a 3D effect characteristic of each frame of the image described in the script information [S240], and refers to the 3D effect characteristic of each of the identified frames. The video is converted into 3D and played back [S250].

That is, when converting the image back to 3D, the controller 180 converts the image to 3D using only the stored script information without performing an analysis operation on each frame of the image in real time.

In addition, the script information provided in the memory 160 may be generated by the terminal 100 using the processes of FIGS. 6 and 7 described above, or may be provided with a counterpart terminal or an external script providing server through the wireless communication unit 110. It may be downloaded from.

The script information may be shared among terminals in the form of a standardized XML document, and a terminal having an image corresponding to the script information may acquire the script information through another terminal or an external script providing server and obtain the It is possible to 3D convert the image using only the script information. In other words, the script information can be shared and used as if the subtitle file of the existing video is shared.

In addition, if the script information does not exist in the memory 160, the controller 180 analyzes in real time a 3D effect characteristic to be applied to each frame of the image as described above with reference to FIG. 6 [S260]. The image is converted into 3D based on the 3D effect characteristic of the frame [S270].

Meanwhile, the controller 180 determines whether the state of the terminal 100 is capable of 3D conversion and playback of the image before converting the image to 3D by referring to the script information of S240 and S250, and according to the result of the determination. The 3D conversion of the image may be canceled or all or a part of each frame in the image may be converted into 3D and reproduced.

For example, the state of the terminal 100 may be a remaining capacity of a battery, and the controller 180 may grasp 3D effect characteristics of each frame described in the script information, and determine the identified 3D effect characteristics. After calculating the battery usage to be consumed when the frame is given, it is determined whether the calculated usage exceeds the remaining capacity of the battery.

If the calculated amount exceeds the remaining capacity of the battery, the controller 180 reproduces the image in 2D without converting to 3D. In addition, if the calculated amount of usage does not exceed the remaining capacity of the battery, the controller 180 converts the image to 3D using the script information. In addition, if the calculated amount of usage does not exceed the remaining capacity of the battery but is close to the controller 180, the controller 180 uses the script information until the remaining capacity of the battery is consumed and the battery becomes a low voltage state. Toggle 3D for each frame.

9A illustrates that a multimedia list including the first image 310 provided with the script information 320 is displayed.

In this case, when the first image 310 is selected, the controller 180 searches for the script information 320 corresponding to the first image in the memory 160, as shown in FIG. 9B. The first image 310 is converted into 3D and reproduced using the retrieved script information 320.

In this case, the controller 180 may display identification information 330 indicating that the first image 310 has been 3D converted based on the script information 320 on the 3D playback screen of the first image 310. have.

Next, as shown in (a) of FIG. 10, when the script information 320 corresponding to the first image 310 is selected, the controller 180 selects (b) and (c) of FIG. 10. As shown in FIG. 2, the script information 320 may be transmitted to an external terminal or an external script information sharing server.

That is, as shown in (b) of FIG. 10, the contact input window of the external terminal to which the script information 320 is to be displayed is displayed, and the script information 320 is received through the contact input window. When a contact is input, the script information 320 is transmitted to the contact of the external terminal through the wireless communication unit 110.

In this case, the external terminal is provided with the first image 310, and when the first image 310 is converted to 3D, the external terminal performs an analysis operation on each frame of the first image 310 in real time. Instead, the first image 310 may be converted into 3D using only the received script information 320.

In addition, as shown in (c) of FIG. 10, the script information 320 is uploaded to an external script information sharing server, and terminals including the terminal 100 according to the present invention are shared with the external script information sharing server. Various script information can be downloaded and used.

Next, as shown in (a) of FIG. 11, when the script information 320 corresponding to the first image 310 is selected, the controller 180 displays as shown in (b) of FIG. 11. As such, when a user interface 340 that provides an editing function for the 3D effect of each frame in the script information 320 is displayed, and the script information is edited through the UI 340, the image is displayed. When switching to 3D again, the image may be converted into 3D based on the edited script information.

In this case, the UI 340 provides a 3D object adding function and a 3D effect editing function, and the user adds a desired 3D object to each frame of the first image 310 or the desired 3D using the UI 340. An effect (eg, 3D depth value) may be further provided. The 3D object may include 3D text, 3D image, 3D menu icon, and the like.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present invention described above can be embodied as computer-readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, which are also implemented in the form of carrier waves (eg, transmission over the Internet). It also includes. Also, the computer may include a control unit 180 of the terminal.

Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

The above-described portable terminal and its control method may not be limitedly applied to the configuration and method of the above-described embodiments, but the embodiments may be selectively or partially replaced with each other so that various modifications may be made. It may be configured in combination.

100: terminal 110: wireless communication unit
111: broadcast receiver 112: mobile communication module
113 wireless Internet module 114 short-range communication module
115: Position information module 120: A / V input section
121: camera 122: microphone
130: user input unit 140: sensing unit
141: proximity sensor 150: output section
151: Display unit 152: Acoustic output module
153: Alarm module 154: Haptic module
155: Projector Module 160: Memory
170: interface unit 180: control unit
181: multimedia module 190: battery

Claims (10)

A memory including at least one of 2D (Dimensional) and 3D switchable images;
A display unit displaying a 2D playback screen or a 3D playback screen of the video; And
By converting the image to 3D by real-time analysis of the characteristics of the 3D effect to be given for each frame of the image when the image is first converted to 3D, by generating script information including the characteristics of the 3D effect of each analyzed frame And a controller configured to store the image in the memory and convert the image into 3D based on the characteristics of the 3D effect for each frame in the stored script information when the image is converted back to 3D. The method according to claim 1,
The controller may convert the image into 3D using only the stored script information without performing an analysis operation on each frame of the image in real time when the image is converted back to 3D. The method according to claim 1,
The script information may include at least one of a total 3D effect value for each frame of the image, a position and 3D effect value for an object in each frame, and a position and 3D effect value for a background image in each frame. Terminal, characterized in that. The method according to claim 1,
The memory further includes one or more 3D effect values optimized for each category of the image,
The controller identifies a category of the image, attaches identification information indicating the identified category to the script information, and then, when the image is converted to 3D again, the identification information attached to the script information in the memory. And a 3D effect value corresponding to the image. The method according to claim 1,
The controller may include the generated script information in the attribute information of the image and store the same in the memory, and convert the image into 3D based on the script information included in the attribute information of the image when the image is converted back to 3D. Terminal characterized in that switching to 3D. The method according to claim 1,
The controller maps the image and the generated script information, stores the image as a separate file in the memory, searches for the script information mapped with the image in the memory when the image is converted back to 3D, and the searched Terminal for converting the image to 3D based on the script information. The method according to claim 1,
The controller may display a user interface that provides an editing function for the 3D effect of each frame in the stored script information, and when the script information is edited through the UI, when the image is converted to 3D again. And converting the image into 3D based on the edited script information. The method according to claim 1,
Further comprising: a wireless communication unit for transmitting the stored script information,
When the contact information of the external terminal to receive the stored script information is input, the controller transmits the stored script information to the contact information of the external terminal through the wireless communication unit.
The external terminal, characterized in that for reproducing the 3D video to play using only the received script information. The method according to claim 1,
The controller, when the image is converted back to 3D, determines whether the state of the terminal is capable of 3D conversion and playback of the image based on the script information. The terminal, characterized in that for switching to play only part of the 3D. Analyzing a characteristic of a 3D effect to be applied for each frame of the image when an initial 3D conversion command of an image capable of 2D (Dimensional) and 3D conversion is input;
Converting the image into 3D based on a characteristic of the 3D effect of each frame analyzed in real time;
Generating script information including a characteristic of a 3D effect of each frame analyzed in real time;
Storing the generated script information in a memory; And
And converting the image to 3D based on a characteristic of a 3D effect for each frame in the stored script information when the image is converted back to 3D.

Images