KR101883376B1 - Mobile terminal and control method thereof - Google Patents

Abstract

The present invention relates to a mobile terminal capable of improving the image quality of a three-dimensional image and a control method thereof. A mobile terminal according to an embodiment of the present invention includes a camera module and a control unit. The camera module includes a first photographing unit for photographing a subject and acquiring the first image, a second photographing unit disposed apart from the first photographing unit, for photographing the subject and acquiring the second image, An actuator disposed in at least one of the first and second photographing units, and a driving unit for driving the actuator. The control unit analyzes the three-dimensional image formed using the first and second images, and calculates a relative position correction value corresponding to the relative displacement of the first and second photographing units according to the analysis result. The driving unit drives the actuator to control the relative positions of the first and second photographing units based on the relative position correction value.

Description

[0001] MOBILE TERMINAL AND CONTROL METHOD THEREOF [0002]

The present invention relates to a mobile terminal, and more particularly, to a mobile terminal capable of generating a three-dimensional image and a control method thereof.

A terminal can be divided into a mobile or portable terminal and a stationary terminal depending on whether the terminal is movable or not. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

As the functions of such terminals are diversified, they are implemented in the form of a multimedia player having complex functions such as photographing and photographing of movies, reproduction of music and video files, games, and broadcasting reception . Further, in order to support and enhance the function of the terminal, it may be considered to improve the structural and software parts of the terminal.

With this improvement, a three-dimensional photographing function can be mounted on the mobile terminal to provide a three-dimensional image to the user. The three-dimensional photographing function is a function of acquiring a left eye image and a right eye image by photographing a subject using two cameras (stereo cameras), and synthesizing a left eye image and a right eye image to generate a three-dimensional image. However, if the relative position between the two cameras is shifted due to an external impact or the like, the image quality of the three-dimensional image may deteriorate.

An object of the present invention is to provide a mobile terminal and a control method thereof capable of improving the image quality of a three-dimensional image.

A mobile terminal according to an embodiment of the present invention includes a camera module and a control unit. The camera module includes a first photographing unit for photographing a subject and acquiring a first image, a second photographing unit disposed apart from the first photographing unit, for photographing the subject to acquire a second image, And an actuator disposed in at least one of the first and second photographing units, and a driving unit for driving the actuator. The control unit analyzes the three-dimensional image formed using the first and second images, and calculates a relative position correction value corresponding to the relative displacement of the first and second photographing units according to the analysis result. The driving unit drives the actuator to control the relative positions of the first and second photographing units based on the relative position correction value.

In an embodiment, the control unit may calculate the relative position correction value based on an amount of data lost in at least one of the first and second images to form the three-dimensional image.

In an embodiment, each of the first and second photographing units may include a lens unit and an image sensor that converts an optical signal incident through the lens unit into an electrical signal. The actuator may control a position of at least one of the lens unit and the image sensor in response to a driving signal of the driving unit.

In an embodiment, the actuator may include a first actuator for controlling the position of the first photographing portion, and a second actuator for controlling the position of the second photographing portion. The driving unit may drive at least one of the first and second actuators based on the relative position correction value.

In an exemplary embodiment, the camera module may further include a displacement measuring sensor for measuring an absolute displacement of the reference position of the first photographing unit. The driving unit may drive the first actuator to control the position of the first photographing unit based on an absolute position correction value corresponding to the absolute displacement.

In an embodiment, the first and second images may be obtained after the position of the first photographing unit is controlled based on the absolute position correction value.

In the embodiment, the center of the lens section of the first photographing section may coincide with the optical axis at the reference position of the first photographing section.

A method of controlling a mobile terminal having a camera module according to an embodiment of the present invention includes: acquiring first and second images by photographing a subject through first and second photographing units of the camera module; Analyzing a three-dimensional image formed using the first and second images, and calculating a relative position correction value corresponding to a relative displacement of the first and second photographing units according to an analysis result; And controlling relative positions of the first and second photographing units based on the relative position correction value.

In an embodiment, in calculating the relative position correction value, the relative position correction value may be calculated based on an amount of data lost in at least one of the first and second images to form the three-dimensional image .

In an embodiment, the camera module may include a first actuator that controls a position of the first photographing unit, and a second actuator that controls a position of the second photographing unit. In the step of controlling the relative position, at least one of the first and second actuators may be driven based on the relative position correction value.

In the embodiment, in controlling the relative position, the position of at least one of the lens unit and the image sensor provided in each of the first and second photographing units may be controlled according to driving of the at least one actuator .

In one embodiment of the present invention, the control method includes: measuring an absolute displacement of the first photographing unit with respect to a reference position; And driving the first actuator to control the position of the first photographing unit based on an absolute position correction value corresponding to the absolute displacement.

In an embodiment, the position of the first photographing unit may be controlled based on the absolute position correction value before acquiring the first and second images.

According to the present invention, a correction value is calculated according to an analysis result of a three-dimensional image to be photographed first, and the relative displacement of the two photographing units provided in the 3D camera module is corrected based on the calculated correction value, Can be improved.

Further, according to the present invention, there is no need for a sensor for detecting the displacement of at least one of the two photographing units, and a hardware structure (e.g., a fixing housing) for blocking the deviation, Size reduction, cost reduction, etc. can be expected.

According to the present invention, even if an external impact is frequently applied to the mobile terminal, since the relative displacements of the two photographing units can be corrected at any time thereafter, the reliability of the three-dimensional imaging can be enhanced.

1 is a block diagram illustrating a mobile terminal according to the present invention.
2A and 2B are perspective views showing an appearance of a mobile terminal according to the present invention.
3 is a block diagram illustrating a mobile terminal having a 3D camera module according to an embodiment of the present invention.
4 is a flowchart illustrating a method of controlling a mobile terminal according to an embodiment of the present invention.
FIG. 5 is a conceptual diagram showing an example of a form in which a 3D camera module according to an embodiment of the present invention is deformed due to an external impact.
6A and 6B are conceptual diagrams showing a first embodiment of the first and second cameras related to the control method shown in FIG.
7A and 7B are conceptual diagrams showing a second embodiment of the first and second cameras related to the control method shown in FIG.
8A to 8C are conceptual diagrams for explaining a method of calculating a relative position correction value according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art can easily carry out the technical idea of the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly explain the present invention, parts not related to the description are omitted, and like parts are denoted by similar reference numerals throughout the specification.

The mobile terminal described herein may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigator, have. However, it should be understood that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless it is applicable only to a mobile terminal. It will be easy to see.

1 is a block diagram illustrating a mobile terminal 100 according to the present invention. 1, a mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, A controller 160, an interface unit 170, a controller 180, and a power supply unit 190. The components shown in Fig. 1 are not essential, so that a mobile terminal having more or fewer components can be implemented.

Hereinafter, the components 110 to 190 of the mobile terminal 100 will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115.

The broadcast receiving module 111 receives broadcast signals and broadcast related information from an external broadcast management server through a broadcast channel. Here, the broadcast-related information means information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information can also be provided through a mobile communication network. In this case, the broadcast-related information may be received by the mobile communication module 112. The broadcast signal and the broadcast-related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. Such wireless signals may include various types of data depending on a voice call signal, a video call signal, a text message, or a multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be embedded in the mobile terminal 100 or externally. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 114 refers to a module for short-range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used.

The position information module 115 is a module for acquiring the position of the mobile terminal 100, and a representative example thereof is a Global Position System (GPS) module.

1, an A / V (Audio / Video) input unit 120 is for inputting audio signals and video signals and includes a camera 121, a microphone 122, a 3D camera module 200, May be included. The camera 121 processes image frames such as still images and moving images obtained by the image sensor in the video communication mode or the photographing mode. The image frame processed by the camera 121 can be displayed on the display unit 151. [ The image frame may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [

The microphone 122 processes the sound signal input from the outside in the communication mode, the recording mode, the voice selection mode, and the like as electrical voice data. The voice data processed by the microphone 122 in the communication mode can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 and output. The microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated when an external sound signal is input.

The 3D camera module 200 may include two cameras to capture a left-eye image and a right-eye image that form a three-dimensional (3D) image. One of the two cameras may be configured to take a left eye image and take another right eye image.

The user input unit 130 generates input data for controlling the operation of the mobile terminal 100 by a user. The user input unit 130 may include a key pad, a dome switch, a touch pad (static pressure and static electricity), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as presence or absence of a user, the open / close state of the mobile terminal 100, position, orientation, acceleration, deceleration, And generates a sensing signal. For example, when the mobile terminal 100 is in the form of a slide phone, the sensing unit 140 may detect whether the slide phone is opened or closed. The sensing unit 140 may sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like.

The sensing unit 140 may include a proximity sensor 141. The sensing unit 140 may include a touch sensor (not shown) for sensing a touch operation with respect to the display unit 151.

The touch sensor may have the form of a touch film, a touch sheet, a touch pad, or the like. The touch sensor may be configured to convert a pressure applied to a specific portion of the display portion 151 or a change in capacitance generated at a specific portion of the display portion 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the touch pressure.

When the touch sensor and the display unit 151 have a mutual layer structure, the display unit 151 can be used as an input device in addition to the output device. The display unit 151 may be referred to as a 'touch screen'.

If there is a touch input via the touch screen, corresponding signals are sent to a touch controller (not shown). The touch controller processes the signals transmitted from the touch sensor, and then transmits data corresponding to the processed signals to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched.

In the case where the touch screen is electrostatic, it can be configured to detect the proximity of the sensing object by a change of the electric field along the proximity of the sensing target. Such a touch screen may be classified as proximity sensor 141. [

The proximity sensor 141 refers to a sensor that detects the presence or absence of an object to be sensed without mechanical contact using an electromagnetic force or infrared rays. The proximity sensor 141 has a longer life than the contact type sensor and its utilization is also high. Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

Hereinafter, for convenience of explanation, a proximity action is referred to as " proximity touch " while an object to be sensed does not touch the touch screen, and an action of touching the sensing object on the touch screen is called & touch ".

The proximity sensor 141 detects the presence or absence of a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, The information corresponding to the presence / absence of proximity touch and the proximity touch pattern can be output to the touch screen.

The output unit 150 generates an output related to visual, auditory, tactile, and the like. The output unit 150 may include a display unit 151, an audio output module 153, an alarm unit 154, and a haptic module 155.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal 100 operates in the call mode, the display unit 151 displays a UI (User Interface) or a GUI (Graphic User Interface) related to the call. When the mobile terminal 100 operates in the video communication mode or the photographing mode, the display unit 151 displays the photographed image, the received image, the UI, the GUI, and the like.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT-LCD), an organic light emitting diode (OLED), a flexible display display, a 3D display, and an e-ink display.

At least one display (or display element) included in the display unit 151 may be configured to be transparent or light transmissive so that the display can see the outside through it. This can be referred to as a transparent display. A typical example of such a transparent display is TOLED (Transparent OLED). The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 in the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. [ For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from each other or positioned integrally with one another, and may be located on different surfaces.

The sound output module 153 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice selection mode, a broadcast reception mode, The sound output module 153 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The sound output module 153 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 154 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events generated in the mobile terminal 100 include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 154 may output a signal for informing occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The display unit 151 and the audio output module 153 may be classified as a part of the alarm unit 154 because the video signal or the audio signal can be output through the display unit 151 or the audio output module 153. [

The haptic module 155 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 155 is vibration. The intensity, pattern, and the like of the vibration generated by the tactile module 155 are controllable. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 155 may be configured to perform various functions such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or a suction force of the air through the injection port or the suction port, a touch on the skin surface, contact with an electrode, Various effects can be generated such as the effect of heat generation and the effect of reproducing the cold sensation using the heat absorbing or heatable element.

The haptic module 155 can be configured to not only transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sense of the finger or arm. At least two haptic modules 155 may be provided according to the configuration of the mobile terminal 100.

The memory 160 may store a program for the operation of the controller 180 and temporarily store input and output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 160 may store data related to vibrations and sounds of various patterns that are output upon touch input on the touch screen.

The memory 160 may be a flash memory, a hard disk, a multimedia card micro type, a card type memory (e.g., SD or XD memory), a random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM) And may include at least one storage medium. The mobile terminal 100 may operate in association with a web storage that performs storage functions of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data to the external device. For example, the interface unit 170 may include a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, Input / output ports, video I / O ports, earphone ports, and the like.

The identification module is a chip for storing various information for authenticating the usage right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module Universal Subscriber Identity Module (USIM)). A device equipped with an identification module (hereinafter referred to as "identification device") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit 170 may be a path through which the power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle or various command signals input from the cradle by the user, (Not shown). The various command signals or power supplied from the cradle may also act as a signal for recognizing that the mobile terminal 100 is correctly mounted in the cradle.

The controller 180 controls the overall operation of the mobile terminal 100. For example, control and processing related to voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [ The control unit 180 may perform pattern selection processing for selecting handwriting input and drawing input on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be applied to various types of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays Microprocessors, microprocessors, and other electronic units for performing other functions, as will be appreciated by those skilled in the art. In some cases, the embodiments described herein may be implemented by the controller 180 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code may be implemented in a software application written in a suitable programming language. Such software codes may be stored in the memory 160 and executed by the control unit 180. [

Hereinafter, a method of processing user input to the mobile terminal 100 will be described.

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100, and may include a plurality of operation units. The operating units may also be referred to as manipulating portions and may be employed in any manner as long as they are operated in a tactile manner by the user's tactile sense.

Various kinds of time information can be displayed on the display unit 151. [ Such visual information can be displayed in the form of letters, numbers, symbols, graphics, icons, and the like, and can be formed as a three-dimensional stereoscopic image. At least one of a character, a number, a symbol, a graphic, and an icon may be displayed in a predetermined arrangement for inputting time information, thereby being implemented as a keypad. Such a keypad may be referred to as a so-called " soft key ".

The display unit 151 may operate as an entire area or may be divided into a plurality of areas and operated. In the latter case, the plurality of areas can be configured to operate in association with one another. For example, an output window and an input window may be displayed on the upper and lower portions of the display unit 151, respectively. The output window and the input window are areas allocated for outputting or inputting information, respectively. In the input window, a soft key with a number for inputting a telephone number may be output. When the soft key is touched, the number corresponding to the touched soft key is displayed in the output window. When the operating unit is operated, a call connection to the telephone number displayed in the output window may be attempted, or the text displayed in the output window may be entered into the application.

The display unit 151 or the touch pad may be configured to detect touch scrolling. The user can move the cursor or the pointer positioned on the displayed object, for example, the icon, on the display unit 151 by scrolling the display unit 151 or the touch pad. Further, when the finger is moved on the display unit 151 or the touch pad, the path along which the finger moves may be visually displayed on the display unit 151. [ This will be useful for editing the image displayed on the display unit 151. [

One function of the mobile terminal 100 may be executed in response to a case where the display unit 151 and the touch pad are touched together within a predetermined time range. In the case of being touched together, there may be a case where the user clamps the main body of the mobile terminal 100 using the thumb and index finger. At this time, one function of the mobile terminal 100 to be executed may be activation or deactivation for the display unit 151 or the touch pad, for example.

2A and 2B are perspective views showing an appearance of a mobile terminal 100 according to the present invention. FIG. 2A shows a front side and one side of the mobile terminal 100, and FIG. 2B shows a rear side and the other side of the mobile terminal 100. FIG.

Referring to FIG. 2A, the mobile terminal 100 includes a bar-shaped terminal body. However, the mobile terminal 100 is not limited thereto, and may be realized in various forms such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are movably coupled to each other.

The terminal body includes a case (a casing, a housing, a cover, and the like) that forms an appearance. In the embodiment, the case may be divided into a front case 101 and a rear case 102. A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injection molding of a synthetic resin or may be formed to have a metal material such as stainless steel (STS), titanium (Ti) or the like.

The display unit 151, the sound output unit 152, the camera 121, the user input unit 130 (see FIG. 1), the microphone 122, the interface 170, can do.

The display unit 151 occupies a main portion of the front case 101. [ The audio output unit 152 and the camera 121 are located in a region adjacent to one end of the display unit 151 and the first user input unit 131 and the microphone 122 are located in a region adjacent to the other end. The second user input unit 132 and the interface 170 may be located on the sides of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100. [ The user input unit 130 may include a plurality of operation units 131 and 132.

The first or second operating units 131 and 132 can receive various commands. For example, the first operation unit 131 can receive commands such as start, end, scroll, and the like. The second operation unit 132 can receive commands such as adjusting the size of the sound output from the sound output unit 152, switching to the touch selection mode of the display unit 151, and the like.

Referring to FIG. 2B, the 3D camera module 200 may be mounted on the rear surface of the terminal body, that is, the rear case 102. The 3D camera module 200 may include first and second cameras 210 and 220 spaced apart from each other. The first and second cameras 210 and 220 may have a photographing direction opposite to that of the front camera 121 (see FIG. 2A), and may have different pixels from the front camera 121.

For example, the front camera 121 may be configured to have a low pixel, and the first and second cameras 210 and 220 may be configured to have a high pixel. Accordingly, when the front camera 121 is used during a video call, when the user's face is photographed and the photographed image is transmitted to the other party in real time, the size of the transmitted data can be reduced. On the other hand, the first and second cameras 210 and 220 may be used for storing high quality images.

Meanwhile, the first and second cameras 210 and 220 may be installed in the terminal body so as to be rotated or popped up.

The flash 123 and the mirror 124 may be further positioned adjacent to the 3D camera module 200. The flash 123 illuminates the subject when the user photographs the subject with the 3D camera module 200. The mirror 124 illuminates the user's face when the user photographs himself / herself (self-photographing) using the 3D camera module 200.

A rear sound output unit 152 'may be additionally disposed on the rear surface of the terminal body. The rear sound output unit 152 'may perform a stereo function together with the front sound output unit 152 (see FIG. 2A), and may perform a speakerphone function during a call.

In addition to the antenna for communication, an antenna 116 for receiving broadcast signals may be additionally provided on the side surface of the terminal body. The antenna 116 constituting a part of the broadcast receiving module 111 (see FIG. 1) can be installed to be able to be drawn out from the terminal body.

A power supply unit 190 for supplying power to the mobile terminal 100 is mounted on the terminal body. The power supply unit 190 may be built in the main body of the terminal or may be detachable from the outside of the main body of the terminal.

Hereinafter, a mobile terminal 100 capable of improving the quality of a three-dimensional image acquired through the 3D camera module 200 will be described.

3 is a block diagram illustrating a mobile terminal 100 having a 3D camera module 200 according to an embodiment of the present invention. Referring to FIG. 3, the 3D camera module 200 may include two cameras 210 and 220 and a driver 230. The 3D camera module 200 may also be referred to as a stereoscopic camera.

Although the figure shows a 3D camera module 200 with two actuators 214 and 224, the 3D camera module 200 according to an embodiment of the present invention is not limited to this, And the second photographing units 212 and 222. In this case,

The first camera 210 includes a first photographing unit 212 and a first actuator 214. The second camera 220 is disposed apart from the first camera 210, A second actuator 222, Here, the first camera 210 will be mainly described, and redundant description between the first and second cameras 210 and 220 will be omitted.

The first camera 210 may include a case (not shown) that accommodates the first photographing unit 212 and the first actuator 214. An inner space sufficient for the first photographing unit 212 to move may be formed in the case under the control of the first actuator 214. For example, movement, rotation, tilting, etc. of the first photographing unit 212 may be performed under the control of the first actuator 214 in the inner space of the case.

The first photographing unit 212 can photograph a subject to obtain a first image. To this end, the first photographing unit 212 may include a lens unit, an image sensor, and the like. The process of acquiring the first image will be described in more detail. The optical signal input through the lens unit from the subject can be converted into an electrical signal by the image sensor to acquire the first image. The second image can also be acquired by the second image pickup unit 222 in the same manner. A three-dimensional image can be formed using the first and second images thus obtained. At this time, the first and second images may be referred to as a left eye image and a right eye image, respectively.

The first actuator 214 may be disposed in the first photographing unit 212 to control the first photographing unit 212 in response to the driving signal of the driving unit 230. [ The first actuator 214 may be implemented as a voice coil motor (VCM), a piezoelectric motor, or the like.

The driving unit 230 may drive at least one of the first and second actuators 214 and 224 to control the relative positions of the first and second photographing units 212 and 222 under the control of the controller 180 . To this end, the driving unit 230 may generate a driving signal based on the relative position correction value provided from the control unit 180. The driving unit 230 may transmit the generated driving signal to at least one of the first and second actuators 214 and 224.

Here, the relative position correction value may correspond to the relative displacement of the first and second photographing units 212 and 222. The relative positions of the first and second cameras 210 and 220 or the first and second photographing units 212 and 222 may be displaced due to an external impact applied to the mobile terminal 100, for example. At this time, the relative position correction value means the degree of deviation of the relative position of the first and second photographing units 212 and 222, i.e., the value calculated to correct the relative displacement.

The control unit 180 may receive the first image from the first image capturing unit 212 and receive the second image from the second image capturing unit 222. [ The controller 180 may generate the three-dimensional image using the first and second images, and may calculate the relative position correction value according to the analysis result.

Image distortion may occur when a three-dimensional image is formed using first and second images obtained in a state in which the relative positions of the first and second photographing units 212 and 222 are misaligned. Data may be lost in at least one of the first and second images as the image distortion is corrected. At this time, the amount of data loss is related to the degree to which the relative positions of the first and second photographing units 212 and 222 are shifted. Accordingly, the control unit 180 can calculate the relative position correction value based on the amount of data lost in at least one of the first and second images.

As described above, according to the present invention, the correction value is calculated according to the result of analysis of the three-dimensional image to be photographed first, and the relative displacement of the first and second photographing units 212 and 222 is corrected based on this, The image quality of the photographed three-dimensional image can be improved.

Further, according to the present invention, there is provided a sensor for detecting a shift of at least one of the first and second photographing units 212 and 222, and a hardware structure (e.g., a fixing housing) for blocking the shift The effect of reducing the size and cost of the mobile terminal 100 can be expected.

According to the present invention, even if an external impact is frequently applied to the mobile terminal 100, since the relative displacements of the first and second photographing units 212 and 222 can be corrected at any time thereafter, reliability of three- .

4 is a flowchart illustrating a method of controlling the mobile terminal 100 according to an embodiment of the present invention. Referring to FIG. 4, a step S110 of capturing a subject through the first and second photographing units 212 and 222 (see FIG. 3) and acquiring the first and second images, respectively, is performed.

Next, a three-dimensional image formed using the first and second images is analyzed, and a relative position correction value corresponding to the relative displacement of the first and second photographing units 212 and 222 is calculated according to the analysis result Step S120 is proceeded. An example of a method of calculating the relative position correction value will be described in detail with reference to Figs. 8A to 8C below.

Thereafter, the relative position of the first and second photographing units 212 and 222 is controlled based on the relative position correction value (S130). The control of the relative positions of the first and second photographing units 212 and 222 means that the position of at least one of the first and second photographing units 212 and 222 is controlled. This is for correcting the relative displacement of the first and second photographing units 212 and 222.

In the embodiment of the present invention, when the position of any one of the first and second photographing units 212 and 222 is deviated due to an external impact, the present invention is not limited to controlling only the photographing unit whose position is shifted, The relative positions of the first and second photographing units 212 and 222 may be controlled.

In the embodiment of the present invention, before relative displacement of the first and second photographing units 212 and 222 is corrected, any one of the first and second photographing units 212 and 222, for example, The displacement with respect to the reference position of the first photographing unit 212 (hereinafter referred to as absolute displacement) can be corrected. 3) may have a displacement measuring sensor for measuring the absolute displacement of the first photographing unit 212, for example, a hole sensor, a light emitting-light receiving diode, and the like have. Then, the control unit 180 (see Fig. 3) can calculate the absolute position correction value corresponding to the absolute displacement. The driving unit 230 may drive the first actuator 214 (see FIG. 3) to control the position of the first imaging unit 212 based on the absolute position correction value.

As the absolute displacement of the first photographing unit 212 is corrected, the first photographing unit 212 can be positioned at the reference position. At this time, the center of the lens section of the first photographing section 212 and the optical axis can coincide with each other.

(S110) of obtaining the first and second images described above after the absolute displacement of the first photographing unit 212 is corrected, calculating the relative position correction value (S120), and calculating the relative position correction value (S130) of controlling the relative positions of the first and second sensors 212 and 222 may be performed.

5 is a conceptual diagram showing an example of a form in which the 3D camera module 200 according to the embodiment of the present invention is deformed due to an external impact. Referring to FIG. 5, one side of the back side of the mobile terminal 100, for example, where the exposed portion of the 3D camera module 200 is disposed, is shown.

When an external impact is applied to the mobile terminal 100 in a state where the first and second cameras 210 and 220 of the 3D camera module 200 are positioned at their respective reference positions, the first and second cameras 210 and 220, 220 may be displaced. For example, as shown in the figure, the position of the first camera 210 may be displaced by a rotation angle? Relative to the reference position. At this time, the position of the first photographing unit 212 (see FIG. 3) provided in the first camera 210 may also be displaced.

Hereinafter, a specific method of correcting the relative displacements of at least one of the first and second photographing units 210 and 220 will be described.

6A and 6B are conceptual diagrams showing a first embodiment of the first and second cameras 210 and 220 associated with the control method shown in FIG. 7A and 7B are conceptual diagrams illustrating a second embodiment of the first and second cameras 210 and 220 associated with the control method shown in FIG. Figs. 6A-7B show cross sections of the first and second cameras 210,220 taken along line A-A shown in Fig. 5, respectively.

Referring to FIG. 6A, the first actuator 214 may be disposed in the first photographing unit 212 to control both the first lens unit 212a and the first image sensor 212b. In other words, the first actuator 214 can be arranged to control the entire first imaging section 212. [ In this case, the first photographing unit 212 can be controlled while the optical axis is maintained. The second camera 220 may be implemented in the same manner as the first camera 210.

However, as the position of the first camera 210 is shifted by the rotation angle? Due to an external impact, the position of the first photographing unit 212 may also be displaced. On the other hand, the position of the second camera 220 or the second photographing unit 222 may not be shifted. This means that relative displacements of the first and second photographing units 212 and 222 are generated by the rotation angle?.

Although not shown, the relative displacements of the first and second photographing units 212 and 222 may or may not occur when the first and second photographing units 212 and 222 are all deviated from their reference positions . For example, when at least one of the direction and the degree (e.g., angle, distance, etc.) of the first and second photographing units 212 and 222 are different from each other is different from that of the first and second photographing units 212 and 222 Relative displacement occurs. On the other hand, when the first and second photographing units 212 and 222 are misaligned with each other in the same direction, relative displacement of the first and second photographing units 212 and 222 does not occur.

At least one of the first and second actuators 214 and 224 may be driven to compensate for the relative displacement of the first and second photographing units 212 and 222 when they occur. For example, the first photographing unit 212 may be rotated according to the driving of the first actuator 214 so that the relative displacements of the first and second photographing units 212 and 222 are corrected. 6B, the first photographing unit 212 may be rotated such that the optical axis of the first photographing unit 212 and the optical axis of the second photographing unit 222 are parallel to each other.

Referring to FIG. 7A, the first actuator 214 may be disposed in the first photographing unit 212 to control the first lens unit 212a. Although not shown, the first actuator 214 may be disposed in the first imaging section 212 to control the first image sensor 212b. In other words, the first actuator 214 may be arranged to control a part of the first photographing part 212. [ The second camera 220 may be implemented in the same manner as the first camera 210.

At least one of the first and second actuators 214, 224 may be driven to correct the relative displacement of the first and second photographing sections 212, 222. For example, a part of the first photographing unit 212 may be moved according to the driving of the first actuator 214 so that the relative displacement of the first and second photographing units 212 and 222 is corrected. More specifically, as shown in FIG. 7B, the first lens unit 212a may be moved such that the optical axis of the first photographing unit 212 and the optical axis of the second photographing unit 222 are parallel to each other.

8A to 8C are conceptual diagrams for explaining a method of calculating a relative position correction value according to an embodiment of the present invention.

Referring to FIG. 8A, first and second images 310 and 320 forming a three-dimensional image are shown. The first image 310 can be obtained by photographing the object through the first photographing unit 212 (see Fig. 3) whose position is shifted. Then, the second image 320 can be obtained by photographing the object through the second photographing unit 222 (see FIG. 3) which is in the reference position, that is, in the normal state. In this case, image distortion may occur between the first and second images 310 and 320. Accordingly, an image processing process for correcting the image distortion can be performed.

First, as shown in FIG. 8B, the controller 180 (see FIG. 3) controls the first image 310 and the second image 320 so that the shape of the subject included in the first image 310 and the shape of the subject included in the second image 320 correspond to each other. And the second image 310, 320, for example, the first image 310 may be rotated. To this end, the control unit 180 can detect and compare the outline information of each object.

Then, the controller 180 may combine the first and second images 310 and 320 to form a three-dimensional image. At this time, data constituting an area where the first and second images 310 and 320 are not overlapped with each other may be lost. As the relative displacement of the first and second photographing units 212 and 222 corresponding to the degree of image distortion increases, the amount of data lost during the synthesis of the first and second images 310 and 320 may increase. Accordingly, the control unit 180 can calculate the relative position correction value based on the amount of data lost during the synthesis of the first and second images 310 and 320.

According to the embodiment disclosed herein, the above-described method can be implemented by a code that can be read by a processor on a medium on which the program is recorded. Examples of the medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of a carrier wave (e.g., transmission over the Internet) .

In the mobile terminal disclosed in this specification, the configuration and method of the above-described embodiments are not limitedly applied, but all or some of the embodiments may be selectively combined so that various modifications can be made.

Claims (15)

A first photographing unit photographing a subject to acquire a first image,
A second photographing unit which is disposed apart from the first photographing unit and photographs the subject to acquire a second image,
A camera module having an actuator disposed in at least one of the first and second photographing units, and a driving unit driving the actuator; And
And a controller for analyzing an image formed using the first and second images and calculating a relative position correction value corresponding to a relative displacement of the first and second photographing units according to an analysis result,
Wherein the driving unit drives the actuator to control relative positions of the first and second photographing units so that the optical axes of the first and second photographing units are parallel to each other based on the relative position correction value,
The first photographing unit and the second photographing unit each include an image sensor and a lens unit,
The first photographing unit is provided in the first case, the second photographing unit is provided in the second case,
The driving unit may be configured such that the first photographing unit itself is rotated with respect to the first case so that the optical axes of the first and second photographing units are parallel to each other or the second photographing unit itself rotates about the second case And the actuator is driven so as to move the actuator. The method according to claim 1,
Wherein,
Wherein the relative position correction value is calculated based on an amount of data lost in at least one of the first and second images to form the image. The method according to claim 1,
Wherein each of the first and second photographing units includes a lens unit and an image sensor that converts an optical signal incident through the lens unit into an electrical signal. The method of claim 3,
Wherein the actuator comprises:
And controls the position of at least one of the lens unit and the image sensor in response to a driving signal of the driving unit. The method according to claim 1,
Wherein the actuator comprises:
A first actuator for controlling a position of the first photographing unit, and a second actuator for controlling a position of the second photographing unit,
The driving unit includes:
And at least one of the first actuator and the second actuator is driven based on the relative position correction value. 6. The method of claim 5,
The camera module includes:
And a displacement measuring sensor for measuring an absolute displacement of the first photographing unit with respect to a reference position. The method according to claim 6,
The driving unit includes:
And drives the first actuator to control the position of the first photographing unit based on an absolute position correction value corresponding to the absolute displacement. 8. The method of claim 7,
Wherein the first and second images are obtained after the position of the first photographing unit is controlled based on the absolute position correction value. The method according to claim 6,
Wherein a center of the lens section of the first photographing section coincides with an optical axis at a reference position of the first photographing section. A method of controlling a mobile terminal having a camera module, the method comprising:
Capturing a subject through the first and second photographing units of the camera module to obtain first and second images, respectively;
Analyzing an image formed using the first and second images, and calculating relative position correction values corresponding to relative displacements of the first and second photographing units according to an analysis result; And controlling the relative positions of the first and second photographing units so that optical axes of the first and second photographing units are parallel to each other based on the relative position correction value,
The first photographing unit and the second photographing unit each include an image sensor and a lens unit,
The first photographing unit is provided in the first case, the second photographing unit is provided in the second case,
Wherein the controlling step comprises the steps of rotating the first photographing unit per se with respect to the first case so that the optical axes of the first and second photographing units are parallel to each other, And the actuator is driven so as to rotate by a predetermined angle. delete delete delete delete delete

Images