WO2018159864A1 - Mobile terminal and control method for mobile terminal - Google Patents

Abstract

The present invention relates to a mobile terminal having a camera and a control method for the mobile terminal, the mobile terminal comprising: first and second cameras; and a control unit for controlling the first and second cameras in response to an image capturing request such that an image is captured through the first and/or second cameras, wherein, when the image capturing request corresponds to a continuous photographing request, the control unit controls the first and second cameras such that the first and second cameras alternately capture images of a focused specific subject, and the first camera or the second camera adjusts a focus for the specific subject while the other one captures an image.

Description

Mobile terminal and control method thereof

The present invention relates to a mobile terminal having a camera and a control method of the mobile terminal.

Terminals may be divided into mobile / portable terminals and stationary terminals according to their mobility. The mobile terminal may be further classified into a handheld terminal and a vehicle mounted terminal according to whether a user can directly carry it.

The functions of mobile terminals are diversifying. For example, data and voice communication, taking a picture and video with a camera, recording a voice, playing a music file through a speaker system, and outputting an image or video to a display unit. Some terminals have an electronic game play function or a multimedia player function. In particular, recent mobile terminals may receive multicast signals that provide visual content such as broadcasting, video, and television programs.

As the terminal functions are diversified, for example, such a terminal is a multimedia player having a complex function such as taking a picture or a video, playing a music or video file, playing a game, or receiving a broadcast. Is being implemented.

Meanwhile, in recent years, as the utilization of the camera provided in the terminal increases, functions for improving the function of the camera have been actively developed. As one of such improvements, a continuous shooting function capable of continuously photographing a subject has emerged.

On the other hand, in order to capture a clear image of the subject, the subject must be precisely focused. Therefore, when the subject moves, there is a problem in that it is very difficult for the user to directly focus in the continuous shooting according to the movement of the subject. Accordingly, the continuous shooting function may be generally performed when a fixed subject or a focal length between the subject and the image sensor is fixed.

Meanwhile, recent cameras may provide an AF (Auto Focusing) function that automatically adjusts focus according to a distance between a subject and an image sensor. Accordingly, when using the AF function, by automatically detecting a subject and tracking the detected subject, the focus may be adjusted according to the moving subject even if the user does not directly adjust the focus.

In the case of the AF function, when the distance between the subject and the image sensor is changed, posture adjustment of the image sensor for focusing on the subject may be performed according to the changed distance. When the posture adjustment of the image sensor is performed, a minimum time required for changing the posture of the image sensor, for example, a time (stabilization time) until the vibration caused by the movement of the image sensor is eliminated is required. do.

Therefore, even if the AF function can automatically adjust the focus, if the subject moves after the image is captured, the focus may be adjusted according to the AF function only after the minimum time necessary for adjusting the posture of the image sensor is taken. Can be. Therefore, when continuous shooting is performed while automatically adjusting the focus according to the AF function, a predetermined time interval occurs between shots, and high speed shooting is difficult due to these predetermined time intervals.

An object of the present invention is to solve the above-mentioned problems and other problems, and to provide a mobile terminal and a method of controlling the mobile terminal capable of performing continuous shooting at high speed while automatically adjusting the focus according to the AF function. will be.

It is also an object of the present invention to provide a mobile terminal and a method of controlling the mobile terminal to automatically start and end the continuous shooting based on an image sensor for sensing a subject.

According to an aspect of the present invention to achieve the above or another object, a mobile terminal according to an embodiment of the present invention, the first and second cameras, and in response to the image photographing request, of the first and second cameras; And a controller configured to control the first and second cameras to perform image capturing through at least one, and wherein the controller includes a specific subject to be focused when the image capturing request corresponds to a continuous capturing request. The first and second cameras are controlled so that the first and second cameras alternately perform image capturing, wherein one of the first and second cameras is the specific subject while the other is capturing the image. It is characterized by adjusting the focus on.

The controller may be configured to sense whether the specific subject is included in the first angle of view of the first camera and the second angle of view of the second camera, and according to a sensing result, the first and second objects. The camera alternately captures an image to determine whether to perform the continuous shooting.

The display device may further include a touch screen for outputting a preview image received through at least one of the first and second cameras, wherein the preview image includes graphic objects corresponding to at least one subject, respectively. And when the specific graphic object of the graphic objects included in the preview image is touched through the touch screen, the controller sets the touched graphic object as a specific subject to be the focus, and the specific subject is the It is characterized by monitoring whether or not included in both the first and second angle of view.

The control unit may include the first and second cameras alternately performing image capturing when the specific subject is included in both the first angle of view and the second angle of view. And controlling the second camera.

According to an embodiment of the present disclosure, the controller stops the continuous shooting of capturing an image alternately between the first and second cameras when the specific subject is out of one of the first and second angles of view. The first and second cameras may be controlled to capture an image through one of the first and second cameras corresponding to an angle of view including the specific subject.

According to an embodiment of the present disclosure, when the specific subject is out of one of the first angle of view and the second angle of view, the controller outputs notification information for notifying the deviation of the angle of view of the specific subject, and the notification information includes: The apparatus may further include information related to the movement of the terminal body including the first and second cameras.

According to an embodiment of the present disclosure, when the specific subject is included in any one of the first angle of view and the second angle of view, the controller is configured to capture an image through a camera having an angle of view including the specific subject. And controlling the first and second cameras.

The display device may further include a touch screen on which a preview image received through at least one of the first and second cameras is output, and wherein the controller is configured to view angles of view of the first and second cameras among the preview images. And a guide image for displaying the overlapping areas on the touch screen.

The control unit may generate the preview image by synthesizing the first image sensed according to the first angle of view of the first camera and the second image sensed according to the second field of view of the second camera. The preview image may include an image of an area sensed by both the first camera and a second camera and an image of an area sensed by only one of the first and second cameras according to the overlap of the angles of view. It is done.

The control unit synthesizes the preview image based on the image sensed by both the first camera and the second camera, and the first angle of view of the first camera and the second angle of view of the second camera. In this case, a picture in picture (PIP) format is inserted by inserting an image sensed by a camera having a narrower angle of view into an image sensed by a camera having a wider angle of view among the images sensed by the first and second cameras. And outputting the synthesized preview image.

The control unit synthesizes the preview image based on the image sensed by both the first camera and the second camera, and the first angle of view of the first camera and the second angle of view of the second camera. In this case, a preview image obtained by synthesizing the image sensed only by the first camera and the image sensed only by the second camera may be output around the image sensed by both the first camera and the second camera.

According to an embodiment of the present disclosure, if the specific subject is included only in an image sensed by the first camera among the images sensed by the first camera and the second camera, the controller may start the continuous shooting. When the first camera and the second camera are controlled and the specific subject is included only in the image sensed by the second camera among the images sensed by the first camera and the second camera, the continuous shooting is performed. And controlling the first camera and the second camera to stop.

In one embodiment, the first camera is characterized in that the image containing the specific subject is a camera that is sensed before other cameras.

According to an embodiment, the apparatus may further include a sensing unit configured to detect a movement of the specific subject, wherein the controller is configured to adjust the focus while the one camera adjusts the focus and the other camera photographs the image. The sensing unit detects the movement state of the specific subject, and when the focus adjustment of the one Kakera is completed, the camera is photographed based on the detected movement state of the subject while the camera is capturing an image. Another camera is characterized by controlling to adjust the focus.

According to an embodiment of the present disclosure, when the image capturing corresponds to continuous capturing of a plurality of preset focus regions, the controller may include the first and second cameras for each of the plurality of focus regions to be focused. The first and second cameras are controlled to alternately perform image capturing, wherein one of the first and second cameras adjusts a focus on another focus area while the other is capturing the image. .

The touch screen may further include a touch screen configured to output a preview image generated through at least one of images received through at least one of the first and second cameras. The controller may further include a touch screen. When a plurality of touch inputs are applied, the plurality of focus areas having a preset order may be set according to the order in which the touch inputs are applied.

In an embodiment, the apparatus may further include a sensing unit configured to detect a movement of the specific subject, wherein the controller is configured to detect the movement of the specific subject based on a result of detecting the movement of the specific subject or an input applied from a user. The plurality of focus areas may be set according to the prediction and the predicted result.

According to an embodiment, when the image is captured according to the continuous shooting, the controller detects images that are out of focus among the captured images, and further performs the continuous shooting by the number of times corresponding to the number of detected images. It is characterized by performing.

According to an aspect of the present invention to achieve the above or another object, a control method of a mobile terminal according to an embodiment of the present invention, the step of activating a first camera and a second camera, from each of the activated cameras Generating a preview image based on the sensed images, outputting the generated preview image, receiving a plurality of focus areas or at least one subject based on a user input applied to the preview image, and And performing continuous shooting alternately through the first camera and the second camera according to a preset order of the plurality of focus areas and a result of detecting the specific subject. The performing of the continuous shooting may include: , While one camera is shooting an image, the other camera is in the next order of focus area or Is characterized in that characterized in that the step of adjusting focus according to a result of detecting a subject.

The continuous shooting may include determining whether an image of the specific subject is detected in an area sensed only by the first camera, and performing the continuous shooting according to the determination result. And determining whether an image of the specific subject is detected in an area sensed only by the second camera, and stopping the continuous shooting according to the determination result. do.

The effects of the mobile terminal and its control method according to the present invention are as follows.

According to at least one of the embodiments of the present disclosure, the second image sensor performs an autofocus function while the first image sensor captures an image, and when the autofocus adjustment of the second image sensor is completed, By allowing the first image sensor to perform auto focus while the two image sensors capture the image, there is an effect that the image capture can be continuously performed without a time interval caused by the AF function between each shooting.

Further, according to at least one of the embodiments of the present invention, the start and end of the continuous shooting is automatically determined according to whether the image of the subject is detected in the preview area other than the area where the continuous image capture, that is, the continuous shooting can be performed. By controlling it, when a preset subject is detected, it is possible to perform continuous shooting at high speed while automatically adjusting the focus according to the detected subject.

1A is a block diagram illustrating a mobile terminal related to the present invention.

1B and 1C are conceptual views of one example of a mobile terminal, viewed from different directions.

2A and 2B are conceptual views illustrating the camera of the present invention.

3 is a flowchart illustrating an operation process of performing continuous photographing at high speed while the mobile terminal automatically adjusts focus according to an AF function according to an exemplary embodiment of the present invention.

FIG. 4 is a detailed flowchart illustrating an operation of performing an AF function and capturing an image by crossing the first and second image sensors during the operation of FIG. 3.

5 is a conceptual diagram illustrating an operation process performed in the first and second cameras over time in a mobile terminal according to an embodiment of the present invention.

6A and 6B are exemplary diagrams illustrating examples of generating and displaying a preview image based on images sensed by first and second image sensors in a mobile terminal according to an exemplary embodiment of the present invention.

7 is a diagram illustrating an example of performing continuous shooting according to an embodiment of the present invention with respect to preset focus areas from a user in a mobile terminal according to an embodiment of the present invention.

8 is an exemplary diagram illustrating another example of performing continuous shooting on preset focus areas in a mobile terminal according to an exemplary embodiment of the present invention.

9 and 10 are exemplary diagrams illustrating examples of performing continuous shooting on a subject moving at high speed in a mobile terminal according to an exemplary embodiment of the present invention.

11 is an exemplary diagram illustrating an example of performing continuous shooting according to a user's input continuously applied in a mobile terminal according to an embodiment of the present invention.

12 is an exemplary diagram illustrating an example of generating a composite image using captured images in a mobile terminal according to an embodiment of the present invention.

FIG. 13A illustrates an example of further capturing a supplementary image according to whether a captured image is in focus in a mobile terminal according to an exemplary embodiment of the present invention.

FIG. 13B is an exemplary diagram illustrating an example of editing captured images by using the images supplemented in FIG. 13A.

14A and 14B are exemplary views showing examples of preview images including regions in which a start and end of a continuous shooting function can be controlled in a mobile terminal according to an exemplary embodiment of the present invention.

FIG. 15 is a diagram illustrating an example in which a start and end of a continuous shooting function are controlled according to a position on a preview area in which a subject image is detected in a mobile terminal according to an exemplary embodiment of the present invention.

Hereinafter, exemplary embodiments disclosed herein will be described in detail with reference to the accompanying drawings, and the same or similar components will be given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used in consideration of ease of specification, and do not have distinct meanings or roles from each other. In addition, in describing the embodiments disclosed herein, when it is determined that the detailed description of the related known technology may obscure the gist of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, the accompanying drawings are intended to facilitate understanding of the embodiments disclosed herein, but are not limited to the technical spirit disclosed herein by the accompanying drawings, all changes included in the spirit and scope of the present invention. It should be understood to include equivalents and substitutes.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, the terms "comprises" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

The following description will be made in the order of the upper left drawing, in a clockwise order. The upper left drawing is the first drawing, the upper right drawing is the second drawing and the lower right drawing. Is described as 'third drawing', and the lower left drawing is named 'fourth drawing'. Or from the top of the drawing, in order from top to bottom, and the top of the drawing is the first drawing, the drawing below the first drawing is the second drawing, and the bottom is the drawing. The description will be given by naming the third drawing.

The mobile terminal described herein includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant, a portable multimedia player, a navigation, a slate PC , Tablet PCs, ultrabooks, wearable devices, such as smartwatches, glass glasses, head mounted displays, and the like. have.

However, it will be readily apparent to those skilled in the art that the configuration according to the embodiments described herein may also be applied to fixed terminals such as digital TVs, desktop computers, digital signages, etc., except when applicable only to mobile terminals. .

1A to 1C, FIG. 1A is a block diagram illustrating a mobile terminal according to the present invention, and FIGS. 1B and 1C are conceptual views of one example of the mobile terminal, viewed from different directions.

The mobile terminal 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a controller 180, and a power supply unit 190. ) May be included. The components shown in FIG. 1A are not essential to implementing a mobile terminal, so that the mobile terminal described herein may have more or fewer components than those listed above.

More specifically, the wireless communication unit 110 of the components, between the mobile terminal 100 and the wireless communication system, between the mobile terminal 100 and another mobile terminal 100, or the mobile terminal 100 and the external server It may include one or more modules that enable wireless communication therebetween. In addition, the wireless communication unit 110 may include one or more modules for connecting the mobile terminal 100 to one or more networks.

The wireless communication unit 110 may include at least one of the broadcast receiving module 111, the mobile communication module 112, the wireless internet module 113, the short range communication module 114, and the location information module 115. .

The input unit 120 may include a camera 121 or an image input unit for inputting an image signal, a microphone 122 for inputting an audio signal, an audio input unit, or a user input unit 123 for receiving information from a user. , Touch keys, mechanical keys, and the like. The voice data or the image data collected by the input unit 120 may be analyzed and processed as a control command of the user.

The sensing unit 140 may include one or more sensors for sensing at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity. G-sensor, Gyroscope Sensor, Motion Sensor, RGB Sensor, Infrared Sensor, Infrared Sensor, Finger Scan Sensor, Ultrasonic Sensor Optical sensors (e.g. cameras 121), microphones (see 122), battery gauges, environmental sensors (e.g. barometers, hygrometers, thermometers, radiation detection sensors, Thermal sensors, gas sensors, etc.), chemical sensors (eg, electronic noses, healthcare sensors, biometric sensors, etc.). Meanwhile, the mobile terminal disclosed herein may use a combination of information sensed by at least two or more of these sensors.

The output unit 150 is used to generate an output related to sight, hearing, or tactile sense, and includes at least one of a display unit 151, an audio output unit 152, a hap tip module 153, and an optical output unit 154. can do. The display unit 151 forms a layer structure with or is integrally formed with the touch sensor, thereby implementing a touch screen. The touch screen may function as a user input unit 123 that provides an input interface between the mobile terminal 100 and the user, and may also provide an output interface between the mobile terminal 100 and the user.

The interface unit 160 serves as a path to various types of external devices connected to the mobile terminal 100. The interface unit 160 connects a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, and an identification module. It may include at least one of a port, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. In the mobile terminal 100, in response to an external device being connected to the interface unit 160, appropriate control associated with the connected external device may be performed.

In addition, the memory 170 stores data supporting various functions of the mobile terminal 100. The memory 170 may store a plurality of application programs or applications driven in the mobile terminal 100, data for operating the mobile terminal 100, and instructions. At least some of these applications may be downloaded from an external server via wireless communication. In addition, at least some of these application programs may exist on the mobile terminal 100 from the time of shipment for basic functions of the mobile terminal 100 (for example, a call forwarding, a calling function, a message receiving, and a calling function). The application program may be stored in the memory 170 and installed on the mobile terminal 100 to be driven by the controller 180 to perform an operation (or function) of the mobile terminal.

In addition to the operation related to the application program, the controller 180 typically controls the overall operation of the mobile terminal 100. The controller 180 may provide or process information or a function appropriate to a user by processing signals, data, information, and the like, which are input or output through the above-described components, or by driving an application program stored in the memory 170.

In addition, the controller 180 may control at least some of the components described with reference to FIG. 1A in order to drive an application program stored in the memory 170. Furthermore, the controller 180 may operate by combining at least two or more of the components included in the mobile terminal 100 to drive the application program.

The power supply unit 190 receives power from an external power source and an internal power source under the control of the controller 180 to supply power to each component included in the mobile terminal 100. The power supply unit 190 includes a battery, which may be a built-in battery or a replaceable battery.

At least some of the components may operate in cooperation with each other to implement an operation, control, or control method of the mobile terminal according to various embodiments described below. In addition, the operation, control, or control method of the mobile terminal may be implemented on the mobile terminal by driving at least one application program stored in the memory 170.

Hereinafter, the components listed above will be described in detail with reference to FIG. 1A before looking at various embodiments implemented through the mobile terminal 100 described above.

First, referring to the wireless communication unit 110, the broadcast receiving module 111 of the wireless communication unit 110 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or switching of broadcast channels for at least two broadcast channels.

The mobile communication module 112 may include technical standards or communication schemes (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), and EV). Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced) and the like to transmit and receive a radio signal with at least one of a base station, an external terminal, a server on a mobile communication network.

The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The wireless internet module 113 refers to a module for wireless internet access and may be embedded or external to the mobile terminal 100. The wireless internet module 113 is configured to transmit and receive wireless signals in a communication network according to wireless internet technologies.

Examples of wireless Internet technologies include Wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), Wireless Fidelity (Wi-Fi) Direct, Digital Living Network Alliance (DLNA), Wireless Broadband (WiBro), and WiMAX (World Interoperability for Microwave Access (HSDPA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), and the like. 113) transmits and receives data according to at least one wireless Internet technology in a range including the Internet technologies not listed above.

In view of the fact that the wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is made through a mobile communication network, the wireless Internet module 113 for performing a wireless Internet access through the mobile communication network 113 ) May be understood as a kind of mobile communication module 112.

The short range communication module 114 is for short range communication, and includes Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and NFC. (Near Field Communication), at least one of Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus) technology can be used to support short-range communication. The short-range communication module 114 may be configured between a mobile terminal 100 and a wireless communication system, between the mobile terminal 100 and another mobile terminal 100, or through the wireless area networks. ) And a network in which the other mobile terminal 100 (or an external server) is located. The short range wireless communication network may be short range wireless personal area networks.

Here, the other mobile terminal 100 is a wearable device capable of exchanging (or interworking) data with the mobile terminal 100 according to the present invention (for example, smartwatch, smart glasses). (smart glass), head mounted display (HMD). The short range communication module 114 may sense (or recognize) a wearable device that can communicate with the mobile terminal 100, around the mobile terminal 100. Further, when the detected wearable device is a device that is authenticated to communicate with the mobile terminal 100 according to the present invention, the controller 180 may include at least a portion of data processed by the mobile terminal 100 in the short range communication module ( The transmission may be transmitted to the wearable device through 114. Therefore, the user of the wearable device may use data processed by the mobile terminal 100 through the wearable device. For example, according to this, when a call is received by the mobile terminal 100, the user performs a phone call through the wearable device or when a message is received by the mobile terminal 100, the received through the wearable device. It is possible to check the message.

The location information module 115 is a module for obtaining a location (or current location) of a mobile terminal, and a representative example thereof is a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, when the mobile terminal utilizes the GPS module, the mobile terminal may acquire the location of the mobile terminal using a signal transmitted from a GPS satellite. As another example, when the mobile terminal utilizes the Wi-Fi module, the mobile terminal may acquire the location of the mobile terminal based on information of the wireless access point (AP) transmitting or receiving the Wi-Fi module and the wireless signal. If necessary, the location information module 115 may perform any function of other modules of the wireless communication unit 110 to substitute or additionally obtain data regarding the location of the mobile terminal. The location information module 115 is a module used to obtain the location (or current location) of the mobile terminal, and is not limited to a module that directly calculates or obtains the location of the mobile terminal.

Next, the input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user. For input of image information, the mobile terminal 100 is one. Alternatively, the plurality of cameras 121 may be provided. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170. On the other hand, the plurality of cameras 121 provided in the mobile terminal 100 may be arranged to form a matrix structure, and through the camera 121 forming a matrix structure in this way, the mobile terminal 100 may have various angles or focuses. The plurality of pieces of image information may be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for implementing a stereoscopic image.

The microphone 122 processes external sound signals into electrical voice data. The processed voice data may be variously used according to a function (or an application program being executed) performed by the mobile terminal 100. Meanwhile, various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in the process of receiving an external sound signal.

The user input unit 123 is for receiving information from a user. When information is input through the user input unit 123, the controller 180 may control an operation of the mobile terminal 100 to correspond to the input information. . The user input unit 123 may be a mechanical input unit (or a mechanical key, for example, a button, a dome switch, a jog wheel, or the like located on the front, rear, or side surfaces of the mobile terminal 100). Jog switch, etc.) and touch input means. As an example, the touch input means may include a virtual key, a soft key, or a visual key displayed on the touch screen through a software process, or a portion other than the touch screen. It may be made of a touch key disposed in the. The virtual key or the visual key may be displayed on the touch screen while having various forms, for example, graphic, text, icon, video, or the like. It can be made of a combination of.

Meanwhile, the sensing unit 140 senses at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a sensing signal corresponding thereto. The controller 180 may control driving or operation of the mobile terminal 100 or perform data processing, function or operation related to an application program installed in the mobile terminal 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without using a mechanical contact by using an electromagnetic force or infrared rays. The proximity sensor 141 may be disposed in an inner region of the mobile terminal covered by the touch screen described above or near the touch screen.

Examples of the proximity sensor 141 include a transmissive photoelectric sensor, a direct reflective photoelectric sensor, a mirror reflective photoelectric sensor, a high frequency oscillating proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is capacitive, the proximity sensor 141 may be configured to detect the proximity of the object by the change of the electric field according to the proximity of the conductive object. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of explanation, an action of allowing the object to be recognized without being in contact with the touch screen so that the object is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching an object on a screen is called a "contact touch." The position at which the object is in close proximity touch on the touch screen means a position where the object is perpendicular to the touch screen when the object is in close proximity touch. The proximity sensor 141 may detect a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). have. Meanwhile, the controller 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern detected through the proximity sensor 141 as described above, and further, provides visual information corresponding to the processed data. It can be output on the touch screen. Further, the controller 180 may control the mobile terminal 100 to process different operations or data (or information) according to whether the touch on the same point on the touch screen is a proximity touch or a touch touch. .

The touch sensor senses a touch (or touch input) applied to the touch screen (or the display unit 151) using at least one of various touch methods such as a resistive film method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method. do.

As an example, the touch sensor may be configured to convert a change in pressure applied to a specific portion of the touch screen or capacitance generated at the specific portion into an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the touch, a capacitance at the touch, and the like, when the touch object applying the touch on the touch screen is touched on the touch sensor. Here, the touch object is an object applying a touch to the touch sensor and may be, for example, a finger, a touch pen or a stylus pen, a pointer, or the like.

As such, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and then transmits the corresponding data to the controller 180. As a result, the controller 180 can know which area of the display unit 151 is touched. Here, the touch controller may be a separate component from the controller 180 or may be the controller 180 itself.

Meanwhile, the controller 180 may perform different control or perform the same control according to the type of touch object that touches the touch screen (or a touch key provided in addition to the touch screen). Whether to perform different control or the same control according to the type of touch object may be determined according to the operation state of the mobile terminal 100 or an application program being executed.

Meanwhile, the touch sensor and the proximity sensor described above may be independently or combined, and may be a short (or tap) touch, a long touch, a multi touch, a drag touch on a touch screen. ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, etc. A touch can be sensed.

The ultrasonic sensor may recognize location information of a sensing object using ultrasonic waves. On the other hand, the controller 180 can calculate the position of the wave generation source through the information detected from the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the property that the light is much faster than the ultrasonic wave, that is, the time that the light reaches the optical sensor is much faster than the time when the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generation source may be calculated using a time difference from the time when the ultrasonic wave reaches the light as the reference signal.

On the other hand, the camera 121, which has been described as the configuration of the input unit 120, includes at least one of a camera sensor (eg, CCD, CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to detect a touch of a sensing object with respect to a 3D stereoscopic image. The photo sensor may be stacked on the display element, which is configured to scan the movement of the sensing object in proximity to the touch screen. More specifically, the photo sensor mounts a photo diode and a transistor (TR) in a row / column and scans contents mounted on the photo sensor by using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor calculates coordinates of the sensing object according to the amount of light change, and thus, the position information of the sensing object can be obtained.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven in the mobile terminal 100 or user interface (UI) and graphical user interface (GUI) information according to the execution screen information. .

In addition, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

The stereoscopic display unit may be a three-dimensional display method such as a stereoscopic method (glasses method), an auto stereoscopic method (glasses-free method), a projection method (holographic method).

The sound output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output unit 152 may also output a sound signal related to a function (for example, a call signal reception sound or a message reception sound) performed in the mobile terminal 100. The sound output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various haptic effects that a user can feel. A representative example of the tactile effect generated by the haptic module 153 may be vibration. The intensity and pattern of vibration generated by the haptic module 153 may be controlled by the user's selection or the setting of the controller. For example, the haptic module 153 may synthesize different vibrations and output or sequentially output them.

In addition to vibration, the haptic module 153 may be used to stimulate pins that vertically move with respect to the contact skin surface, jetting force or suction force of air through the jetting or suction port, grazing to the skin surface, contact of electrodes, and electrostatic force. Various tactile effects can be generated, such as effects by the endothermic and the reproduction of a sense of cold using the elements capable of endotherm or heat generation.

The haptic module 153 may not only deliver a tactile effect through direct contact, but also may allow a user to feel the tactile effect through a muscle sense such as a finger or an arm. Two or more haptic modules 153 may be provided according to a configuration aspect of the mobile terminal 100.

The light output unit 154 outputs a signal for notifying occurrence of an event by using light of a light source of the mobile terminal 100. Examples of events occurring in the mobile terminal 100 may be message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the light output unit 154 is implemented as the mobile terminal emits light of a single color or a plurality of colors to the front or the rear. The signal output may be terminated by the mobile terminal detecting the user's event confirmation.

The interface unit 160 serves as a path to all external devices connected to the mobile terminal 100. The interface unit 160 receives data from an external device, receives power, transfers the power to each component inside the mobile terminal 100, or transmits data inside the mobile terminal 100 to an external device. For example, a port connecting a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, and an identification module. The port, audio input / output (I / O) port, video input / output (I / O) port, earphone port, etc. may be included in the interface unit 160.

On the other hand, the identification module is a chip that stores a variety of information for authenticating the usage rights of the mobile terminal 100, a user identification module (UIM), subscriber identity module (SIM), universal user authentication And a universal subscriber identity module (USIM). A device equipped with an identification module (hereinafter referred to as an 'identification device') may be manufactured in the form of a smart card. Therefore, the identification device may be connected to the terminal 100 through the interface unit 160.

In addition, when the mobile terminal 100 is connected to an external cradle, the interface unit 160 may be a passage for supplying power from the cradle to the mobile terminal 100 or may be input from the cradle by a user. Various command signals may be a passage through which the mobile terminal 100 is transmitted. Various command signals or power input from the cradle may operate as signals for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and may temporarily store input / output data (for example, a phone book, a message, a still image, a video, etc.). The memory 170 may store data regarding vibration and sound of various patterns output when a touch input on the touch screen is performed.

The memory 170 may include a flash memory type, a hard disk type, a solid state disk type, an SSD type, a silicon disk drive type, and a multimedia card micro type. ), Card-type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read It may include at least one type of storage medium of -only memory (PROM), programmable read-only memory (PROM), magnetic memory, magnetic disk and optical disk. The mobile terminal 100 may be operated in connection with a web storage that performs a storage function of the memory 170 on the Internet.

On the other hand, as described above, the controller 180 controls the operation related to the application program, and generally the overall operation of the mobile terminal 100. For example, if the state of the mobile terminal satisfies a set condition, the controller 180 may execute or release a lock state that restricts input of a user's control command to applications.

In addition, the controller 180 may perform control and processing related to voice call, data communication, video call, or the like, or may perform pattern recognition processing for recognizing handwriting input or drawing input performed on a touch screen as text and images, respectively. Can be. Furthermore, the controller 180 may control any one or a plurality of components described above in order to implement various embodiments described below on the mobile terminal 100 according to the present invention.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component. The power supply unit 190 includes a battery, and the battery may be a built-in battery configured to be rechargeable, and may be detachably coupled to the terminal body for charging.

In addition, the power supply unit 190 may be provided with a connection port, the connection port may be configured as an example of the interface 160 is electrically connected to the external charger for supplying power for charging the battery.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 uses one or more of an inductive coupling based on a magnetic induction phenomenon or a magnetic resonance coupling based on an electromagnetic resonance phenomenon from an external wireless power transmitter. Power can be delivered.

Meanwhile, various embodiments of the present disclosure may be implemented in a recording medium readable by a computer or a similar device using, for example, software, hardware, or a combination thereof.

Hereinafter, as illustrated in FIGS. 1B and 1C, the display unit 151, the first sound output unit 152a, the proximity sensor 141, the illuminance sensor 142, and the light output unit may be disposed on the front surface of the terminal body. 154, the front camera 121a and the first operation unit 123a are disposed, and the second operation unit 123b, the microphone 122, and the interface unit 160 are disposed on the side of the terminal body. The mobile terminal 100 having a dual camera structure of the second sound output unit 152b and a plurality of cameras, that is, the first camera 121b and the second camera 121c, will be described as an example.

However, these configurations are not limited to this arrangement. These configurations may be excluded or replaced as needed or disposed on other sides. For example, the first manipulation unit 123a may not be provided on the front surface of the terminal body, and the second sound output unit 152b may be provided on the side of the terminal body instead of the rear surface of the terminal body.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven in the mobile terminal 100 or user interface (UI) and graphical user interface (GUI) information according to the execution screen information. .

The display unit 151 may include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (flexible display). display, a 3D display, or an e-ink display.

In addition, two or more display units 151 may exist according to an implementation form of the mobile terminal 100. In this case, the plurality of display units may be spaced apart or integrally disposed on one surface of the mobile terminal 100, or may be disposed on different surfaces.

The display unit 151 may include a touch sensor that senses a touch on the display unit 151 so as to receive a control command by a touch method. By using this, when a touch is made to the display unit 151, the touch sensor may sense the touch, and the controller 180 may generate a control command corresponding to the touch based on the touch sensor. The content input by the touch method may be letters or numbers, or menu items that can be indicated or designated in various modes.

Meanwhile, the touch sensor is formed of a film having a touch pattern and disposed between the window 151a and the display (not shown) on the rear surface of the window 151a or directly patterned on the rear surface of the window 151a. It can also be Alternatively, the touch sensor may be integrally formed with the display. For example, the touch sensor may be disposed on a substrate of the display or provided inside the display.

As such, the display unit 151 may form a touch screen together with the touch sensor, and in this case, the touch screen may function as the user input unit 123 (see FIG. 1A). In some cases, the touch screen may replace at least some functions of the first manipulation unit 123a.

The first sound output unit 152a may be implemented as a receiver for transmitting a call sound to the user's ear, and the second sound output unit 152b may be a loud speaker for outputting various alarm sounds or multimedia reproduction sounds. It can be implemented in the form of).

A sound hole for emitting sound generated from the first sound output unit 152a may be formed in the window 151a of the display unit 151. However, the present invention is not limited thereto, and the sound may be configured to be emitted along an assembly gap between the structures (for example, a gap between the window 151a and the front case 101). In this case, an externally formed hole may be invisible or hidden for sound output, thereby simplifying the appearance of the mobile terminal 100.

The light output unit 154 is configured to output light for notifying when an event occurs. Examples of the event may include message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like. When the user's event confirmation is detected, the controller 180 may control the light output unit 154 to end the light output.

The front camera 121a processes an image frame of a still image or a moving image obtained by the image sensor in a shooting mode or a video call mode. The processed image frame may be displayed on the display unit 151 and stored in the memory 170.

The first and second manipulation units 123a and 123b may be collectively referred to as a manipulating portion as an example of the user input unit 123 operated to receive a command for controlling the operation of the mobile terminal 100. have. The first and second manipulation units 123a and 123b may be adopted in any manner as long as the user is tactile manner such as touch, push, scroll, and the like while the user is tactile. In addition, the first and second manipulation units 123a and 123b may be employed in such a manner that the first and second manipulation units 123a and 123b are operated without a tactile feeling by the user through proximity touch, hovering touch, or the like.

In the drawing, the first operation unit 123a is illustrated as being a touch key, but the present invention is not limited thereto. For example, the first manipulation unit 123a may be a mechanical key or a combination of a touch key and a push key.

The contents input by the first and second manipulation units 123a and 123b may be variously set. For example, the first operation unit 123a receives a command such as a menu, a home key, a cancellation, a search, etc., and the second operation unit 123b is output from the first or second sound output units 152a and 152b. The user may receive a command such as adjusting the volume of the sound and switching to the touch recognition mode of the display unit 151.

Meanwhile, as another example of the user input unit 123, a rear input unit (not shown) may be provided on the rear surface of the terminal body. The rear input unit is manipulated to receive a command for controlling the operation of the mobile terminal 100, and the input contents may be variously set. For example, commands such as power on / off, start, end, scroll, etc., control of the volume of sound output from the first and second sound output units 152a and 152b, and the touch recognition mode of the display unit 151. Commands such as switching can be received. The rear input unit may be implemented in a form capable of input by touch input, push input, or a combination thereof.

The rear input unit may be disposed to overlap the front display unit 151 in the thickness direction of the terminal body. For example, the rear input unit may be disposed at the rear upper end of the terminal body so that the user can easily manipulate the index body when the user grips the terminal body with one hand. However, the present invention is not necessarily limited thereto, and the position of the rear input unit may be changed.

As such, when the rear input unit is provided at the rear of the terminal body, a new type user interface using the same may be implemented. In addition, when the touch screen or the rear input unit described above replaces at least some functions of the first operation unit 123a provided in the front of the terminal body, the first operation unit 123a is not disposed on the front of the terminal body. The display unit 151 may be configured with a larger screen.

Meanwhile, the mobile terminal 100 may be provided with a fingerprint recognition sensor for recognizing a user's fingerprint, and the controller 180 may use fingerprint information detected through the fingerprint recognition sensor as an authentication means. The fingerprint recognition sensor may be embedded in the display unit 151 or the user input unit 123.

The microphone 122 is configured to receive a user's voice, other sounds, and the like. The microphone 122 may be provided at a plurality of locations and configured to receive stereo sound.

The interface unit 160 serves as a path for connecting the mobile terminal 100 to an external device. For example, the interface unit 160 may be connected to another device (eg, an earphone or an external speaker), a port for short-range communication (for example, an infrared port (IrDA Port), or a Bluetooth port (Bluetooth). Port), a wireless LAN port, or the like, or a power supply terminal for supplying power to the mobile terminal 100. The interface unit 160 may be implemented in the form of a socket for receiving an external card such as a subscriber identification module (SIM) or a user identity module (UIM), a memory card for storing information.

Meanwhile, a plurality of cameras may be disposed at the rear of the terminal body. For example, the first camera 121b and the second camera 121c may be disposed on the rear surface of the terminal body to form a dual camera structure. In this case, the first camera 121b and the second camera 121c have a photographing direction substantially opposite to that of the front camera 121a.

The first camera 121b and the second camera 121c may be cameras formed to capture an image at different angles of view. Here, the 'view angle' refers to a photographing range of a lens mounted on a camera, and the larger the angle of view, the wider the image can be taken.

The larger the angle of view of the lens of the camera, the shorter the focal length. Therefore, when taking an image with a lens having a large angle of view, it is possible to take a wide range of images. However, a wide range of images may be distorted by the angle of view, thereby increasing the perspective.

According to the angle of view, the lens may be classified into a wide-angle lens, a standard lens, and a telephoto lens. Here, the standard lens may mean a lens having an angle of view of 45 degrees and a focal length of 50 mm. Based on the angle of view of the standard lens, a lens having an angle of view greater than 45 degrees may be classified into a wide angle lens and a lens having an angle of view smaller than 45 degrees as a telephoto lens.

Meanwhile, in the mobile terminal 100 according to an exemplary embodiment of the present invention, two cameras 121b and 121c having different angles of view may be disposed on the rear surface of the mobile terminal 100. In the following description, an angle of view at which the first camera 121b captures an image is referred to as a first angle of view, and an angle of view at which the second camera 121c captures an image is referred to as a second angle of view. Meanwhile, the two cameras 121b and 121c may be disposed adjacent to each other to have the same photographing direction.

The first camera 121b and the second camera 121c may include a plurality of lenses arranged along at least one line. The plurality of lenses may be arranged in a matrix format. Such a camera may be referred to as an 'array camera'. When the first camera 121b or the second camera 121c is configured as an array camera, images may be photographed in various ways using a plurality of lenses, and images of better quality may be obtained.

The flash 124 may be disposed adjacent to the first camera 121b and the second camera 121c. The flash 124 emits light toward the subject when the subject is photographed by at least one of the first camera 121b and the second camera 121c.

The second sound output unit 152b may be additionally disposed on the terminal body. The second sound output unit 152b may implement a stereo function together with the first sound output unit 152a and may be used to implement a speakerphone mode during a call.

The terminal body may be provided with at least one antenna for wireless communication. The antenna may be built in the terminal body or formed in the case. For example, an antenna that forms part of the broadcast receiving module 111 (refer to FIG. 1A) may be configured to be pulled out from the terminal body. Alternatively, the antenna may be formed in a film type and attached to the inner side of the rear cover 103, or may be configured such that a case including a conductive material functions as an antenna.

The terminal body is provided with a power supply unit 190 (see FIG. 1A) for supplying power to the mobile terminal 100. The power supply unit 190 may include a battery 191 embedded in the terminal body or detachably configured from the outside of the terminal body.

The battery 191 may be configured to receive power through a power cable connected to the interface unit 160. In addition, the battery 191 may be configured to enable wireless charging through a wireless charger. The wireless charging may be implemented by a magnetic induction method or a resonance method (magnetic resonance method).

Meanwhile, in the drawing, the rear cover 103 is coupled to the rear case 102 to cover the battery 191 to limit the detachment of the battery 191 and to protect the battery 191 from external shock and foreign matter. To illustrate. When the battery 191 is detachably configured to the terminal body, the rear cover 103 may be detachably coupled to the rear case 102.

An accessory may be added to the mobile terminal 100 to protect the appearance or to assist or expand the function of the mobile terminal 100. An example of such an accessory may be a cover or pouch that covers or accommodates at least one surface of the mobile terminal 100. The cover or pouch may be configured to be linked with the display unit 151 to expand the function of the mobile terminal 100. Another example of the accessory may be a touch pen for assisting or extending a touch input to a touch screen.

On the other hand, the mobile terminal 100 according to the present invention may include a driving unit for changing the posture of the lens included in the camera, in particular, the posture of the image sensor. The driving unit may be coupled to the camera 121, and the posture of the image sensor included in the camera 121, in particular, the camera 121 may be at least one of a roll, pitch, and yaw direction. Can change according to the direction.

The driver may be provided in at least one of the cameras (front camera 121a and cameras 121b and 121c provided on the rear surface) of the mobile terminal 100. 2A illustrates an example in which the driving unit is coupled to the camera, and the driving unit is coupled to the first camera 121b and the second camera 121c, respectively.

Meanwhile, the driving unit 201 may be implemented in various ways and have various structures. Therefore, in this specification, the structure, shape, and positional relationship with the camera of the drive unit 201 are not specifically limited.

Meanwhile, in the mobile terminal 100 according to the present invention, the focus of each of the cameras 121b and 121c photographing the subject may be automatically adjusted by using the driving unit 201.

More specifically, in the present invention, when the image sensor is activated for image capturing, the movement or movement of the subject to be photographed may be sensed. The process of sensing the movement or movement of the subject may be performed by the controller 180 or the sensing unit 140 under the control of the controller 180.

In the present invention, the movement of the subject (or movement, hereinafter referred to as "movement") may be sensed in various ways. For example, the controller 180 senses the graphic object corresponding to the subject in the image input through the image sensor and senses the positional movement of the graphic object in the input image, thereby sensing the movement of the subject as a result. Can be. As another example, the controller 180 may actually sense the subject and sense the movement of the sensed subject through a hardware configuration such as an infrared sensor.

When the movement of the subject is sensed as described above, the controller 180 may control the driving unit to automatically adjust the focus according to the movement of the subject. For example, when a distance from the subject is sensed by an infrared sensor or a laser sensor included in the sensing unit 140, the controller 180 may determine the angle according to depth information determined based on the sensed distance information. The driver 201 may be controlled to change the posture and the position of the image sensor included in the camera.

Then, the image sensor 121b or 121c may be moved under the control of the driving unit 201. When the image sensor 121b or 121c is moved, the preview image may be displayed in the region where the focus is matched according to the sensed distance from the subject. 2B shows an example in which the focus is automatically adjusted by moving the image sensor 121b or 121c under the control of the driving unit 201.

Firstly, the first drawing of FIG. 2B shows an example in which the position of the image sensor is adjusted by the driving unit 201. In this case, as shown in the right figure of the first drawing of FIG. 2B, the preview image 230 whose focus is matched according to the position of the adjusted image sensor may be displayed on the display unit 151. In addition, the preview image 230 may display an area that is currently in focus, that is, a focus area 220.

In this state, when the movement of the subject is detected by the sensing unit 140, the driving unit 201 may readjust the position of the image sensor according to a result of detecting the moving subject.

For example, when the distance between the subject and the mobile terminal 100 approaches as the subject moves, the driver 201 may move the position of the image sensor according to the distance change. That is, the driving unit 201 may cause the image sensor to be moved from the first position 210 to the second position 212 as shown in the second view of FIG. 2B.

Meanwhile, as described above, when the image sensor moves, vibrations due to the movement may occur. This vibration may take a predetermined time until it is resolved, and as the predetermined time elapses, as shown in the third drawing of FIG. 2B, the moved image sensor may be stabilized. As shown in the right side of the third drawing of FIG. 2B, the controller 180 can display the preview image 230 on which the focus is automatically adjusted on the moved subject on the display unit 151.

In addition, the driving unit 201 may be utilized to perform image stabilization. Image stabilization refers to changing the posture of the image sensor through the driving unit 201 so as to detect the shaking of the camera, in particular the image sensor, so that the image sensor can move in a direction opposite to the direction in which the image sensor moves. it means.

When the image stabilization is performed, it is possible to prevent the shake image from being taken or the image that is out of focus due to the shaking of the image sensor during image capture.

In detail, the image sensor may move in at least one of a roll, pitch, and yaw direction under the control of the driving unit 201. Therefore, the posture of the image sensor may be changed to face the direction in which the subject moves, and the focus may be automatically adjusted according to the moving subject. Meanwhile, in the present invention, the movement information of the subject may include at least one of a movement direction and a movement speed of the subject.

The controller 180 may extract vector information of the subject by using the sensed movement information of the subject. The posture information of the change target of the image sensor may be obtained by using a difference between the vector information of the subject and the vector information of the main body provided with the image sensor.

The controller 180 may control the driver 201 such that the attitude of the image sensor is changed based on the acquired attitude information. Here, the extracted change target posture information of the image sensor may include vector information corresponding to a value obtained by subtracting vector information of the main body from vector information of the subject.

The controller 180 may change the posture of the image sensor such that the image sensor tracks the subject based on or using vector information corresponding to the subtraction of the vector information. Through this, in the present invention, by allowing the image sensor to track the subject, it is possible to prevent the subject from being photographed with shaking when the moving subject is photographed.

Hereinafter, embodiments related to a control method that can be implemented in the mobile terminal 100 configured as described above will be described with reference to the accompanying drawings. It is apparent to those skilled in the art that the present invention can be embodied in other specific forms without departing from the spirit and essential features of the present invention.

First, FIG. 3 is a flowchart illustrating an operation process in which the mobile terminal 100 performs continuous shooting at high speed while automatically adjusting focus according to an AF function according to an exemplary embodiment of the present invention.

Referring to FIG. 3, the controller 180 of the mobile terminal 100 according to an exemplary embodiment of the present invention may use the image sensor (hereinafter referred to as a first image sensor) and the second camera of the first camera 121b according to a user's selection. An image sensor (hereinafter referred to as a second image sensor) of 121c may be activated (S300). Then, an image may be received through the activated first and second image sensors.

The controller 180 may generate a preview image for continuous shooting according to an embodiment of the present invention, from the images received from the first and second image sensors (S302). In the continuous shooting, the second image sensor performs an auto focus function while the first image sensor captures an image, and when the auto focus adjustment of the second image sensor is completed, the second image sensor performs an image. By capturing the first image sensor to perform the auto focus adjustment, it may mean to capture the image at high speed while automatically adjusting the focus according to the auto focus function.

In order to alternately perform a process in which the first image sensor and the second image sensor perform an autofocus function on the same subject and alternately capture a sensed image when the autofocus function is completed, Both the subject should be included in the image sensed by the first image sensor and the image sensed by the second image sensor. That is, the continuous shooting may be performed only in a region that is simultaneously sensed by the first image sensor and the second image sensor.

However, the images sensed by the first image sensor and the second image sensor may be different. This is because the first image sensor and the second image sensor may be lenses having different angles of view as described above. For example, when the lens of any one of the first camera 121b and the second camera 121c, that is, the image sensor is composed of a wide-angle lens, and the lens of the other camera is composed of a standard lens, the wide-angle lens is formed from the standard lens. Since an image of a wider area may be captured than the photographed area, the image sensed by the first image sensor and the second image sensor may be different from each other. Accordingly, the controller 180 synthesizes the images sensed by the first image sensor and the second image sensor, generates a preview image in which the continuous imageable region and the other region are not distinguished, and generates the preview image. It may be displayed on the display unit 151.

Meanwhile, in the above description, the case where the angles of view of the first image sensor and the second image sensor are different from each other is described. However, when the angles of view of the first image sensor and the second image sensor are the same, the first image sensor and the second image sensor The sensed images may be different.

For example, as shown in FIG. 2A, the first camera 121b and the second camera 121c may be disposed adjacent to each other. Therefore, even if the first camera 121b and the second camera 121c are disposed to face the same direction, the first camera 121b and the second camera 121c may be located in different areas by different positions of the first camera 121b and the second camera 121c. The image can be sensed. That is, although the size of the subject area sensed by the first image sensor of the first camera 121b and the subject area sensed by the second image sensor of the second camera 121c may be the same, the positions may be different. Accordingly, the images sensed by the first image sensor and the second image sensor may be different from each other. Accordingly, the controller 180 synthesizes the images sensed by the first image sensor and the second image sensor, and thus the continuous shooting is performed. A preview image in which a possible area and an other area are distinguished may be displayed on the display unit 151. Examples of generating and displaying a preview image using the images sensed by the first image sensor and the second image sensor will be described in more detail with reference to FIGS. 6A and 6B.

In addition, according to an exemplary embodiment of the present invention, an image captured, that is, captured by the first and second image sensors, may be a region in which the continuous image may be captured, that is, the first image sensor and the second image sensor. It may be an image of a region that is commonly sensed.

Meanwhile, when the preview image is displayed on the display unit 151 in step S300, the controller 180 may receive at least one focus area or subject from the user (S304).

In operation S304, when a user's touch input is detected at a point of the preview image displayed on the display unit 151, the controller 180 may determine a subject based on the detected touch input. For example, the controller 180 may set a specific object among objects included in the current preview image as a subject based on an image included in a predetermined region formed based on the touch input point. Here, the subject may mean an object to which image capturing is performed.

When the subject is set, the controller 180 may detect the movement of the set subject and alternately control the first and second cameras 121b and 121c according to the detected result. Accordingly, the first and second cameras 121b and 121c alternate the autofocusing process and the image capturing process according to the result of sequentially sensing the subject, thereby the first camera 121b performs the autofocusing. While the second camera 121c captures an image, and while capturing the image according to the completion of the auto focus adjustment of the first camera 121b, the second camera 121c may perform auto focus adjustment. There is (S306). As described above, examples of sequentially detecting a specific subject set by the user and performing continuous shooting according to the detection result will be described in detail with reference to FIGS. 9 and 10.

On the other hand, step S304 may be a step of setting at least one focus area selected by a user, not a step of setting a subject. Here, the focus area may mean an area that is preset to match the focus of the image sensor to a specific area.

For example, if a user's touch input is detected at one point of the preview image displayed on the display unit 151 in step S304, the controller 180 may set a predetermined area as a focus area based on the one point. have. Alternatively, when a user's drag input from one point of the preview image to another point is detected, the controller 180 may set an area formed according to the drag input as the focus area.

When the user's touch input or drag input is detected a plurality of times, the controller 180 may set the plurality of focus areas. The set focus areas may be displayed separately. For example, the controller 180 may display the focus areas so as to be distinguishable using graphic objects such as guide lines displayed around the focus areas. Alternatively, the controller 180 may display the focus areas so as to be distinguishable by displaying the contrast and the saturation differently from inside and outside the focus areas.

Meanwhile, the photographing order of the focus areas may be determined according to an application order of a touch input or a drag input that generates each focus area. Then, in step S306, the controller 180 may include a driving unit coupled to the first and second image sensors such that the first and second image sensors are sequentially aligned with respect to the respective focus areas in the photographing order. 201) can be controlled. As described above with reference to FIGS. 7 and 8, continuous shooting is performed based on a plurality of focus areas having a preset shooting order based on a user's touch input or drag input. do.

4 is a detailed flowchart illustrating an operation of the controller 180 performing AF function and capturing an image by crossing the first and second image sensors in the operation process of FIG. 3. For convenience, the following description will be given by taking a case where the first image sensor first captures and stores an image.

Referring to FIG. 4, the controller 180 of the mobile terminal 100 according to an exemplary embodiment of the present invention may determine that the first object is selected or a plurality of focus areas are selected in step S304 of FIG. 3. The image sensor or both the first image sensor and the second image sensor may be controlled to start the AF function, that is, the auto focus function (S400).

For example, when a specific subject is set in operation S304 of FIG. 3, the controller 180 adjusts the focus of the first image sensor based on a result of first detecting the distance to the specific subject. The auto focus adjustment function of the first camera 121b may be started as much as possible.

In addition, the controller 180 may check whether the auto focus adjustment of the first image sensor is completed (S402). If the automatic focus adjustment of the first image sensor is completed, an image sensed by the first image sensor may be captured and stored. Here, the case where the auto focus adjustment of the first image sensor is completed may include a state in which a position of the first image sensor is moved according to a distance between the sensed subject and the first image sensor, and vibrations caused by the movement are eliminated. That is, it may mean that the first image sensor is in a stabilized state.

On the other hand, if it is checked in step S402 that the automatic focus adjustment of the first image sensor is completed, the controller 180 captures and stores an image sensed by the first image sensor and simultaneously focuses the second image sensor. The automatic focusing function of the second camera 121c may be started to be adjusted (S404). Herein, the controller 180 is configured to detect the movement of the subject when the auto focus adjustment of the first image sensor is performed, that is, the second image sensor based on a second sensing distance from the subject. You can adjust the focus. Therefore, in step S404, the controller 180 captures the image sensed by the first image sensor so that the focus of the second image sensor can be adjusted while the image sensed by the first image sensor is captured. The automatic focus adjustment function of the second camera 121c may be started.

Therefore, while the auto focus adjustment of the second image sensor is performed, the image sensed by the first image sensor may be captured and stored. Then, the controller 180 can check whether the auto focus adjustment of the second image sensor is completed (S406).

On the other hand, if it is determined that the automatic focus adjustment of the second image sensor is completed as a result of the check in step S406, the controller 180 captures and stores an image sensed by the second image sensor as opposed to step S404, The autofocus adjustment function of the first camera 121b may be restarted so that the focus of the first image sensor is adjusted (S408).

Herein, the control unit 180 detects the movement of the subject when the auto focus adjustment of the second image sensor is performed, that is, the first image sensor based on a third sensing distance from the subject. You can adjust the focus. Therefore, in step S408, the controller 180 captures the image sensed by the second image sensor so that the focus of the first image sensor can be adjusted while the image sensed by the second image sensor is captured. In addition, the auto focus adjustment function of the first camera 121b may be started.

Meanwhile, when the auto focus adjustment function of the first camera 121b is started in step S408, the controller 180 may proceed to step S402 again to check whether the auto focus adjustment of the first image sensor is completed. . According to the check result, the process proceeds to step S404, and the processes from steps S404 to S408 are repeated until continuous shooting is terminated by the user. Accordingly, the present invention provides that the second image sensor adjusts the focus while the first image sensor captures and stores the sensed image, and when the focus adjustment is completed, the second image sensor captures and stores the image while the first image sensor adjusts the focus. By allowing the image sensor to focus, the image can be captured continuously without the time intervals required for focusing. Accordingly, the present invention not only enables the continuous shooting while the focus is automatically adjusted according to the movement of the subject, and further shortens the time intervals between the captured images, thereby enabling a higher speed continuous shooting.

Meanwhile, in the above description of FIG. 4, an example in which a specific subject is set has been described, but when a plurality of focus areas are set by a user, the same method may be performed as described above.

That is, the controller 180 may control the first camera 121b to start the auto focus control function based on the first focus area among the plurality of preset focus areas. In operation S402, it is checked whether the focus adjustment of the first image sensor is completed, and in step S404, the capturing and storing of the image sensed by the first image sensor is started. The second camera 121c may be controlled to adjust the focus of the second image sensor based on a second focus area among the plurality of focus areas. In operation S406, the process proceeds to operation S408 according to whether the focus adjustment of the second image sensor is completed, and the capturing and storing of the image sensed by the second image sensor are started. The first camera 121b may be controlled to adjust the focus of the first image sensor based on the third focus area. When step S408 is completed, the process proceeds to step S402 again, and the processes from step S404 to step S408 may be repeated.

Meanwhile, when a plurality of focus areas are set as described above, the controller 180 may perform focus adjustment on the second focus area in advance. In this case, before the process of capturing the image by the first image sensor is still performed, that is, before the first image is captured, the controller 180 may determine that the first image sensor and the second image sensor are respectively the first focal region. The first camera 121b and the second camera 121c may be simultaneously controlled to adjust the focus based on the second focus area.

5 is a conceptual diagram illustrating an operation process performed by the first and second cameras 121b and 121c according to the passage of time in the mobile terminal 100 according to an exemplary embodiment of the present invention.

5, the first camera 121b and the second camera 121c of the mobile terminal 100 according to an exemplary embodiment of the present invention, if continuous shooting starts under the control of the controller 180, are currently set. Focus adjustment may be started with respect to the first focus area among the plurality of focus areas (S500 and S512). Herein, the controller 180 may control the first camera 121b to adjust the focus according to the first focus area among the plurality of focus areas, and the second camera 121c may control two of the plurality of focus areas. The focus may be controlled to be adjusted according to the first focus area. On the other hand, if the continuous shooting is performed on a specific subject, the controller 180 may control only the first camera 121b to adjust the focus based on a detection result first detected for the subject.

The controller 180 may capture and store an image sensed by the first image sensor according to whether the focus adjustment of the first camera 121b is completed (S502).

Meanwhile, when the continuous shooting is performed on a specific subject, the step S512 of performing focus adjustment of the second camera 121c may be performed while the step S502 of capturing an image by the first image sensor is performed. In this case, the focus adjustment of the second camera may be performed based on a result of detecting the object, that is, the second detection of the subject, when the focus adjustment of the first camera 121b is performed in step S500.

Meanwhile, when focus adjustment of the second camera 121c started in step S512 is completed, the controller 180 can start capturing and storing the image sensed by the second image sensor (S514). In addition, the first camera 121b may be controlled to start adjusting the focus of the first image sensor at the same time as starting the step S514 (S504). The controller 180 may adjust the focus of the first image sensor according to a third region of the plurality of focus regions or based on a third sensing result of the specific subject. Accordingly, while the second camera 121c detects the second subject or the image sensed according to the focus adjusted based on the second focus area, the first camera 121b captures the third subject. The focus of the first image sensor may be adjusted based on the detected result or the third focus area.

The controller 180 of the mobile terminal 100 according to an exemplary embodiment of the present invention may, while the image is captured by one image sensor in the manner described above, the other image sensor may detect a focus area or a subject according to the above order. The focus can be adjusted according to the detection result again. Accordingly, in the present invention, as shown in FIG. 5, the first image sensor and the second image sensor alternately adjust the focus according to the plurality of focus areas having a predetermined order or the results of sequentially detecting the subject. You can capture an image. Accordingly, while automatically adjusting the focus, the photographing may be continuously performed without time intervals due to the focus adjustment, thereby enabling high speed continuous photographing.

Meanwhile, in the above description, the operation of performing continuous shooting by the controller 180 of the mobile terminal 100 according to an embodiment of the present invention has been described in detail with reference to a plurality of flowcharts.

In the following description, an example of generating a preview image for continuous shooting from images sensed by a plurality of image sensors in the mobile terminal 100 according to an embodiment of the present invention, and a plurality of focus areas set by a user Alternatively, examples in which continuous shooting is performed according to the movement of the subject will be described in more detail with reference to a plurality of exemplary views.

6A and 6B illustrate examples of generating and displaying a preview image based on images sensed by the first and second image sensors in the mobile terminal 100 according to an exemplary embodiment of the present invention.

First, referring to FIG. 6A, FIG. 6A illustrates an example in which the first image sensor included in the first camera 121b and the second image sensor provided in the second camera 121c have the same angle of view. It is. In this case, even if the first camera 121b and the second camera 121c have the same angle of view, the first image sensor and the second camera 121c may be disposed according to the position where the first camera 121b and the second camera 121c are disposed. Images sensed by the second image sensor may be different from each other.

For example, when the first camera 121b and the second camera 121c are arranged side by side adjacent to each other in the horizontal direction (when the first camera 121b is disposed on the left side of the second camera 121c). 6A illustrates an example of an image sensed by the first image sensor (hereinafter, referred to as a first image 600), and an upper right diagram illustrates an image sensed by the second image sensor. 2 shows an example of the image 610.

As described above, when the first image 600 and the second image 650 are sensed by the first image sensor and the second image sensor, the controller 180 may focus on a portion common to each other from the images. And a second image 650 may be synthesized. That is, when the first camera 121b and the second camera 121c are arranged adjacent to each other in the horizontal direction side by side as described above, and the first camera 121b is disposed on the left side of the second camera 121c. If not, an area where the first image 600 and the second image 650 do not overlap may be on the left side of the first image 600 and on the right side of the second image 650.

That is, as shown in the drawing of the middle portion of FIG. 6A, when the first image 600 and the second image 610 overlap with each other in common with each other, the overlapped image is the first image 600. ) And an area 620 overlapping the second image 610, an area 622 included only in the first image 600, and an area 624 included only in the second image 610. have.

Meanwhile, as shown in the drawing of FIG. 6A, the controller 180 converts the image overlapping the first image 600 and the second image 610 into one image 630 based on a region common to each other. Can be synthesized. In this case, the synthesized image may include the region 622 of the region 622 included only in the first image 600, on both sides of the region 620 in which the first image 600 and the second image 610 overlap each other. An image and an image of an area 624 included only in the second image 610 may be connected to each other. In addition, the controller 180 may display the synthesized image 630 on the display unit 151 as preview images generated from the first and second image sensors.

Meanwhile, as described above, in the case of continuous shooting according to an exemplary embodiment of the present invention, high speed continuous shooting may be performed by allowing two image sensors to alternately capture an image of the same subject or the same area. Hereinafter, continuous shooting that is performed at high speed by allowing another image sensor to adjust focus while one image sensor captures an image will be referred to as 'fast burst shot'.

 Meanwhile, the high speed continuous shooting may be performed only in a region that is commonly sensed by the two image sensors. Accordingly, when the preview image 630 in which the first image 600 and the second image 610 are synthesized is generated, the high speed continuous shooting may be performed by the first image 600 and the first image 600 of the preview image 630. This may be performed only when an image of the subject is included in the region 620 in which the two images 610 are overlapped or a plurality of focus regions are set in the overlapping region.

Accordingly, the controller 180 can display a region capable of high-speed continuous shooting from the preview image 630 so that a user can identify it. For example, as shown in the lower view of FIG. 6A, the controller 180 may include one region 642 (hereinafter referred to as second-second) in the preview image 630 corresponding to the region 622 sensed only by the first image sensor. A graphic object such as a preset guide line around a region 644 (hereinafter referred to as region 2-2) in the preview image 630 corresponding to the region 624 sensed only by the second image sensor. Or by displaying brightness or saturation of the inside of the second-first area 642 and the second-second area 644 differently, the second-first area 642 and the second-second area ( In operation 644, the high speed continuous shooting may not be performed.

Alternatively, a graphic such as a guide line around a region 640 (hereinafter referred to as a first region) in the preview image 630 corresponding to the region 622 commonly sensed by the first image sensor and the second image sensor. By displaying an object or displaying brightness or saturation differently, the corresponding area may be an area in which high speed continuous shooting may be performed. Alternatively, the controller 180 may display only the image of the first region 640 as the preview image displayed on the display unit 151, and display only the region on which the high-speed continuous shooting is possible on the display unit 151. Of course.

Meanwhile, referring to FIG. 6B, FIG. 6B illustrates an example in which the first image sensor included in the first camera 121b and the second image sensor included in the second camera 121c have different angles of view. It is. As such, when the first camera 121b and the second camera 121c have different angles of view, the size of the area sensed by the first image sensor and the size of the area sensed by the second image sensor are different from each other.

For example, assuming that an angle of view of the first image sensor is a standard angle of view and that of the second image sensor is a wide angle of view, an upper left view of FIG. 6B is an image sensed by the first image sensor (hereinafter, referred to as a first image). The upper right figure shows an example of an image sensed by the second image sensor (hereinafter referred to as a second image 660).

As such, when the first image 650 and the second image 660 are sensed by the first image sensor and the second image sensor having different angles of view, the controller 180 is configured to focus on a portion common to each other from the images. The first image 650 and the second image 660 may be synthesized. Therefore, when the angle of view of the second image sensor is greater than the angle of view of the first image sensor, the area covered by the image sensed by the first image sensor is included in the area covered by the image sensed by the second image sensor. Can be.

The controller 180 may overlap the first image 650 and the second image 660 with respect to areas common to each other. In this case, as shown in the drawing of the interruption portion of FIG. 6B, the overlapped image includes an area 652 in which the first image 650 and the second image 660 overlap and the second image 660. It may be divided into an area 662 included only.

On the other hand, as shown in the drawing of FIG. 6B, the controller 180 converts the image overlapping the first image 650 and the second image 660 into one image 670 based on a region common to each other. Can be synthesized. In this case, the synthesized image 670 is an area 662 included only in the second image 660 around the image of the region 652 in which the first image 650 and the second image 660 overlap. ) May be an image combined with each other. The controller 180 may display the synthesized image 670 on the display unit 151 as preview images generated from the first and second image sensors.

Meanwhile, as described above, the controller 180 synthesizes the first image 650 and the second image 660 based on areas common to each other, and the first image 650 or the second image 660. Of course, it is also possible to adjust the magnification of any one to make the size of the objects displayed on both images the same.

For example, since the image sensor having the wide angle senses an image of a wider area as described above, the size of an object sensed may be smaller than the image sensed by the image sensor having the standard angle. In this case, the controller 180 reduces the image sensed by the image sensor having a narrower angle of view or enlarges the image sensed by the image sensor having a wider angle of view, thereby allowing the same object included in the image sensed by both image sensors. You can ensure that the sizes of are the same.

Meanwhile, as shown in the interruption diagram of FIG. 6B, when the synthesized image 670 including the first image 650 and the second image 660 is generated, the controller 180 selects the synthesized image 670. The preview images generated from the first and second image sensors may be displayed on the display unit 151. In this case, the synthesized image 670 may be reduced in size according to the size of the displayable area on the display unit 151.

On the other hand, the controller 180 can display the area capable of high-speed continuous shooting from the preview image 670 so that the user can identify. For example, as shown in the lower view of FIG. 6B, the controller 180 may control the inside of the preview image 670 corresponding to the area 652 commonly sensed by the first image sensor and the second image sensor. By displaying a graphic object such as a guide line around a region 680 (hereinafter referred to as a first region), or displaying the inside of the first region 680 differently in brightness or saturation, the region may be subjected to high speed continuous shooting. It can also indicate that the area can be.

Alternatively, a graphic object such as a preset guide line may be displayed around a region 682 (hereinafter referred to as a second region) of the preview image 670 corresponding to the region 662 sensed only by the second image sensor. By displaying the area corresponding to the second area 682 at a brightness or saturation different from that of the first area 680, the second area 682 may be an area where high speed continuous shooting is not performed. Alternatively, the controller 180 may display only the image of the first region 680 as the preview image displayed on the display unit 151, and display only the region capable of high speed continuous shooting on the display unit 151. Of course.

6A and 6B, when a preview image is generated from a plurality of image sensors, the controller 180 receives a plurality of subjects or a plurality of subjects from the user's input applied to the preview image. The focus area may be set.

In the following description of FIGS. 7 to 11, examples of performing high speed continuous shooting on a subject or a focus area set according to the user's input will be described in detail with reference to the accompanying drawings. In the description of FIG. 7 to FIG. 11, for convenience of description, the case where only the region capable of high-speed continuous shooting is displayed on the display unit 151 will be described as an example.

First, FIG. 7 illustrates an example of performing continuous shooting according to an embodiment of the present invention with respect to focus areas preset by a user in the mobile terminal 100 according to an embodiment of the present invention.

Referring to the first drawing of FIG. 7, the first drawing of FIG. 7 illustrates a case in which a user's touch input is first applied while a region capable of high-speed continuous shooting is displayed as a preview image as described above. . As shown in the first drawing of FIG. 7, when a user's touch input 700 is first detected at a point in the preview image, the controller 180 may be configured based on the point where the touch input 700 is applied. One region 702 of the preview image may be set as a focus region. Meanwhile, the focus area formed at one point where the touch input 700 is first detected may be set as the first focus area, that is, the first focus area.

Meanwhile, the controller 180 may detect touch inputs that are continuously applied to the display unit 151. In this case, the controller 180 may set different focus areas with respect to each of the continuously applied touch inputs based on the points where the touch inputs are applied. That is, if four touch inputs are further applied after the first touch input 700 is applied in the first drawing of FIG. 7, the controller 180 is as shown in the second drawing of FIG. 7. The second to fifth focus regions 704, 706, 708, and 710 may be further set according to an order in which touch inputs are further applied. In this case, each of the focal regions 702, 704, 706, 708, and 710 may have an order in which the touch inputs corresponding to the respective focal regions are applied, which is shown in the second drawing of FIG. 7. As described above, a graphic object indicating an order set in each focus area may be displayed around or inside each focus area so that a user can identify an order set in each focus area.

Meanwhile, the controller 180 may perform high speed continuous shooting in the state where the plurality of focus areas are set. For example, the controller 180 may start the high speed continuous shooting when the start of the high speed continuous shooting is selected by the user. Alternatively, as shown in the third drawing of FIG. 7, a change in one of the currently set focus areas (for example, the first focus area 702), that is, the subject 720 is detected as the subject 720 is detected. When a change of more than a predetermined level occurs, the high speed continuous shooting can be started automatically.

Meanwhile, when the high speed continuous shooting is performed, the controller 180 may allow the first image sensor and the second image sensor to cross each other and perform focus adjustment and image capture according to each focus area as described above. That is, when the first image sensor captures an image in which the focus is adjusted in the first focus area 702, the second image sensor may capture an image in which the focus is adjusted in the second focus area 704. have. And you can repeat this.

Therefore, if the first image sensor captures an image in which the focus is adjusted to the first focus area 702, the first image sensor may include the first focus area 702, the third focus area 706, and the fifth image sensor. The focused image may be captured according to the focus area 710. Also, the second image sensor may capture images whose focus is adjusted according to the second focus area 704 and the fourth focus area 708. Meanwhile, while the image is captured by the first image sensor, the second image sensor performs a focus adjustment on a focus area next to the image currently captured by the first image sensor, so that each image is continuously captured. Continuous shooting can be achieved without the time required for focusing.

7 illustrates an example in which a predetermined area is automatically set as a focus area when a user's touch input is applied, the focus area may be arbitrarily set by the user. An example in which the focus area is arbitrarily set by the user will be described with reference to FIG. 8.

First, FIG. 8 illustrates a case in which a user's touch input is first applied in a state where a region capable of high-speed continuous shooting is displayed as a preview image as described above. In this case, as shown in the first drawing of FIG. 8, when a user applies a drag input following the touch input, a predetermined area formed by the drag input may be set as the focus area 800. If the touch input is first detected in the preview image, the currently set focus area 800 may be set as the first focus area. In addition, a graphic object indicating a currently set order of the focus area 800 may be displayed around the currently set focus area 800.

Meanwhile, when a new touch input is detected while the first focus area 800 is set, the controller 180 may determine this as a user input for setting the second focus area. A new focus area, that is, a second focus area, may be generated according to the drag input applied after the new touch input. Accordingly, as shown in the second drawing of FIG. 8, a second focal region 810 may be formed, and a graphic object for indicating the order of the second focal regions is displayed around the second focal region 810. Can be.

On the other hand, by repeating the above method, as shown in the third and fourth drawings of FIG. 8, the controller 180 and the third focal region 820 according to the user's drag inputs applied to the preview image. The fourth focus area 830 may be further set. When the user performs continuous shooting while the plurality of focus areas 800, 810, 820, and 830 are set, the first image sensor and the second image sensor are alternately disposed, and the plurality of focus areas 800, 810, 820 and 830 may perform focus adjustment and image capture on each of the focus areas in a predetermined order.

That is, the controller 180 starts the continuous shooting. First, the first image sensor captures a preview image whose focus is adjusted according to the first focus region 800, and the second image sensor adjusts the focus according to the second focus region 810. The first image sensor and the second image sensor may be simultaneously controlled. When the focus adjustment of the second image sensor is completed, the second image sensor captures a preview image whose focus is adjusted according to the second focus area 810, and the first image sensor performs a third image capture. The first image sensor and the second image sensor may be simultaneously controlled to adjust the focus according to the focus area 810. Accordingly, the mobile terminal 100 according to an exemplary embodiment of the present invention may focus on preview images captured while the focus is adjusted according to each of the plurality of focus areas 800, 810, 820, and 830. Can be stored continuously without time intervals.

Meanwhile, the foregoing description has described examples in which a plurality of focus areas are preset by a user. However, unlike this, when a specific subject is selected, the controller 180 of the mobile terminal 100 according to an embodiment of the present invention may sense the selected subject and perform continuous shooting based on the sensed result. . 9 and 10 are exemplary views illustrating examples of performing continuous shooting according to an exemplary embodiment of the present invention with respect to a subject moving at a high speed in the mobile terminal according to an exemplary embodiment of the present invention.

First, FIG. 9 illustrates an example in which an object moving at high speed is selected as a subject. The first drawing of FIG. 9 illustrates an example in which a specific object is set as the subject 902 among images sensed through the preview image. It is. As such, when a specific object is set as the subject 902, a user may identify the currently set subject 902 by displaying a graphic object such as a guide line around the subject 902.

As such, when the subject 902 is set, the controller 180 may sense the subject 902. For example, the controller 180 may use the at least one sensor provided in the sensing unit 140 to determine a distance between the subject 902 and the mobile terminal 100, whether the subject 902 moves, and the subject. The direction of movement of 902 may be sensed.

Meanwhile, when the subject 902 is sensed as described above, the controller 180 may control the first camera 121b to adjust the focus of the first image sensor according to the sensing result. Therefore, as shown in the first drawing of FIG. 9, when the subject 902 is set, the controller 180 adjusts the focus of the first image sensor based on the first sensing result (first sensing result) of the subject 902. Can be.

Meanwhile, the controller 180 may detect the subject 902 again while the focus of the first image sensor is adjusted. Accordingly, the subject 902 moving to the second position 912 may be sensed according to the fast movement shown in the second drawing of FIG. 9. Accordingly, as a result of detecting the object 902 again, that is, the second sensing result has a different position value and a different distance value (position value and distance value according to the second position 912) from the first sensing result. Can be.

The controller 180 may check whether focus adjustment of the first image sensor is completed. When the focus adjustment of the first image sensor is completed, the controller 180 may capture and store the current preview image. The controller 180 may control the second camera 121b to capture the preview image sensed by the first image sensor and to simultaneously adjust the focus of the second image sensor. In addition, the focus adjustment of the second image sensor may be performed according to the second sensing result, which is performed while the focus of the first image sensor is adjusted.

Meanwhile, as described above, while the focus of the second image sensor is adjusted, the controller 180 may detect the subject 902 again. Accordingly, the subject 902 moving to the third position 914 may be sensed according to the fast movement shown in the third drawing of FIG. 9. Accordingly, as a result of detecting the object 902 again, that is, the third sensing result, the second sensing result may have a different position value and a different distance value (position value and distance value according to the third position 914). Can be.

In addition, the controller 180 may check whether focus adjustment of the second image sensor is completed. When the focus adjustment of the second image sensor is completed, the preview image sensed by the second image sensor may be captured. At the same time, the controller 180 can control the first camera 121a to adjust the focus of the first image sensor according to the third sensing result while the second image sensor captures the preview image.

By repeating this process, the controller 180 may sense the fast-moving subject 902 a plurality of times, and may acquire images whose focus is adjusted based on the sensing results. For example, as shown in the fourth drawing of FIG. 9, the results of sensing the subject 902 during continuous shooting may indicate that the subject 902 is at the first position 900 to the fifth position 918. If the respective times correspond to each other, the controller 180 may capture and store the pre-images whose focus is adjusted according to the distance between the subject 902 and the mobile terminal 100 at each position. That is, when continuously photographing the fast-moving subject 902 as shown in FIG. 9, five images having different distances between the subject 902 and the mobile terminal 100 may be continuously captured.

Meanwhile, as described above, when continuously photographing a fast moving subject, the controller 180 may adjust the focus by tracking a subject moved by another image sensor while one image sensor performs image capture. Accordingly, the present invention allows the focus to be automatically adjusted according to the fast-moving subject, while also allowing high-speed continuous shooting.

Meanwhile, FIG. 9 illustrates a case in which the mobile terminal 100 directly senses a fast moving subject and performs continuous shooting according to the sensed result. However, the mobile terminal 100 according to an embodiment of the present invention Of course, the moving direction and the moving speed of the subject 902 may be predicted based on the sensing result of the subject 902. The focus of the first image sensor and the second image sensor may be adjusted in advance by controlling the first and second cameras according to the predicted result. That is, the controller 180 sets a plurality of focus regions on the predicted movement trajectory of the subject 902 according to the moving direction and the moving speed of the subject 902, and the first image sensor for the set focus regions. The second image sensor and the second image sensor alternately control to perform focus adjustment and image capture, so that continuous shooting of the fast-moving subject may be performed.

Meanwhile, in the case of predicting the moving direction and the moving speed of the subject 902, the controller 180 may predict the moving direction and the moving speed of the subject 902 based on information previously input from the user.

FIG. 10 illustrates an example of performing continuous shooting according to an exemplary embodiment of the present disclosure in response to a user's input to a moving subject in the mobile terminal 100 according to an exemplary embodiment of the present disclosure.

First, FIG. 10 illustrates an example in which a specific object is set as the subject 1000 among images sensed through the preview image. As such, when a specific object is set as the subject 1000, the user may identify the currently set subject 1000 by displaying a graphic object such as a guide line around the subject 902.

Meanwhile, while the subject 1000 is set as described above, the controller 180 can detect a user's input applied to the preview image. For example, as shown in the second drawing of FIG. 9, the user input may be a drag input 1010 leading from one point of the currently set subject 1000 to another point 1002. The controller 180 may predict the movement state of the currently set subject 1000 based on the drag input.

For example, the controller 180 may set the movement trajectory of the currently set subject 1000 according to the trajectory of the drag input applied in the second drawing of FIG. 10. In addition, whether the drag input is applied in a direction away from the mobile terminal 100 or in a direction approaching may be set in advance. In this case, if it is set to be applied in a direction away from the mobile terminal 100, the controller 180 determines that the currently set subject 1000 is moved from the mobile terminal 100 according to the trajectory of the drag input. The movement of the subject 1000 may be predicted by moving in a direction farther away. On the other hand, if it is set to be applied in a direction closer to the mobile terminal 100, the controller 180 is further from the mobile terminal 100 according to the trajectory of the drag input, the current object 1000 is set from the start position of the drag input The movement of the subject 1000 may be predicted by moving in a direction that is approaching.

Meanwhile, the controller 180 may determine whether the drag input is applied in a direction away from or in a direction closer to the mobile terminal 100 based on the start position and the end position of the drag input. For example, the controller 180 may determine this based on the preview image displayed on the display unit 151. That is, the controller 180 is based on the distance values sensed for the respective points corresponding to the start position and the end position of the drag input in the preview image, at a point where the drag input is closer to the mobile terminal 100. It may be determined whether it is applied to a farther point or to a point closer to the farther point. The moving direction of the subject 1000 according to the drag input may be predicted.

Accordingly, when the drag input is applied as shown in the second drawing of FIG. 10, the controller 180 moves the subject 1000 to a point farther from the mobile terminal 100 according to the drag input trajectory. A movement of 1000 can be predicted. Then, as shown in the third drawing of FIG. 10, the controller 180 may display the graphic object 1020 corresponding to the predicted result on the preview image. In this case, the controller 180 may display a movement direction of the currently predicted subject 1000 by varying the thickness at which the graphic object 1020 is displayed according to the distance from the mobile terminal 100.

Meanwhile, when the movement of the subject 1000 is predicted as described above, the controller 180 may perform continuous shooting of the subject 1000 based on the predicted result. In this case, as described above, the controller 180 controls the first and second image sensors to alternately focus and capture an image based on each position of the subject 1000 according to the prediction. Can be controlled.

Meanwhile, in the above description, a case in which a plurality of focus areas are preset by a user or a continuous shooting is performed on a specific set subject is taken as an example. Alternatively, the above-described description is based on an input continuously applied by the user. Of course, continuous shooting can also be performed.

FIG. 11 illustrates an example of performing continuous shooting according to a user's input continuously applied in the mobile terminal 100 according to an exemplary embodiment of the present invention.

First, referring to the first drawing of FIG. 11, the first drawing of FIG. 11 illustrates an example in which the user applies a touch input to a specific object (state located at the first position 1120) displayed on the preview image. It is showing. As such, when a user applies a touch input to a specific object in the preview image, the controller 180 may set an object or person corresponding to the specific object 1100 as a subject, and guide lines around the specific object 1100. The display may be distinguished using graphic objects such as the user to identify the currently set subject.

Meanwhile, when the user applies the touch input 1110 to the specific object 1100, the controller 180 senses the subject and adjusts the focus of the first image sensor according to the sensed result (first sensing result). The first camera 121b may be controlled. The subject may be sensed again (second sensing) while the focus of the first image sensor is adjusted.

The subject may be a moving object or a person. Accordingly, as shown in the second drawing of FIG. 11, when the subject moves while the focus of the first image sensor is adjusted, the specific object 1100 corresponding to the subject, that is, the subject of the subject, moves according to the movement of the subject. The image may be moved from the first location 1120 to the second location 1122. Second sensing information including distance information between the subject and the mobile terminal 100, which is sensed by the moving subject, may be generated.

In this case, the controller 180 may check whether the focus of the first image sensor is completed. When the focus adjustment of the first image sensor is completed, the preview image sensed by the first image sensor may be captured and stored. At the same time, the controller 180 can check whether the user maintains a touch input to the specific object 1100 until the focus adjustment of the first image sensor is completed, that is, whether a drag input is applied.

When the touch input for the specific object 1100 is maintained, that is, when a drag input is applied along the specific object 1100, the controller 180 may view the preview image sensed by the first image sensor. During capturing, the second camera 121c may be controlled to adjust the focus of the second image sensor according to a result of sensing the subject again (second sensing result).

The controller 180 may sense (the third sensing) the specific object 1100 again while the focus of the second image sensor is adjusted. As shown in the third drawing of FIG. 11, when the focus adjustment of the second image sensor on the specific object 1100 moved to the second position 1122 is completed, the preview image sensed by the second image sensor is completed. Can be captured. In addition, when the touch input is maintained, the first camera 121b may be controlled to adjust the focus of the first image sensor according to the third sensing result simultaneously with capturing the preview image. And the above process can be repeated.

Therefore, if the user continuously applies a touch input to the image (specific object 1100) of the subject whose position is continuously changed in the preview image as the subject moves, the high speed using the first image sensor and the second image sensor Continuous shooting can be made. Therefore, as shown in the fourth drawing of FIG. 11, when the user's touch input is continuously maintained on the image of the subject whose position is changed according to the movement of the subject, that is, the specific object 1100, the focus is performed at each image sensor. When the adjustment is completed (first position 1120 to fifth position 1128), the preview image may be captured and stored. Therefore, when the user maintains a touch input with respect to the image of the moving subject, that is, when the user applies a drag input along the image of the moving subject in the preview image, images of continuously photographing the moving subject may be stored. Can be.

In the above description, various embodiments of the mobile terminal 100 according to an embodiment of the present invention to perform high speed continuous shooting have been described. In the following description, examples of synthesizing and editing images captured by the continuous shooting will be described with reference to a plurality of exemplary views.

First, FIG. 12 illustrates an example of generating a composite image using captured images in a mobile terminal according to an embodiment of the present invention.

12 illustrates an example of a captured image list 1200 including thumbnail images corresponding to a plurality of captured images as a result of continuous photographing performed according to an exemplary embodiment of the present invention. For example, when continuous shooting is performed according to the embodiment described with reference to FIG. 9, the captured image list 1200 includes thumbnail images 1210, 1212, 1214, 1216, and 1218 shown in the first drawing of FIG. 12. ) May be displayed.

Meanwhile, the captured image list 1200 may include a menu for allowing a user to select various functions related to the captured images. When the user's input is applied to the menu, as shown in the second drawing of FIG. 12, a list 1220 of various functions related to the captured images may be displayed. If any one function is selected, the selected function may be performed.

Meanwhile, as a function related to the captured images, a function of synthesizing the captured images may be performed. For example, when the compositing menu 1222 is selected from the function list 1220 shown in the second drawing of FIG. 12, the controller 180 controls the thumbnail image displayed on the captured image list 1200. Corresponding images can be synthesized. For example, the synthesis may be performed by synthesizing an image of a subject or a focus area included in other images with the first captured image on the background of the first captured image.

That is, as shown in FIG. 9, when the subject, that is, the ball moving at a high speed, is set as the subject, the controller 180 may first select the first captured image (hereinafter, the base image). In addition, only images (subject images) of the English language corresponding to the subject may be extracted from other images except the base image. The extracted subject images may be displayed in each region on the base image corresponding to each of the extracted subject images, thereby generating one image obtained by synthesizing the captured images. Therefore, as shown in the third drawing of FIG. 12, in the first captured image, a composite image in which the images of the balls 1226 extracted from the other images are displayed at the same position as that shown in each of the other captured images ( 1230 may be generated.

Meanwhile, as a function related to the captured images, a function of editing the captured images may be performed. For example, when an editing menu 1224 is selected from the function list 1220 shown in the second drawing of FIG. 12, the controller 180 controls the thumbnail image displayed in the captured image list 1200. Corresponding images can be edited. For example, the editing may be a function of deleting a specific captured image or changing the order of the captured images. Accordingly, the controller 180 may delete an image that is out of focus according to a user's selection, or change the order of captured images according to a user's selection.

On the other hand, the controller 180 of the mobile terminal 100 according to an embodiment of the present invention may detect whether the out of focus image is captured during the continuous shooting. And when an image out of focus is captured, it is possible to further capture a supplementary image to compensate for it.

FIG. 13A illustrates an example in which the supplementary image is further captured according to the focus state of the captured image in the mobile terminal 100 according to the embodiment of the present invention.

Referring to FIG. 13A, the first drawing illustrates an example in which continuous shooting is performed based on a plurality of focus areas preset by a user. In this case, as described above with reference to FIG. 7, when the user sets the first to fourth focusing areas 1300, 1302, 1304, and 1306, the first and second image sensors may adjust focusing and focusing for each focusing area. Continuous shooting can be performed by alternating image capture.

Meanwhile, as shown in the first drawing of FIG. 13A, when performing continuous shooting, the controller 180 may detect a match state of the focus with respect to each of the captured images. For example, when capturing each image, the controller 180 may check whether a blur of a predetermined level or more occurs in a region corresponding to the focus region in the captured image. Here, if continuous shooting is performed on a specific subject instead of a plurality of focus areas, the controller 180 may check whether a captured image or more blur occurs from the image of the subject in the captured image. .

On the other hand, if a blur occurs in a captured image or more than a predetermined level, the controller 180 may further capture a supplementary image to compensate for this. That is, as shown in the first drawing of FIG. 13A, the captured image when the subject is located in the second focus region 1302 and the third focus region 1304, that is, the second captured image and the third captured image. If it is determined that the image is an image that is out of focus, the controller 180 may further set the number of focus areas by the number corresponding to the number of images to supplement the images that are not in focus.

For example, the controller 180 may further set the focus areas based on a result of detecting the movement of the subject. That is, as shown in the second drawing of FIG. 13A, the controller 180 may detect the movement of the subject detected in the focus area or based on a result of sensing the movement of the subject directly selected by the user. At least one focusing area may be set according to the moving speed and the direction of the. In this case, as described above, in the case where there are two images that do not coincide with each other, as shown in the second drawing of FIG. 13A, the controller 180 may include the fifth focusing area 1310 and the sixth focusing area 1312. ) Can be set further.

Meanwhile, when the fifth focusing area 1310 and the sixth focusing area 1312 are further set, the controller 180 may display the fifth focusing area 1310 and the sixth as shown in the third drawing of FIG. 13A. An image in which focus is matched with respect to the focus area 1312 may be further captured. Therefore, even when the user sets only the first to fourth focusing areas 1300, 1302, 1304, and 1306, two focusing points of the fifth focusing area 1310 and the sixth focusing area 1312 are matched. The preview image may be further captured. When the capturing of the focused image is completed according to the sixth focal region 1312, the controller 180 captures the captured image through the captured image list 1200 as shown in the fourth drawing of FIG. 13A. The thumbnail images 1352, 1354, 1355, 1356, 1358, and 1360 corresponding to the displayed images may be displayed.

In this state, the controller 180 may detect a user's input for selecting an editing function of the captured images. For example, as shown in the first drawing of FIG. 13B, the controller 180 controls a user of an area in which an edit menu 1224 of the captured images is displayed while the functions related to the captured images are displayed. Can detect the input of.

Meanwhile, when the edit menu 1224 is selected, the controller 180 may delete thumbnail images 1352, 1354, 1355, 1356, 1358, and 1360 corresponding to the currently captured images, and the thumbnail images. Display a menu. That is, the controller 180 displays an area 1380 for deleting the thumbnail image in one area of the menu screen displayed on the display unit 151 as shown in the second drawing of FIG. 13B. When a user selects a specific thumbnail image and drags it to the deletion area 1380, it may be determined that deletion of a capture image corresponding to the selected thumbnail image is selected.

On the other hand, the controller 180 may display the capture image selected as such to be distinguished from other captured images. For example, when the second captured image and the third captured image of the images captured through the continuous shooting described with reference to FIG. 13A are not in focus, the user may display thumbnail images corresponding to the images. 1352 and 1354 may be dragged to the deletion area 1380 to select deletion of the corresponding images.

The image selected for deletion may be displayed as a dotted line or blurred as shown in the third drawing of FIG. 13B. Alternatively, brightness or saturation may be displayed differently from other thumbnail images, or a graphic object such as a preset guideline may be displayed. The image may be deleted according to whether the user confirms the deletion. Here, whether the user is confirmed may be whether the user inputs a preset menu item 1342.

When the deletion of the selected images is performed, the controller 180 sorts the thumbnail images 1350, 1356. 1358, and 1360 corresponding to the remaining images, according to the captured order, so that the captured image list 1200 ) May be displayed.

Meanwhile, in the above description, it is assumed that continuous shooting is performed according to a user's selection, or that continuous shooting is performed when a subject is detected in a preset focus area. However, in contrast, a subject selected by a user is sensed. Of course, the start and end of the continuous shooting may be controlled based on the preview area.

For example, referring to FIG. 6A to FIG. 6B, the preview image displayed on the display unit 151 in the mobile terminal 100 according to the embodiment of the present invention is shared by the first and second image sensors. Area (the first area) sensed by the sensor and the area sensed only by one of the first and second image sensors (the second area or the 2-1 area and the 2-2 area, the second area below) It has been mentioned that it may include. In addition, as described above, the high speed continuous shooting according to the embodiment of the present invention may be performed in the first area jointly sensed by the first and second image sensors, whereby the first area and the second area may be It has been described that they may be distinguished from each other.

Therefore, the controller 180 of the mobile terminal 100 according to an exemplary embodiment of the present invention can start and end the continuous shooting based on the position where the subject is sensed in the preview image including the first area and the second area. It can also be controlled. More specifically, when the subject is detected in the second area, the controller 180 may cause the camera that first detected the subject to adjust the focus of the image sensor based on a result of sensing the subject. When the subject is detected in the first area, the camera on which the focus adjustment of the image sensor is performed captures an image, and the focus adjustment of the image sensor is performed based on a result of another camera detecting the subject again. By doing so, the above-mentioned high speed continuous shooting may be automatically started.

14A and 14B and FIG. 15 are conceptual diagrams and exemplary diagrams for describing examples in this case.

14A and 14B are exemplary views showing examples of a preview image including regions in which a start and end of the high speed continuous shooting function can be controlled in a mobile terminal according to an exemplary embodiment of the present invention.

First, referring to FIG. 14A, FIG. 14A illustrates examples of preview images that may be generated when two image sensors performing the high speed continuous shooting have the same angle of view.

First, referring to (a) of FIG. 14A, FIG. 14A (a) illustrates that a first camera 121b and a second camera 121c including image sensors having the same angle of view are disposed adjacent to each other in a horizontal direction. An example of the preview image 1420 that may be generated in this case is shown. The first region 1400 of the preview image 1420 may be a region commonly sensed by the first image sensor of the first camera 121b and the second image sensor of the second camera 121c. In the preview image 1420, other regions 1402 and 1404 except for the first region 1400 may be a region that is sensed by only one of the first image sensor and the second image sensor.

In this case, the controller 180 may designate one of the other regions 1402 and 1404 except for the first region as an area related to the start of the high speed continuous shooting (hereinafter referred to as a start area), and the other end of the high speed continuous shooting. It may be determined as an area related to (hereinafter, referred to as an end area). Here, the start area may mean an area in which the controller 180 can start continuous shooting based on whether or not an image of a preset subject is detected, and the end area is currently performed based on whether or not an image of a preset subject is detected. It may mean an area where the continuous shooting can be terminated.

For example, the controller 180 may set a preset specific region among the other regions 1402 and 1404 except for the first region as the start region, and set the other region as the end region. Meanwhile, the controller 180 may set, as a start area, an area in which an image of a preset subject is first detected among other areas 1402 and 1404 except for the first area, and set the other area as an end area. have.

Meanwhile, referring to FIG. 14A (b), FIG. 14A (b) shows that the first camera 121b and the second camera 121c including the image sensor having the same angle of view are disposed adjacent to each other in the vertical direction. An example of the preview image 1430 that may be generated in this case is shown. The first region 1410 of the preview image 1430 may be a region commonly sensed by the first image sensor of the first camera 121b and the second image sensor of the second camera 121c. In the preview image 1430, other regions 1412 and 1414 except for the first region 1410 may be an area which is sensed only by one of the first image sensor and the second image sensor. The other areas 1412 and 1414 except for the first area 1410 may be set to start and end areas as described with reference to FIG. 14A (a).

Meanwhile, referring to FIG. 14B, FIG. 14B illustrates examples of preview images that may be generated when two image sensors performing the high speed continuous shooting have the same angle of view.

First, referring to (a) of FIG. 14B, (a) of FIG. 14B is based on images sensed by the first camera 121b and the second camera 121c having image sensors having different angles of view. An example of the generated preview image 1460 is shown. As described above, when the angle of view of the first image sensor of the first camera 121b and the angle of view of the second image sensor of the second camera 121c are different from each other, the image sensed by the image sensor having a narrower angle of view is more A preview image may be displayed in a picture in picture (PIP) format on an image sensed by an image sensor having a wide angle of view.

The first region 1450 of the preview image 1460 may be a region commonly sensed by the first image sensor of the first camera 121b and the second image sensor of the second camera 121c. In the preview image 1460, other regions except for the first region 1450 (a second region below 1470) may be a region that is sensed only by an image sensor having a wider angle of view among the first image sensor and the second image sensor. Can be.

Then, the controller 180 may divide the second area 1470 into two areas. For example, as shown in (a) of FIG. 14B, the controller 180 divides the second area 1470 vertically based on the center of the preview image 1460 and divides the second area 1470 into two areas 1452 and 1454. Can be partitioned Alternatively, as shown in (b) of FIG. 14B, the second area 1470 may be horizontally divided based on the center of the preview image 1460 or diagonally as shown in (b) of FIG. 14B. Can be partitioned into two regions 1452 and 1454.

Meanwhile, when the second area 1470 is divided into two areas 1452 and 1454, the controller 180 may designate one of the two areas 1452 and 1454 as the start area and the other as the start area. It can be determined as an end region. In this case, as shown in FIG. 14A, the start area and the end area may mean an area where the controller 180 can start continuous shooting based on whether a preset image of the subject is detected, and the end area is a preset object. Based on whether or not to detect the image may mean an area capable of ending the continuous shooting. Herein, the controller 180 may set an area in which the image of the subject is first detected among the partitioned areas 1452 and 1454 as a start area, and set the other area as the end area.

Meanwhile, when the start area and the end area are set for the preview image as described above, the controller 180 may perform the high speed continuous shooting based on whether the image of the subject is detected in the start area and the end area.

FIG. 15 illustrates an example in which a start and end of a high speed continuous shooting function according to an embodiment of the present invention are controlled according to a position on a preview area in which an image of a subject is detected. In the following description, it is assumed that the first camera 121b and the second camera 121c of the mobile terminal 100 are arranged adjacent to each other in the horizontal direction for convenience of description. The first image sensor of the first camera 121b and the second camera 121c will be described on the assumption that the second image sensor has the same angle of view.

FIG. 15 assumes an example in which the horse-shaped mobile is preset as a subject when the horse-shaped mobile makes a rotational movement in a counterclockwise direction about a central axis. The subject may be set based on a user's touch input.

First, referring to the first drawing of FIG. 15, as shown in FIG. 14A (a), the controller 180 is an area commonly sensed by the first image sensor and the second image sensor 1550 (hereinafter referred to as a first area). ), And a preview image 1570 in which all of the areas 1552 (hereinafter, referred to as the 2-1 region) sensed only in the first image sensor, and the areas 1554 (hereinafter, referred to as region 2-2) which are sensed only in the second image sensor are displayed. Can be displayed.

In this state, as shown in the first drawing of FIG. 15, when the image of the preset object 1500 is detected in the second-first area 1552, which is sensed only by the first image sensor, the controller 180 is detected. May set the second-first area 1552 as a start area and start high-speed continuous shooting according to an embodiment of the present invention. In addition, when the second-first area 1552 is set as the start area, the controller 180 may set the second-second area 1554 as the end area.

Meanwhile, when the high speed continuous shooting starts, the controller 180 can sense the subject 1500. In addition, when the subject 1500 is detected in the first area 1550 that is commonly sensed by the first image sensor and the second image sensor, the second 1500 is detected according to a result of sensing the subject 1500. The focus of the image sensor corresponding to the area 1552, that is, the first image sensor may be adjusted. When the focus of the first image sensor is adjusted, the preview image sensed by the first image sensor may be captured. Accordingly, as shown in the second drawing of FIG. 15, an image in which the focal point is focused on the subject 1500 moving to the first position 1510 may be captured by the first image sensor.

Meanwhile, the controller 180 may capture only an area corresponding to an area that the first image sensor and the second image sensor sense in common in the preview image. That is, an image corresponding to the first area 1550 may be captured by the first image sensor.

Meanwhile, while the focus of the first image sensor is adjusted, the controller 180 may perform sensing of the subject 1500 again. In response to the focus adjustment of the first image sensor being completed, while the image of the first region 1550 is captured by the first image sensor, the controller 180 makes sure that the second image sensor returns to the subject 1500. According to the sensing result, the second camera 121c may be controlled to adjust the focus. In addition, the controller 180 may sense the subject 1500 again while the focus of the second image sensor is adjusted.

On the other hand, by repeating the above-described process, the controller 180 allows the other image sensor to adjust the focus based on a result of sensing the subject while one image sensor is capturing the image, thereby performing the above-described high speed continuous shooting. Can be performed. Therefore, when the moving object 1500 is in the second position 1520, the second object 1500 is moved to the third position 1530 through the second image sensor and as shown in the third drawing of FIG. 15. When present, the image corresponding to the first area 1550 may be captured through the first image sensor.

On the other hand, as shown in the fourth drawing of FIG. 15, when the subject 1500 is moved and detected in the currently set end region, that is, the second-2 region 1554, the controller 180 is currently performing High speed continuous shooting can be terminated. Here, when all regions of the detected object 1500 image are detected in the second-second area 1554, the controller 180 may determine that the subject 1500 is detected in the end area.

When the subject 1500 is detected again in the start area (the 2-1 area 1552), the controller 180 may again perform high speed continuous shooting. The high speed continuous shooting may be performed by repeating the processes from the first drawing to the third drawing of FIG. 15.

Meanwhile, as described above, when the subject 1500 performs the rotational movement, the image of the subject 1500 is periodically detected in the start area (2-1 region 1552) and in the first region 1550. After the detection, the detection may be detected again in the end region (second-2 region 1554). Accordingly, the mobile terminal 100 according to an exemplary embodiment of the present invention may automatically start high speed continuous shooting whenever an image of the subject 1500 is detected in the start area, and further out of the first area 1550. If abnormal high speed continuous shooting is impossible, the high speed continuous shooting can be automatically terminated.

Meanwhile, in the above description, the case where the high speed continuous shooting is performed when the image of the subject is detected in the areas commonly sensed by the first and second image sensors has been described. However, when the subject moves, the image angle of the subject may be sensed only in one of the first and second image sensors, as shown in FIG. 15, by moving out of an angle of view of one of the image sensors. .

In this case, in the description of FIG. 15, an example of automatically terminating the above-described high speed continuous shooting has been described. Alternatively, any image sensor capable of sensing the subject may continue to perform image capturing.

Accordingly, as shown in the first drawing of FIG. 15, when the subject 1500 is sensed only by the first image sensor, that is, the image of the subject 1500 is detected, and a part or the whole of the image of the subject 1500 is detected. The controller 180 uses the first image sensor until an area is detected in the area where the subject 1500 is an overlapping angle of view of the first image sensor with that of the second image sensor, that is, the first area. You can also take pictures.

In addition, as shown in the fourth drawing of FIG. 15, when the image of the subject 1500 is detected in an area that can be sensed only by the second image sensor according to the movement of the subject 1500, the controller 180 performs the above-described high speed continuous shooting. Of course, the image may be taken using the second image sensor.

In addition, the controller 180 may display notification information so that a user can identify an area where the subject 1500 is sensed as the subject 1500 moves. That is, the controller 180 detects an image of the subject 1500 when the image of the subject 1500 is detected in a region within the preview image which can be sensed only by the first image sensor as shown in the first drawing of FIG. 15. One area in the preview image may be displayed to be distinguished from other areas. That is, the periphery of the 2-1 region 1552 may be displayed as a preset graphic object (eg, a guide line) or the brightness or saturation of the 2-1 region 1552 may be displayed differently from other regions. Alternatively, text information or sound information for notifying that the subject 1500 is sensed may be output in an area that can be sensed only by the first image sensor.

On the other hand, as shown in the fourth drawing of FIG. 15, when the image of the subject 1500 is detected in one region within the preview image that can be sensed only by the second image sensor, the controller 180 replaces the second region 1554 with another region. The text information or sound information may be output to display them differently from each other or to inform them of them.

Meanwhile, in the above description, an example of displaying an area in which the subject 1500 is detected by dividing an area in the preview image has been described. However, in contrast, an area corresponding to an angle of view of an image sensor sensing the current subject 1500 is distinguished. Of course, it can be displayed. In this case, when the image of the subject 1500 is located in the second-first area 1552, the second-first area 1552 and the first area 1550 may be displayed separately. When the image of the subject 1500 is located in the second-second area 1554, the first area 1550 and the second-second area 1554 may be displayed separately.

On the other hand, the controller 180 may display the notification information for notifying when the subject is out of view of the specific image sensor. For example, as shown in FIG. 14B, when the first image sensor and the second image sensor have different angles of view, notification information for notifying a change in the image sensor capable of sensing a subject is output. can do. That is, when the image of the subject is moved from a region where both of the first and second image sensors can be sensed to a region that can be sensed by only one image sensor, the controller 180 moves the subject to the first and second images. The sensor may display notification information indicating that the sensor is out of viewable angles.

On the other hand, while sensing the subject as described above, the controller 180 may also detect the position change and movement of the main body of the mobile terminal (100). In this case, the controller 180 may further include information related to the detected position change and movement of the main body of the mobile terminal 100 in the notification information. As such, information related to the movement of the mobile terminal 100 is further displayed on the notification information, so that the controller 180 may correct the posture of the mobile terminal 100 when the subject is out of view of the specific image sensor. can do.

The present invention described above can be embodied as computer readable codes on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. This also includes those implemented in the form of carrier waves (eg, transmission over the Internet). In addition, the computer may include the controller 180 of the terminal. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

Claims (20)

1. First and second cameras; And,

   A control unit for controlling the first and second cameras to perform image shooting through at least one of the first and second cameras in response to the image shooting request;

   The control unit,

   When the image capturing request corresponds to a continuous capturing request, the first and second cameras are controlled so that the first and second cameras alternately perform image capturing on a specific subject to be focused.

   Any one of the first and second cameras adjusts a focus on the specific subject while the other is capturing an image.
2. The method of claim 1, wherein the control unit,

   Sensing whether the specific subject is included in a first angle of view of the first camera and a second angle of view of the second camera,

   And determining whether to perform the continuous shooting by capturing an image alternately between the first and second cameras according to a sensing result.
3. The method of claim 2,

   The touch screen may further include a touch screen to output a preview image received through at least one of the first and second cameras.

   The preview image includes graphic objects corresponding to at least one subject,

   The control unit,

   When a specific graphic object among the graphic objects included in the preview image is touched through the touch screen, the touched graphic object is set as a specific subject to be the focus,

   And monitoring whether the specific subject is included in both the first and second angles of view.
4. The method of claim 2, wherein the control unit,

   When the specific subject is included in both the first angle of view and the second angle of view, the first and second cameras to control the first and second cameras so as to alternately take a picture Mobile terminal.
5. The method of claim 4, wherein the control unit,

   When the specific subject is out of the field of view of any one of the first and second angle of view, the first and second cameras alternately stop the continuous shooting to take an image,

   And controlling the first and second cameras to capture an image through any one of the first and second cameras corresponding to an angle of view including the specific subject.
6. The method of claim 5, wherein the control unit,

   When the specific subject is out of one of the first and second angles of view, output notification information for notifying the deviation of the angle of view of the specific subject,

   The notification information,

   The mobile terminal further comprises information related to the movement of the terminal body having the first and second cameras.
7. The method of claim 2, wherein the control unit,

   When the specific subject is included in any one of the first and second angles of view, controlling the first and second cameras to capture an image through a camera having the angle of view including the specific subject. Mobile terminal characterized in that.
8. The method of claim 1,

   The touch screen may further include a touch screen to output a preview image received through at least one of the first and second cameras.

   The control unit,

   And a guide image for displaying an area in which the angles of view of the first and second cameras overlap each other in the preview image on the touch screen.
9. The method of claim 7, wherein the control unit,

   Generating a preview image by synthesizing a first image sensed according to a first field of view of the first camera and a second image sensed according to a second field of view of the second camera,

   The preview image,

   A mobile terminal comprising an image of an area sensed by both the first camera and a second camera and an image of an area sensed by only one of the first and second cameras according to the overlap of the angles of view.
10. The method of claim 9, wherein the control unit,

    The preview image is synthesized based on the image sensed by both the first camera and the second camera.

    When the first angle of view of the first camera and the second angle of view of the second camera are different from each other, a narrower angle of view in the image sensed by the camera having the wider angle of view among the images sensed by the first camera and the second camera And inserting an image sensed by a camera having a camera to output a preview image synthesized in a PIP (Picture In Picture) format.
11. The method of claim 9, wherein the control unit,

    The preview image is synthesized based on the image sensed by both the first camera and the second camera.

    When the first angle of view of the first camera and the second angle of view of the second camera are the same, an image sensed only by the first camera and the second camera around an image sensed by both the first camera and the second camera The mobile terminal, characterized in that for outputting a preview image synthesized with the image sensed only.
12. The method of claim 2, wherein the control unit,

    When the specific subject is included only in the image sensed by the first camera among the images sensed by the first camera and the second camera, the first camera and the second camera are controlled to start the continuous shooting. and,

    When the specific subject is included only in the image sensed by the second camera among the images sensed by the first camera and the second camera, the first camera and the second camera are controlled to stop the continuous shooting. Mobile terminal, characterized in that.
13. The method of claim 12, wherein the first camera,

    And the image including the specific subject is a camera sensed before other cameras.
14. The method of claim 1,

    Further comprising a sensing unit for detecting the movement of the specific subject,

    The control unit,

    While the one camera adjusts the focus and the other camera captures an image, the sensing unit senses a movement state of the specific subject,

    When the focus adjustment of one of the Kakera is completed, while the one camera is taking an image, the other camera controls to adjust the focus based on the detected movement state of the subject Mobile terminal.
15. The method of claim 1, wherein the control unit,

    When the image capturing corresponds to continuous capturing of a plurality of preset focus regions, the first and second cameras alternately perform image capturing on each of the plurality of focus regions to be focused. And a second camera,

    Any one of the first and second cameras adjusts the focus of the other focus area while the other is taking an image.
16. The method of claim 15,

    The touch screen may further include a touch screen configured to output a preview image generated through at least one of the images received through at least one of the first and second cameras.

    The control unit,

    And when a plurality of touch inputs are applied to the preview image, setting the plurality of focus areas having a preset order according to the order in which the touch inputs are applied.
17. The method of claim 15,

    Further comprising a sensing unit for detecting the movement of the specific subject,

    The control unit,

    And setting the plurality of focus areas according to a result of predicting and predicting the movement of the specific subject based on a result of sensing the movement of the specific subject or an input applied from a user.
18. The method of claim 1, wherein the control unit,

    When an image is taken according to the continuous shooting, images that are out of focus among the captured images are detected.

    And further performing the continuous shooting as many times as the number of detected images.
19. Activating the first camera and the second camera;

    Generating a preview image based on the images sensed by each of the activated cameras, and outputting the generated preview image;

    Selecting a plurality of focus areas or at least one subject based on a user input applied to the preview image; And,

    And performing continuous shooting alternately through the first camera and the second camera according to a preset order of the plurality of focus areas and a result of detecting the specific subject.

    Performing the continuous shooting,

    The method of controlling a mobile terminal according to claim 1, wherein the other camera adjusts the focus according to a result of detecting the next focus area or the specific subject while the other camera photographs the image.
20. The method of claim 19, wherein the performing of the continuous shooting,

    Determining whether an image of the specific subject is detected in an area sensed only by the first camera;

    Starting the continuous shooting according to the determination result;

    Determining whether an image of the specific subject is detected in an area sensed only by the second camera; And,

    And stopping the continuous shooting according to the determination result.

Images