KR20150025268A - Electronic Device And Method Of Controlling The Same - Google Patents

Abstract

The present invention relates to an electronic device and a method of controlling the same The present invention provides the electronic device and the method of controlling the same, which detects a direction of a line of sight of a user and automatically control a focal point according to the direction of the line of sight of the user, then an image is photographed based on the controlled focal point such that the focal point can be conveniently controlled even when a touch input cannot be received;.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an electronic device,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic apparatus and a control method of the electronic apparatus.

[0002] In recent years, in the field of various electronic devices including mobile terminals, a great variety of functions can be provided to users, thanks to the breakthrough of software and hardware.

Accordingly, there is a growing need for the development and dissemination of various user interfaces that allow a user to easily and efficiently control electronic devices that provide various complex functions.

An object of the present invention is to provide an electronic device and a control method of an electronic device that provide a user interface that enables a user to easily and efficiently control an electronic device.

It is another object of the present invention to provide an electronic apparatus and a control method of an electronic apparatus which enable easy automatic focus control even in a situation where focus control is difficult through touch input.

The technical objects to be achieved by the present invention are not limited to the technical problems mentioned above. In addition, other technical objects to be achieved by the present invention will become apparent to those skilled in the art from the following description.

An electronic apparatus according to one aspect of the present invention includes: a first camera; A second camera for photographing a direction different from the first camera; A display unit for displaying a preview image obtained through the first camera; And a second camera for detecting a direction of a user's gaze through the first camera, setting an auto-focus on a point indicated by the gaze direction among the preview images, And a control unit for photographing the image.

The gaze direction of the user may be determined based on the user's facial movement obtained through the second camera.

The control unit may acquire the movement of the user's eye through the second camera and automatically control the image to be photographed when recognizing a predetermined type of eye flicker after the auto focus is set.

The controller may set the first point to auto focus when the user's gaze direction to the first point of the preview image is fixed for a predetermined period of time.

Wherein the control unit sets the first point to auto-focus, and when the direction of the user's gaze is changed so that the gaze direction of the user with respect to the second point of the preview image is fixed for the predetermined time, You can set the point to additional autofocus.

The control unit may perform continuous shooting with the first point and the second point as a focal point.

The control unit may display a thumbnail image of a plurality of images obtained by the continuous shooting on the display unit.

The control unit may obtain the preview image through the second camera, detect the direction of the user's gaze from the preview image, and set the autofocus based on the gaze direction.

The controller may enlarge the size of the preview image around the first area when the user's gaze direction is fixed for a predetermined time or longer with respect to the first area of the preview image.

The control unit may reduce the size of the preview image around the second area when the user's gaze direction is fixed for a predetermined period of time or longer with respect to the second area of the preview image.

According to another aspect of the present invention, there is provided a method of controlling an electronic device, comprising: displaying a preview image obtained through a first camera on a display unit; Sensing a user's gaze direction through a second camera that captures a direction different from the first camera as the capturing is started through the first camera; Setting an auto-focus on a point indicated by the gaze direction among the preview images; And photographing an image based on the set focus.

The electronic device and the control method of the electronic device according to the present invention have the following effects.

According to the present invention, it is possible to provide a user with a user interface that enables easy and efficient control of an electronic device.

In addition, according to the present invention, it is possible to easily take a picture based on a focus set automatically by detecting a direction of a user's eyes in a situation where focus control is difficult through touch input.

Further, according to the present invention, it is possible to set multiple foci by sensing the direction of the user's gaze to be changed, and to perform continuous shooting more easily based on the set focal point.

1 is a block diagram of an electronic apparatus according to an embodiment of the present invention.
2A is a front perspective view of a mobile terminal according to an embodiment of the present invention.
2B is a rear perspective view of a mobile terminal according to an embodiment of the present invention.
3 is a flowchart of a method of controlling an electronic device according to an embodiment of the present invention.
FIG. 4 is a flowchart for explaining an example for sensing the direction of the user's gaze according to the embodiment shown in FIG.
Figs. 5 to 6 are diagrams showing a method of obtaining a user's gaze direction based on the flowchart shown in Fig. 4. Fig.
FIGS. 7 to 8 are diagrams illustrating an example of setting auto-focus based on the viewing direction according to an embodiment of the present invention.
9 to 10 are views illustrating an example of setting multiple focuses according to the direction of a user's eyes according to an embodiment of the present invention.
11 is an example showing an image photographed according to one embodiment of the present invention on a preview screen.
12 is an example of self-photographing according to a control method of an electronic device according to an embodiment of the present invention.
13A to 14B are diagrams illustrating an example of editing a preview image according to a direction of a user's gaze according to another embodiment of the present invention.

The foregoing objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Like reference numerals designate like elements throughout the specification. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

Hereinafter, an electronic apparatus related to the present invention will be described in more detail with reference to the drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

Examples of the electronic devices described herein include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs) And the like.

1 is a block diagram of an electronic apparatus according to an embodiment of the present invention.

The electronic device 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory unit 160, An interface unit 170, a control unit 180, a power supply unit 190, and the like. The components shown in FIG. 1 are not essential, and an electronic device having components having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the electronic device 100 and the wireless communication system or between the electronic device 100 and the network in which the electronic device 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems that provide broadcast signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory unit 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 can be embedded in the electronic device 100 or externally. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

The position information module 115 is a module for confirming or obtaining the position of the electronic device. The location information module 115 may obtain location information using a global navigation satellite system (GNSS). Here, the Global Positioning System (GNSS) is a term used to describe the radionavigation satellite systems in which certain types of radionavigation receivers orbiting the Earth are transmitting reference signals that can determine their position near the surface or ground surface . The Global Positioning System (GNSS) includes Global Positioning System (GPS) in the United States, Galileo in Europe, GLONASS (Global Orbiting Navigational Satelite System) in Russia, COMPASS in China and Japan QZSS (Quasi-Zenith Satellite System), which is operated by the Ministry of Land, Infrastructure and Transport.

As a representative example of the GNSS, the location information module 115 may be a Global Position System (GPS) module. The GPS module calculates information on a distance (distance) from three or more satellites to a point (object), information on a time when the distance information is measured, and then applies a trigonometric method to the calculated distance information, Dimensional position information according to latitude, longitude, and altitude with respect to a point (object) in the three-dimensional space. Further, a method of calculating position and time information using three satellites and correcting the error of the calculated position and time information using another satellite is also used. The GPS module continuously calculates the current position in real time, and calculates velocity information using the current position.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory unit 160 or may be transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the electronic device 100, such as the open / close state of the electronic device 100, the position of the electronic device 100, the presence or absence of user contact, the orientation of the electronic device, And generates a sensing signal for controlling the operation of the electronic device 100. For example, when the electronic device 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. In addition, it may be responsible for a sensing function related to whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor 141.

The output unit 150 may include a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154, for example, for generating output related to visual, auditory, have.

The display unit 151 displays and outputs information processed by the electronic device 100. [ For example, when the electronic device is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with the call is displayed. When the electronic device 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display).

Some of these displays may be transparent or light transmissive so that they can be seen through. This may be referred to as a transparent display. A typical example of the transparent display is a transparent LCD or the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the electronic device 100. [ For example, in the electronic device 100, a plurality of display portions may be spaced apart from one another or may be disposed integrally with each other, or may be disposed on different surfaces.

The display unit 151 is connected to a display unit 151 and a sensor for sensing a touch operation (hereinafter, referred to as 'touch sensor') having a mutual layer structure It can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

Referring to FIG. 1, a proximity sensor 141 may be disposed in an inner area of an electronic device to be wrapped by the touch screen or in the vicinity of the touch screen. The proximity sensor 141 refers to a sensor that detects the presence of an object approaching a predetermined detection surface or an object existing in the vicinity of the detection surface without mechanical contact using an electromagnetic force or an infrared ray. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch & The act of actually touching the pointer on the screen is called "contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory unit 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 outputs sound signals associated with functions (e.g., call signal reception tones, message reception tones, etc.) performed in the electronic device 100. The audio output module 152 may include a receiver, a speaker, a buzzer, and the like. Also, the sound output module 152 may output sound through the earphone jack 116. The user can connect the earphone to the earphone jack 116 to hear the sound output.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the electronic device 100. Examples of events that occur in electronic devices include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or the audio signal may also be output through the display unit 151 or the audio output module 152.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may be configured to perform various functions such as an effect of stimulation by a pin arrangement vertically moving with respect to a contact skin surface, an effect of stimulation by air spraying force or suction force through a jet opening or a suction opening, A variety of tactile effects such as an effect of stimulation through contact of an electrode, an effect of stimulation by an electrostatic force, and an effect of reproducing a cold sensation using a heat absorbing or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to feel the tactile effect through the muscles of the user's finger or arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the portable terminal 100.

The memory unit 160 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory unit 160 may store data related to vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory unit 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory) (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The electronic device 100 may operate in association with a web storage that performs a storage function of the memory unit 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the electronic device 100. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to the respective components in the electronic device 100 or to transmit data in the electronic device 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the usage right of the electronic device 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

When the electronic device 100 is connected to an external cradle, the interface unit may be a path through which electric power from the cradle is supplied to the electronic device 100 or various command signals input from the cradle by the user It can be a passage to the device. Various command signals input from the cradle or the power source may be operated as a signal for recognizing that the electronic device is correctly mounted on the cradle.

The control unit 180 typically controls the overall operation of the electronic device. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, And may be implemented by the control unit 180.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. Also, the software codes may be stored in the memory unit 160 and executed by the control unit 180. [

2A is a perspective view of an example of the electronic device 100 viewed from the front.

The electronic device 100 includes a bar-shaped terminal body. The present invention is not limited thereto, but can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are movably coupled to each other.

The body includes a case (casing, housing, cover, etc.) constituting an image. The case may be divided into a front case 101 and a rear case 102. A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be manufactured by a method of injecting a synthetic resin, or may be made of a metal material, for example, a metal material such as stainless steel (STS) or titanium (Ti).

The front case 101 is provided with a display module 151, an acoustic output unit 152, a camera 121, a plurality of operation units 131 and 132 of a user input unit, a microphone 122, an interface 170, have.

The display module 151 occupies most of the main surface of the front case 101. The audio output unit 152 and the camera 121 are disposed in an area adjacent to one end of the both ends of the display unit 151 and the first operation unit 131 and the microphone 122 are disposed in an area adjacent to the other end . The second operation unit 132 and the interface 170 are disposed on the side surfaces of the front case 101 and the rear case 102. [

The camera 121 may correspond to a second camera to illustrate embodiments of the present invention. That is, the second camera 121 may be a front camera positioned toward the user when the user is staring at the display module 151. The face of the user looking at the display module 151 through the second camera 121 can be photographed.

The user input unit 130 is operated to receive a command for controlling the operation of the portable terminal 100 and may include a plurality of operation units 131 and 132. The operation units 131 and 132 can be employed in any manner as long as the user is in a tactile manner while making the user feel the change of the tactile sense. For example, the operation units 131 and 132 may include a keypad, a dome switch, a touch pad, a jog wheel, a jog switch, a trackball, a joystick, and the like.

The contents inputted by the operation units 131 and 132 can be variously set. For example, the first operation unit 131 receives commands such as start, end, scroll, and the like, and the second operation unit 132 controls the size of the sound output from the sound output unit 152 or the size of the sound output from the display unit 151 To the touch recognition mode of the touch screen.

2B is a rear perspective view of the electronic device 100 shown in FIG. 2A.

Referring to FIG. 2B, a camera 121 'may be additionally mounted on the rear case 102. The camera 121 'may have a photographing direction substantially opposite to the front camera 121 (see FIG. 2A), and may be a camera having different pixels from the camera 121.

The camera 121 'is a first camera 121' for implementing a control method of an electronic device according to an embodiment of the present invention, and is a rear camera. It is possible to photograph an image corresponding to the gaze direction of the user through the rear camera.

For example, the second camera 121 has low pixels so that it is easy to capture a face of a user in case of a video call or the like and send the face to a counterpart. The first camera 121 ' It is preferable to have a high pixel. The first camera 121 'and the second camera 121 may be installed in the terminal body such that they can be rotated or pop-up.

A flash 123 and a mirror 124 may be additionally disposed adjacent to the first camera 121 '. The flash 123 illuminates the subject toward the subject when the subject is photographed by the first camera 121 '. The mirror 124 enables the user to illuminate the user's own face or the like when the user intends to photograph himself / herself (self-photographing) using the first camera 121 '.

The rear surface of the rear case 102 may further have an acoustic output section 152 '. The sound output unit 152 'may implement a stereo function together with the front sound output unit 152 (see FIG. 2A), and may be used for implementing a speakerphone mode at the time of a call.

An antenna for receiving a broadcast signal may be additionally disposed on the side of the electronic device body in addition to an antenna for a call or the like. An antenna constituting a part of the broadcast receiving module 111 of FIG. 1 may be installed to be able to be drawn out from the mobile terminal body. Meanwhile, the side of the mobile terminal body may include an operation unit for performing a function related to an event occurring in the electronic device 100 when the electronic device 100 is received in the case and is in the lock mode.

The electronic apparatus body is equipped with a power supply unit 190 for supplying power to the electronic apparatus 100. The power supply unit 190 may be built in the terminal body or may be detachable from the outside of the terminal body.

The rear case 102 may further include a touch pad 135 for sensing a touch. The touch pad 135 may also be of a light transmission type like the display unit 151. [ If the display unit 151 is configured to output time information on both sides, the time information can be recognized through the touch pad 135 as well. The information output on both sides may be all controlled by the touch pad 135. [ Alternatively, a display may be additionally mounted on the touch pad 135, and a touch screen may be disposed on the rear case 102.

The touch pad 135 operates in correlation with the display portion 151 of the front case 101. The touch pad 135 may be disposed parallel to the rear of the display unit 151. The touch pad 135 may have a size equal to or smaller than that of the display unit 151.

Hereinafter, embodiments of the present invention will be described. In the present invention, the display unit 151 is assumed to be the touch screen 151 for convenience of explanation. As described above, the touch screen 151 can perform both the information display function and the information input function. However, it should be clearly understood that the present invention is not limited thereto. Also, the touch mentioned in this document can include both touch and proximity touch.

FIG. 3 is a flowchart of a method of controlling an electronic device according to an embodiment of the present invention, and is a flowchart for explaining an example for detecting a user's gaze direction according to the embodiment shown in FIG.

A control method of an electronic device according to an embodiment of the present invention can be implemented in the electronic device 100 described with reference to Figs. 1 and 2A to 2B. Hereinafter, a method of controlling an electronic device according to an embodiment of the present invention and an operation of the electronic device 100 for implementing the same will be described in detail with reference to the drawings.

Referring to FIG. 3, the control unit 180 of the electronic device 100 may display a preview image on the display unit 151 (S100).

The control unit 180 drives the second camera 121 'in FIG. 2B to display a preview image obtained through the second camera on the display unit 151. The second camera 121 'is a rear camera.

When the second camera 121 'is driven, a user interface for controlling image taking including a preview image can be displayed on the display unit 151. The user interface for the image photographing may include a still image or a moving image selection item, a photographing button, various menus for photographing options, and the like.

On the other hand, when a camera application is generally executed to drive a camera module, a focus guide for focus adjustment is displayed on a preview image. The position of the focus guide FG can be changed by the user's touch input. When the focus guide FG is set at a specific position, the focus guide FG is set to a focus position, have. Accordingly, the user can adjust the focus while moving the focus guide in the preview image. Meanwhile, the display unit 151 may be a touch screen, and the focus can be adjusted on the preview image by a user's touch input.

The control unit 180 may sense the direction of the user's gaze after the shooting starts (S110).

The controller 180 displays a preview image and a user interface for capturing an image on the touch screen 151 when the camera is driven and receives an input to a capture button (not shown) of the user interface. (121 of FIG. 2A) to detect the first camera (FIG.

That is, according to an embodiment of the present invention, the controller 180 acquires a preview image of the forward image through the first camera 121 '(FIG. 2B) The direction of the user's gaze can be detected.

The gaze direction may indicate a direction and / or a point that the user of the electronic device 100 is watching. For example, it means that it is possible to include at what point the user is gazing at the display unit 151, whether or not the user is gazing at the point for a certain period of time, . A specific example of sensing the user's gaze direction will be described in more detail with reference to FIG.

The control unit 180 may control the focus according to a point indicated by the gaze direction in the preview image (S120).

The control unit 180 recognizes the point indicated by the user's gaze direction on the preview image and automatically recognizes that the gaze direction of the user indicates the point for a predetermined time, (S130).

The control unit 180 can photograph the image based on the set focus (S140).

Here, if at least one focus is set based on the gaze direction of the user, the controller 180 may photograph an image based on the set focus based on a predetermined input.

The predetermined input may be an input to the image taking button in a state in which the focus is set.

In addition, the predetermined input may be a specific gesture input of the user in the state in which the focus is set. The specific gesture input may be a gesture input to the user's eyes obtained through the second camera (121 of FIG. 2A). For example, when the focus is set and the blinking of the user's eyes is repeated a predetermined number of times, or when it is recognized that the blinking of the eyes has progressed for a predetermined time, the controller 180 can photograph the image based on the set focus.

Hereinafter, a method of recognizing a user's gaze direction through an image of the user's eyes acquired through a camera will be described in detail with reference to FIGS. 4 to 6. FIG.

As shown in FIG. 4, the step of sensing the user's gaze direction (S120 in FIG. 3) in the electronic device 100 according to the embodiment of the present invention includes a step S121 of acquiring an image of the eye can do.

An image of the eye can be obtained through the second camera 121 (Fig. 2A) of the electronic device 100. Fig. For example, it is possible to obtain an image of the user's eyes operating the electronic device 100 through a camera (121 in Fig. 2A) located at one point in the front surface of the electronic device 100. [

A user operating the electronic device 100 typically gazes at the display unit 151 located on the front surface of the electronic device 100. [ Therefore, the face image of the user can be relatively easily photographed through the camera on the front face (121 in FIG. 2A). The control unit 180 can extract the image of the eye part from the photographed face image.

The control unit 180 may calculate the center point of the pupil of the image of the eyes of the user (S123).

The pupil is a central portion of the eye and a rounded portion surrounded by an iris. The pupil is darker than other parts of the eye, and is usually black for Asians.

The user's gaze may be closely related to the pupil. For example, the point of interest with which the user is interested can be substantially coincident with the direction of the pupil center point.

The center point of the pupil can be extracted based on the relatively dark shape and / or color feature located in the center of the acquired eye image. For example, the center point of the image of the pupil in the round shape can be determined as the center point of the pupil.

Then, the control unit 180 can calculate the gaze direction in consideration of the movement of the user's face (S125).

Facial movements can be an element for calculating the user's gaze direction along with the pupil center point. For example, even when the pupil is facing the front without moving the pupil, the gaze direction may be changed if the face is moved to the left or right.

The movement of the face can be detected through the face image of the user photographed through the camera 121. The control unit 180 can extract the direction of the face based on the characteristic elements such as forehead, eye, nose, mouth, ear, and the like in the face image.

Based on the motion of the face and the calculated center point of the pupil, the control unit 180 can calculate the gaze direction. For example, this means that the user can determine which point of the current display unit 151 is gazing. For convenience of understanding, the following description will be made in a unified manner in the direction of the eyes, but the gaze direction may include the direction of the face due to the movement of the user's face.

Figs. 5 to 6 are diagrams showing a method of obtaining a user's gaze direction based on the flowchart shown in Fig. 4. Fig.

5, the control unit 180 of the electronic device 100 according to the embodiment of the present invention acquires the direction of the user's gaze from the image photographed through the second camera (121 in FIG. 2B) .

The viewpoint of the gaze direction may be the center point EP of the eyeball. That is, the eye vectors GD1 and GD2 starting from the center point EP of the eyeball and passing through the center point P of the pupil can be in the eye direction.

When the user moves the line of sight, the center point P of the pupil can also be moved. In this case, if there is no movement of the user's face, the center point EP of the eyeball is fixed and only the center point P of the pupil can move. When the center point P of the pupil moves, the gaze vectors GD1 and GD2 can move from the first gaze vector GD1 to the second gaze vector GD2.

If there is movement of the user's face, the center point EP of the eyeball and the center point P of the pupil can move simultaneously. Therefore, the control unit 180 can obtain the gaze direction based on the movement of the user's face and / or the center point of the pupil obtained through the camera 121. [

6, the control unit 180 of the electronic device 100 according to an exemplary embodiment of the present invention determines whether or not a user is watching a certain portion of the display 151 based on the obtained gaze direction Can be determined.

As shown in Fig. 6 (a), the location that the user watches along with the passage of time can be changed. For example, if you are looking at point A until t1, you can move to point B between t1 and t2, you are looking at point B from t2 to t3, and you can move again at t3 it means.

As shown in Fig. 6 (b), the change in the position that the user is watching can be matched to the specific coordinates on the display 151. [ That is, the control unit 180 can match the obtained gaze direction of the user to the coordinates on the display 151. [

According to an embodiment of the present invention, at least one of the coordinates in which the gaze direction stays is set as focal coordinates. For example, the control unit 180 may set the focus coordinates for the coordinates to be watched for a predetermined time among a plurality of positions that the user watches. The controller 180 may store a plurality of sets of focus coordinates together with the set focus coordinates.

For example, in (b) of FIG. 6, a point where the user's gaze has stayed for a predetermined time with respect to the point A, point C, and point F is set as multi-focal point for each of the points A, have.

The control unit 180 photographs the image at the first point in time with the point A as the focal point and then captures the image at the second point in time with the point C as the focal point, The image can be photographed at the third point with the F point as a focal point. The first point of view, the second point of view, and the third point of view may be continuous times. That is, it is possible to take continuous photographs with the focus point A, the point C, and the point F respectively.

FIGS. 7 to 8 are diagrams illustrating an example of setting auto-focus based on the viewing direction according to an embodiment of the present invention.

Referring to FIG. 7, the controller 180 may display the preview image P obtained through the first camera 121 (FIG. 2A) on the touch screen 151. FIG. As the image capturing is requested, the controller 180 detects the direction of the user's gaze through the second camera 121 'in FIG. 2B.

The controller 180 sets the first area R1 as a focus area and the focus area is set as a focus area when the user's eyes are recognized to stay in the first area R1 of the preview image P for a predetermined time An auto focus guide (AFG) for distinguishing the setting of the first area R1 can be displayed in the first area R1.

Referring to FIG. 8, when the user's gaze of the user stays in the second area R2 for a predetermined time after the user's gaze direction moves from the first area R1 to the second area R2 in FIG. 7 , The controller 180 can automatically set the second area R2 as a focus area and take an image with the second area R2 as a focus.

9 to 10 are views illustrating an example of setting multiple focuses according to the direction of the user's eyes according to an embodiment of the present invention.

Referring to FIG. 9, the controller 180 sequentially displays the first region R1, the second region R2, the third region R3, and the fourth region R4 sequentially and automatically So that the focus can be set. The control unit 180 can display the autofocus guides AFG1, AFG2, AFG3, and AFG4 for each area set as the focus.

On the other hand, the control unit 180 may automatically perform continuous shooting based on the focus of the four points, and four images with different focuses may be generated.

Referring to FIG. 10, as the direction of the user's gaze is shifted, sequential and automatically set multiple foci can be set. Then, it is possible to sequentially receive scrolling input for each of the autofocus guides AFG1, AFG2, AFG3, and AFG4. When the scroll input is received for each of the multiple focuses, the controller 180 outputs a fourth image corresponding to the fourth focus in the image corresponding to the first focus as "Fade Out / Fade In "Image correction can be performed to create a transition effect.

That is, according to an embodiment of the present invention, when a scroll input is received for a plurality of regions corresponding to a set multi-focal point in which a multi-focal point is set according to a direction of a user's gaze, A correction operation can be performed after the pre-shooting.

Meanwhile, according to one embodiment of the present invention, when an image is photographed according to a focus set based on a direction of a user's gaze, the photographed image can be automatically stored in a memory. The control unit 180 may store the photographed image in the JPEG format.

11 is an example showing an image photographed according to one embodiment of the present invention on a preview screen.

Referring to FIG. 11, the controller 180 may perform continuous shooting based on a plurality of focal points set based on the user's gaze direction. The control unit 180 may display a plurality of thumbnail images P1, P2, P3, and P4 corresponding to the plurality of continuously captured images on the touch screen 151. [ The control unit 180 may display the thumbnail images P1, P2, P3, and P4 in one area of the preview image P as the continuous shooting ends.

The user can select and store a desired one of the plurality of thumbnail images P1, P2, P3, and P4. That is, according to one embodiment of the present invention, it is possible to more easily select a plurality of focuses desired by the user based on the direction of the user's eyes, even in an environment where focus control through the touch input is impossible, A desired one of the photographed images can be selected and stored.

In the above embodiment, the user's gaze direction is acquired through the second camera (121 in Fig. 2A) provided on the front surface of the electronic device 100. [ However, as shown in FIG. 4, the user's gaze direction obtained through the second camera is calculated in consideration of the eye image, the motion of the pupil center point, etc., and the calculated result is confirmed through the AFG I could.

However, referring to FIG. 12, when the preview image P including the user's face image acquired through the second camera (FIG. 2A) is directly displayed on the display unit 151, Is displayed in real time through the preview image (P), it is possible to check the process of automatically setting the focus according to the viewing direction in real time.

That is, referring to FIG. 12, even in the self-photographing mode, the focus can be automatically set according to the user's gaze direction, and the image can be photographed based on the set focus.

13A to 14B are diagrams illustrating an example of editing a preview image according to a direction of a user's gaze according to another embodiment of the present invention.

The control unit 180 can manage the preview image P obtained through the camera (including both the first camera 121 'and the second camera 121) into a plurality of regions having different functions mapped.

The mapped function may include a function of enlarging or reducing the size of the preview image. In addition, for example, the mapped function may include a function of moving or rotating the position of the preview image along the viewing direction. The scope of the present invention is not limited thereto but can be variously modified.

Referring to FIG. 13A, when the user's gaze direction is determined to gaze at the upper end region r1 of the preview image P1 for a predetermined time, (P1) can be enlarged and displayed. The preview image P2 is an enlarged preview image.

Referring to FIG. 13B, when the user's gaze direction is determined to gaze at the lower end region r2 of the preview image P1 for a predetermined time, the control unit 180 determines that the lower end region r2 The preview image P2 can be displayed in a reduced size. The preview image P4 is a preview image reduced according to the above-described criteria.

Meanwhile, the controller 180 may divide the reference area in which the user's gaze direction is to be maintained, as shown in FIGS. 14A and 14B, to enlarge or reduce the size of the preview image.

For example, when the user's gaze direction is maintained for a predetermined time in the first area r1, the preview image P1 is enlarged based on the gazing point, and the enlarged preview image P2 is displayed on the display unit 151. [ .

In a state where the enlarged preview image P3 is displayed on the display unit 151 and the user's gaze direction is maintained for a certain time with respect to the outer region r2 of the first region r1, The image P3 can be reduced and the reduced preview image P4 can be displayed on the display unit 151. [

In other words, in the case of FIGS. 13A to 14B, the display of the preview image can be controlled according to a specific function mapped to a point where the user's eyes stay, even when the touch input is not received from the user.

Although only the function of enlarging or reducing the size of the preview image has been described above, the present invention is not limited to this, and various modifications may be made.

The above-described method of controlling an electronic device according to the present invention can be provided by being recorded on a computer-readable recording medium as a program for execution on a computer.

The control method of the electronic device according to the present invention can be executed through software. When executed in software, the constituent means of the present invention are code segments that perform the necessary tasks. The program or code segments may be stored in a processor readable medium or transmitted by a computer data signal coupled with a carrier wave in a transmission medium or a communication network.

A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording device include a ROM, a RAM, a CD-ROM, a DVD 占 ROM, a DVD-RAM, a magnetic tape, a floppy disk, a hard disk, The computer-readable recording medium may also be distributed to networked computer devices so that computer readable code can be stored and executed in a distributed manner.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The present invention is not limited to the drawings. In addition, the embodiments described in this document can be applied to not only the present invention, but also all or some of the embodiments may be selectively combined so that various modifications can be made.

100: electronic device 151: display part (touch screen)
121 ': first camera 121: second camera

Claims (11)

A first camera;
A second camera for photographing a direction different from the first camera;
A display unit for displaying a preview image obtained through the first camera; And
A second camera for detecting a direction of a user's gaze through the first camera, setting an auto-focus on a point indicated by the gaze direction of the preview image, and capturing an image based on the set focus, ;
. The method according to claim 1,
The direction of the user's gaze,
Wherein the determination is based on a user's facial movement acquired through the second camera. The method according to claim 1,
Wherein,
Acquires the movement of the user's eye through the second camera, and automatically controls the imaging of the image when recognizing a predetermined type of eye blinking after the auto-focus is set. The method according to claim 1,
Wherein,
And sets the first point to auto focus when the direction of the user's gaze to the first point of the preview image is fixed for a predetermined period of time. 5. The method of claim 4,
Wherein,
When the gaze direction of the user is changed and the gaze direction of the user with respect to the second point of the preview image is fixed for the predetermined time after setting the first point to auto focus, And the focus is set to a focus. 6. The method of claim 5,
Wherein,
And the continuous shooting is performed with the first point and the second point as a focal point. The method according to claim 6,
Wherein,
And displays a thumbnail image of a plurality of images obtained by the continuous shooting on the display unit. The method according to claim 1,
Wherein,
Wherein the preview image is acquired through the second camera, detects a direction of a user's gaze from the preview image, and sets an autofocus based on the gaze direction. The method according to claim 1,
Wherein,
Wherein the enlargement unit enlarges the size of the preview image around the first area when the gaze direction of the user is fixed for a predetermined time or longer with respect to the first area of the preview image. 10. The method of claim 9,
Wherein,
And reduces the size of the preview image around the second area when the gaze direction of the user is fixed for a predetermined time or longer with respect to the second area of the preview image. Displaying a preview image obtained through a first camera on a display unit;
Sensing a user's gaze direction through a second camera that captures a direction different from the first camera as the capturing is started through the first camera;
Setting an auto-focus on a point indicated by the gaze direction among the preview images; And
Capturing an image based on the set focus;
And controlling the electronic device.

Images