KR20110002922A - Electronic device and method of performing function using same - Google Patents

Abstract

PURPOSE: An electronic device for setting a virtual input area to the outside and an executing method using the same are provided to set up the virtual input area by selecting an object existed outside the electronic device. CONSTITUTION: A controller(180) recognizes the movement of an external object forming a closed line from an image acquired through a camera(121). The controller sets up a surface which is parallel to the closed line as a virtual input area. If the starting point of the external objects is within the specific distance, the controller recognizes the closed line.

Description

Electronic device and method of performing function using same

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic device, and relates to an electronic device in which a virtual input area is set outside of the electronic device, and predetermined information is input to the electronic device through the set virtual input area, and a method of performing a function using the same. .

Recently, electronic devices provide various functions. Conventionally, it has been common to provide specialized functions for each electronic device, but recently, since various functions are provided in one electronic device, the importance of an information input function for conveniently controlling the functions or easily inputting information has gradually increased. .

For example, in the case of DTV or IPTV, it is common to input information using a remote control, and in the case of a mobile terminal such as a mobile phone, it is common to input information through a keypad or a touch screen provided.

However, since the input means provided in the conventional electronic device does not function as a means for conveniently controlling the various functions in the recent trend of fusion of various functions, there is a need for a new input means completely different from the conventional one. It is emerging.

An object of the present invention is to provide an electronic device capable of setting a virtual input area for information input outside of the electronic device and a method of performing a function using the same.

Another object of the present invention is to provide an electronic device capable of controlling the function of the electronic device and inputting various information through the virtual input area and a method of performing the function using the same.

An electronic device according to a first aspect of the present invention includes a camera for photographing an external image; And a controller configured to recognize a movement of an external object that substantially forms a closed curve from the image acquired by the camera, and to set a plane substantially parallel to the closed curve as a virtual input area for inputting information.

An electronic device according to a second aspect of the present invention includes a camera for photographing an external image; And a controller configured to select a first external object among at least one external object included in the image acquired through the camera, and to set a surface of the first external object as a virtual input area for inputting information.

In a method of performing a function of an electronic device according to a third aspect of the present invention, a method of performing a function in an electronic device having a camera for capturing an external image includes: substantially closing curves of an external object from an image obtained through the camera; Recognizing the movement to form a; And setting a plane substantially parallel to the closed curve as a virtual input area for inputting information.

In a method of performing a function of an electronic device according to a fourth aspect of the present invention, a method of performing a function in an electronic device having a camera for capturing an external image, the method comprising: at least one external device included in an image acquired through the camera Receiving a selection signal for a first external object of the objects; And setting a surface of the selected first external object as a virtual input area for inputting information.

According to the electronic device and a method of performing a function using the same according to the present invention, the following effects are obtained.

According to the present invention, a virtual input area can be set outside the electronic device. The user may arbitrarily set an area desired by the user outside of the electronic device.

According to the present invention, the user can set the virtual input area by selecting any object existing outside the electronic device.

According to the present invention, even without directly operating the electronic device, various functions of the electronic device may be controlled or predetermined information may be input through the virtual input area.

The above objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Like numbers refer to like elements throughout. In addition, when it is determined that the detailed description of the known function or configuration related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

EMBODIMENT OF THE INVENTION Hereinafter, the electronic device which concerns on this invention is demonstrated in detail with reference to drawings. The suffixes "module" and "unit" for components used in the following description are given or used in consideration of ease of specification, and do not have distinct meanings or roles from each other.

The electronic device described herein includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation terminal, and a digital television (DTV). ), And IPTV (Internet Protocol Television).

1 is a block diagram of an electronic device according to an embodiment of the present invention. 2 is a diagram illustrating a case in which the electronic device 100 according to an embodiment of the present invention is a DTV or an IPTV. 3 is a diagram illustrating a case in which the electronic device 100 according to another embodiment of the present invention is a mobile terminal.

The electronic device 100 may include a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory unit 160, The interface unit 170 may include a controller 180, a power supply unit 190, and the like.

Since the components shown in FIG. 1 are not essential, an electronic device having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the electronic device 100 and the wireless communication system or between a network in which the electronic device 100 and the electronic device 100 are located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information, or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to an electronic device. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 may receive a broadcast signal using various broadcast systems. For example, the broadcast receiving module 111 may include: Digital Multimedia Broadcasting-Terrestrial (DMB-T), Digital Multimedia Broadcasting-Satellite (DMB-S), Media Forward Link Only (MediaFLO), and Digital Video Broadcast (DVB-H). Digital broadcast signals can be received using digital broadcasting systems such as Handheld and Integrated Services Digital Broadcast-Terrestrial (ISDB-T). Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems that provide broadcast signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory unit 160.

The mobile communication module 112 transmits and receives a wireless signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The wireless internet module 113 refers to a module for wireless internet access, and the wireless internet module 113 may be internal or external to the electronic device 100. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication module 114 refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like may be used.

The location information module 115 is a module for checking or obtaining the location of the electronic device. The location information module 115 may obtain location information using a global navigation satellite system (GNSS). Here, the Global Positioning System (GNSS) is a term used to describe the radionavigation satellite systems in which certain types of radionavigation receivers orbiting the Earth are transmitting reference signals that can determine their position near the surface or ground surface . The Global Positioning System (GNSS) includes Global Positioning System (GPS) in the United States, Galileo in Europe, GLONASS (Global Orbiting Navigational Satelite System) in Russia, COMPASS in China and Japan QZSS (Quasi-Zenith Satellite System), which is operated by the Ministry of Land, Infrastructure and Transport.

As a representative example of the GNSS, the location information module 115 may be a Global Position System (GPS) module. The GPS module calculates information about a distance at which a point (object) is separated from three or more satellites, information about a time at which the distance information is measured, and then applies a triangulation method to the calculated distance information. Three-dimensional position information according to latitude, longitude, and altitude of one point (object) can be calculated. Furthermore, a method of calculating position and time information using three satellites and correcting the error of the calculated position and time information using another satellite is also used. The GPS module continuously calculates the current position in real time, and calculates velocity information using the current position.

Referring to FIG. 1, the A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and an audio receiver 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory unit 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be provided according to the configuration aspect of the electronic device 100.

The camera 121 may include at least one of a two-dimensional camera and a three-dimensional camera. When the 3D camera is provided, the 3D camera may convert the 3D depth of the obtained image into a coordinate value and transmit the coordinate to the controller 180. Examples of 3D cameras include stereo cameras, TOF cameras, Structured Light Cameras, and the like.

The audio receiver 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. The audio receiver 122 may implement various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for the user to control the operation of the electronic device 100. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 140 is an electronic device 100 such as an open / closed state of the electronic device 100, a location of the electronic device 100, a presence or absence of a user contact, an orientation of the electronic device 100, an acceleration / deceleration of the electronic device 100, or the like. A sensing signal for controlling the operation of the electronic device 100 is generated by detecting a current state or an external environment. For example, when the electronic device 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, it may be responsible for sensing functions related to whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to an external device, and the like. The sensing unit 140 may include a proximity sensor 142.

The output unit 150 is used to generate an output related to visual, auditory, or tactile, and may include a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154. have.

The display unit 151 displays and outputs information processed by the electronic device 100. For example, when the electronic device 100 is in a call mode, the electronic device 100 displays a user interface (UI) or a graphic user interface (GUI) related to the call. When the electronic device 100 is in a video call mode or a photographing mode, the electronic device 100 displays a photographed and / or received image, a UI, or a GUI.

The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display).

Some of these displays can be configured to be transparent or light transmissive so that they can be seen from the outside. This may be referred to as a transparent display. A representative example of the transparent display is a transparent LCD. The rear structure of the display unit 151 may also be configured as a light transmissive structure. With this structure, the user can see an object located behind the body of the electronic device 100 through the area occupied by the display unit 151 of the body of the electronic device 100.

Two or more display units 151 may exist according to the implementation form of the electronic device 100. For example, a plurality of display units may be spaced apart or integrally disposed on one surface of the electronic device 100, or may be disposed on different surfaces.

When the display unit 151 and a sensor for detecting a touch operation (hereinafter, referred to as a touch sensor) form a mutual layer structure (hereinafter, abbreviated as “touch screen”), the display unit 151 is an output device. It can also be used as an input device. The touch sensor may have, for example, a form of a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display unit 151 or capacitance generated at a specific portion of the display unit 151 into an electrical input signal. The touch sensor may be configured to detect not only the position and area of the touch but also the pressure at the touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and then transmits the corresponding data to the controller 180. As a result, the controller 180 can know which area of the display unit 151 is touched.

Referring to FIG. 1, the proximity sensor 142 may be disposed in an inner region of the electronic device 100 surrounded by the touch screen or near the touch screen. The proximity sensor 142 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without using mechanical force by using an electromagnetic force or infrared rays. The proximity sensor 142 has a longer life and higher utilization than a contact sensor.

Examples of the proximity sensor 142 include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

When the touch screen is capacitive, the touch screen is configured to detect the proximity of the pointer by the change of the electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching the pointer on the screen is called "contact touch." The position where the proximity touch is performed by the pointer on the touch screen refers to a position where the pointer is perpendicular to the touch screen when the pointer is in proximity proximity.

The proximity sensor 142 detects a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory unit 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 152 outputs a sound signal related to a function (for example, a call signal reception sound or a message reception sound) performed in the electronic device 100. The sound output module 152 may include a receiver, a speaker, a buzzer, and the like. In addition, the sound output module 152 may output sound through the earphone jack 116. The user can connect the earphone to the earphone jack 116 to hear the sound output.

The alarm unit 153 outputs a signal for notifying occurrence of an event of the electronic device 100. Examples of events generated in the electronic device 100 include call signal reception, message reception, key signal input, and touch input. The alarm unit 153 may output a signal for notifying an occurrence of an event by vibration, in addition to a video signal or an audio signal. The video signal or the audio signal may also be output through the display unit 151 or the sound output module 152.

The haptic module 154 generates various tactile effects that the user can feel. Vibration is a representative example of the haptic effect generated by the haptic module 154. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or may be sequentially output.

In addition to the vibration, the haptic module 154 may be used for the effects of stimulation by the arrangement of pins vertically moving with respect to the contact skin surface, the effect of the injection force of the air through the injection or inlet or the stimulation through the suction force, and the stimulation that rubs the skin surface. Various tactile effects may be generated, such as effects by stimulation through contact of electrodes, effects by stimulation using electrostatic force, and effects of reproducing a sense of warmth and heat using an endothermic or heat generating element.

The haptic module 154 may not only deliver the haptic effect through direct contact, but also implement the haptic effect through the muscle sense of the user's finger or arm. Two or more haptic modules 154 may be provided according to a configuration aspect of the electronic device 100.

The memory unit 160 may store a program for the operation of the controller 180 and may temporarily store input / output data (for example, a phone book, a message, a still image, a video, etc.). The memory unit 160 may store data regarding vibration and sound of various patterns output when a touch input on the touch screen is performed.

The memory unit 160 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM) Magnetic Memory, Magnetic It may include a storage medium of at least one type of disk, optical disk. The electronic device 100 may operate in association with a web storage that performs a storage function of the memory unit 160 on the Internet.

The interface unit 170 serves as a path with all external devices connected to the electronic device 100. The interface unit 170 receives data from an external device or receives power and transmits the data to each component in the electronic device 100, or transmits the data inside the electronic device 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip that stores various types of information for authenticating the use authority of the electronic device 100, and includes a user identification module (UIM), a subscriber identify module (SIM), and a universal user authentication module ( Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Therefore, the identification device may be connected to the electronic device 100 through a port.

The interface unit 170 may be a passage through which power from the cradle is supplied to the electronic device 100 when the electronic device 100 is connected to an external cradle, or various commands input by the user from the cradle. It may be a passage through which a signal is transmitted to the electronic device 100. Various command signals or power input from the cradle may operate as signals for recognizing that the electronic device 100 is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the electronic device 100. For example, it performs related control and processing for voice call, data communication, video call, voice recognition and the like. The controller 180 may include a multimedia module 181 for playing multimedia. The multimedia module 181 may be implemented in the controller 180 or may be implemented separately from the controller 180.

The controller 180 may perform a pattern recognition process for recognizing a writing input or a drawing input performed on the touch screen as text and an image, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

Various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware or a combination thereof.

According to a hardware implementation, embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing the functions. It may be implemented by the controller 180.

In a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that allow at least one function or operation to be performed. The software code may be implemented by a software application written in a suitable programming language. Also, the software codes may be stored in the memory unit 160 and executed by the control unit 180. [

As illustrated in FIG. 2, the electronic device 100 may be controlled through the remote control 200. The remote control 200 and the electronic device 100 may perform communication by infrared communication or other RF signals.

In addition, the remote control 200 may be implemented in the form of a 3-Dimensional Pointing Device. The 3D pointing device may detect 3D motion and transmit information about the detected 3D motion to the electronic device 100. The three-dimensional movement may correspond to a command for controlling the electronic device 100. The user may transmit a predetermined command to the electronic device 100 by moving the 3D pointing device in space. The 3D pointing device may include various key buttons. The user may input various commands through the key button.

Hereinafter, embodiments of the present invention will be described. Embodiments of the present invention may be implemented in the electronic device 100 described with reference to FIGS. 1 to 3. Hereinafter, for convenience of description, the embodiments of the present invention will be described assuming that the electronic device 100 is the DTV illustrated in FIG. 2. As described above, it is apparent that the technical idea disclosed in this document may be applied to various electronic devices including DTVs, IPTVs, mobile terminals, and the like.

Setting of the virtual input area 20-movement of the external object

4 is a flowchart of a method of performing a function of an electronic device according to a first embodiment of the present invention. 5 to 8 are diagrams for describing a method of performing a function of an electronic device according to a first embodiment of the present invention. Hereinafter, a method of performing a function of an electronic device and an operation of the electronic device 100 to implement the same will be described in detail with reference to the accompanying drawings.

The controller 180 enters a virtual input mode related to the function disclosed in this document [S100].

In the virtual input mode, as described later, a virtual input area is set outside the electronic device 100, and a control signal of the electronic device 100 is received through the set virtual input area or various information is received. Input mode for receiving input.

The virtual input mode may exclude or simultaneously use an input mode through an input means provided in the electronic device 100 itself.

Entry into the virtual input mode may take various forms. For example, a specific key for entering the virtual input mode may be provided in the electronic device 100 and / or the remote control 200.

Also, for example, when the remote control 200 senses a motion in space as a 3D pointing device and transmits the motion information to the electronic device 100, the spatial motion of the remote control 200 is virtual. It can be set for entry into the input mode.

5 illustrates an example in which a specific movement of the remote control 200 is set to enter the virtual input mode.

Referring to FIG. 5, when the remote control 200 detects a motion forming a circle in space, the remote control 200 may be set to enter the virtual input mode.

Alternatively, the controller 180 may enter the virtual input mode when detecting a predetermined movement or gesture of an external object through the image acquired through the camera 121.

The controller 180 acquires an external image through the camera 121 [S110], and analyzes the obtained image to recognize a movement of the external object 10 [S120].

For example, referring to FIG. 6, when a user moves a finger 10 to form a specific trajectory 11 in space, the controller 180 analyzes an image acquired through the camera 121 and analyzes the finger. The movement of 10 can be recognized.

Here, the controller 180 analyzes an image acquired by the camera 121, and when the movement of the external object 10 is detected, a point to track the movement of the external object 10. point) can be set.

For example, referring to FIG. 6, when the user starts to move the finger 10, the controller 180 analyzes an image acquired through the camera 121 to recognize the end of the finger 10. Can be set as a tracking point.

When the tracking point is set, the controller 180 can perform image analysis more efficiently by analyzing the image around the movement of the tracking point.

In addition, after the tracking point is set, the controller 180 may ignore the movement of another external object from the image acquired through the camera 121 and ignore the same and monitor only the set tracking point.

The controller 180 may temporarily or permanently store information regarding the movement of the external object 10 in the memory 160 in performing the step S120.

The information about the movement of the external object 10 may include information about a point at which the external object 10 first started moving, information about a time when the external object 10 first started moving, and the external object 10. Information about a moving path of the "

The controller 180 determines whether the movement of the external object 10 substantially forms a closed curve [S130].

Various criteria regarding the formation of the closed curve may be set. For example, when the movement of the external object 10 enters a predetermined distance from a point where the first movement starts after the movement of the external object 10 forms a specific trajectory, the movement of the external object 10 is substantially reduced. It can be judged that a closed curve is formed.

For example, referring to FIG. 7, the controller 180 forms a specific trajectory from the point A at which the movement of the external object 10 first starts to move, and then enters a predetermined distance d1. At point B, it is determined that the movement of the external object 10 has not yet formed a closed curve.

Also, for example, referring to FIG. 7, when the movement of the external object 10 reaches a point C passing through the point B and within the preset distance d1, It may be determined that the movement of the external object 10 substantially forms a closed curve.

The controller 180 may perform step S130 by using information about the movement of the external object 10 stored in the memory 160.

The controller 180 returns to step S110 when the movement of the external object 10 does not substantially form a closed curve as a result of the determination in step S130.

As a result of the determination in step S130, when the movement of the external object 10 substantially forms a closed curve, the controller 180 sets a plane substantially parallel to the formed closed curve as a virtual input area [S140]. .

For example, referring to FIG. 8, when the movement of the user's finger forms the closed curve 11, the controller 180 sets a plane substantially parallel to the closed curve 11 as the virtual input area 20. Can be.

The surface set as the virtual input area 20 may be set in various ways.

For example, the controller 180 may set a surface substantially parallel to the closed curve 11 and having a substantially boundary of the closed curve 11 as the virtual input area 20.

In addition, for example, the controller 180 may set the plane existing inside the closed curve 11 as the virtual input area 20 as a plane substantially parallel to the closed curve 11.

Setting of virtual input area 20-select external object (1)

9 is a flowchart of a method of performing a function of an electronic device according to a second embodiment of the present invention. 10 is a diagram for describing a method of performing a function of an electronic device according to a second embodiment of the present disclosure. Hereinafter, a method of performing a function of an electronic device according to a second embodiment of the present invention and an operation of the electronic device 100 for implementing the same will be described in detail with reference to the accompanying drawings.

The controller 180 enters a virtual input mode related to a function disclosed in this document [S200], acquires an external image through the camera 121 [S210], and analyzes the obtained image to generate a second image. 1 recognizes the movement of the external object 10 [S220].

The steps S200 to S220 correspond to the steps S100 to S120 in the above-described first embodiment.

The controller 180 determines whether the movement of the first external object 10 recognized in step S220 corresponds to a movement of selecting a second external object 15 different from the first external object 10. [S230].

10 shows a user selecting a book placed on a desk with a finger. In FIG. 10, the finger corresponds to the first external object 10 and the book 20 corresponds to the second external object 15.

For example, referring to FIG. 10, the controller 180 analyzes an image acquired by the camera 121 to recognize a movement of the finger 10, and the finger 10 reads the book ( 15, or when the distance between the finger 10 and the book 15 is within a preset distance, it may be determined that the second external object 15 is selected.

The controller 180 recognizes each of the objects through the depth of the external objects using a 3D camera, or analyzes the colors of the external objects using a 2D camera and / or a 3D camera to analyze each of the external objects. You can also recognize it.

The controller 180 may set the surface of the selected second external object 15 as the virtual input area 20 [S240].

The surface of the second external object 15 is not necessarily flat. The surface of the second external object 15 may be flat or curved.

Setting of virtual input area 20-select external object (2)

11 is a flowchart of a method of performing a function of an electronic device according to a third embodiment of the present invention. 12 to 14 are diagrams for describing a method of performing a function of an electronic device according to a third embodiment of the present invention. Hereinafter, a method of performing a function of an electronic device according to a third embodiment of the present invention and an operation of the electronic device 100 for implementing the same will be described in detail with reference to the accompanying drawings.

The controller 180 enters a virtual input mode related to a function disclosed in this document [S300], and acquires an external image through the camera 121 [S310].

Steps S300 and S310 correspond to steps S100 and S110 in the above-described first embodiment.

The controller 180 displays at least one external object included in the obtained external image on the display unit 151 [S320].

12 illustrates an example in which the acquired external image is displayed on the display unit 151. For example, the external image shown in FIG. 12 includes three external objects 17. The controller 180 may display an indicator 22 on the display unit 151.

The controller 180 receives a selection signal for a first external object of the at least one external object displayed on the display unit 151 [S330].

FIG. 13 is a diagram illustrating an example of selecting a specific external object among the external objects 17 illustrated in FIG. 12. For example, referring to FIG. 13, a user may select the specific external object among the external objects 17 by controlling the indicator 22. The indicator 22 may be controlled by the remote control 200.

The controller 180 may display an external object currently indicated by the indicator 22 to be distinguished from other external objects. Referring to FIG. 13A, the outline of the person 17a indicated by the indicator 22 may be displayed to be thicker than the other objects 17b and 17c. Referring to FIG. 13B, the outline of the book 17b indicated by the indicator 22 may be displayed to be thicker than the other objects 17a and 17c.

As shown in FIG. 13, in the state where the indicator 22 indicates a specific external object, the user instructs the indicator 22 by pressing the confirmation button provided in the remote control 200. You can select specific external objects.

On the other hand, when the display unit 151 is a touch screen, the user may select the specific external object by touching a specific external object desired by the finger.

In addition, the controller 180 may analyze the obtained external image and separately provide external objects recognized from the obtained external image in the form of an icon or an image. 14 is a diagram illustrating an example of external objects provided in the form of an icon.

Referring to FIG. 14, the external objects 17 illustrated in FIGS. 12 and 13 may be provided to the display unit 151 as icons 18, respectively.

As in the case of FIG. 13, the user may control the indicator 22 to select a specific icon among the icons 18. As the user selects the specific icon, the controller 180 selects an external object corresponding to the selected specific icon.

The controller 180 may set a surface of the selected first external object as the virtual input area [S340]. For example, when the user selects the book 17b from the external images shown in FIGS. 12 to 14, the controller 180 sets the surface of the book 17b as the virtual input area.

Setting of the virtual input area 20-selection and movement of external objects

15 is a flowchart illustrating a method of performing a function of an electronic device according to a fourth embodiment of the present invention. 16 and 17 are diagrams for describing a method of performing a function of an electronic device according to a fourth embodiment of the present invention. Hereinafter, a method of performing a function of an electronic device according to a fourth embodiment of the present invention and an operation of the electronic device 100 for implementing the same will be described in detail with reference to necessary drawings.

The controller 180 enters a virtual input mode related to a function disclosed in this document [S400], and acquires an external image through the camera 121 [S410].

The steps S400 and S410 correspond to the steps S100 and S110 in the above-described first embodiment.

The controller 180 analyzes the acquired image to recognize a movement of an external first object [S420], and recognizes a second object indicated by the first object according to the movement of the first object. [S430].

The controller 180 may set the closed curve inner surface of the surface of the second object, which is substantially formed by the movement of the first object, as the virtual input area [S440].

For example, referring to FIG. 16, when the user moves the finger 24 corresponding to the first object near the desk 25 corresponding to the second object or touches the desk 25, the controller ( 180 may recognize that the finger 24 points to the desk 25 by analyzing the movement of the finger 24 [S420, S430].

As illustrated in FIG. 16, the user may draw a trajectory that substantially forms a closed curve on the surface of the desk 25 with the finger 24. The controller 180 may monitor the movement of the finger 24 to set the area 20 surrounded by the formed closed curve as the virtual input area 20 [S440].

For example, referring to FIG. 17, when the user moves the right hand finger 24 corresponding to the first object near the left hand palm 26 corresponding to the second object or touches the palm 26, The controller 180 may analyze the movement of the finger 24 to recognize that the finger 24 points to the palm 26 [S420, S430].

As illustrated in FIG. 17, the user may draw a trajectory that substantially forms a closed curve on the palm 26 with the finger 24. The controller 180 may monitor the movement of the finger 24 to set the area 20 surrounded by the formed closed curve as the virtual input area 20 [S440].

After the virtual input area 20 is set in various ways as described in the above-described first to fourth embodiments, the power of the electronic device 100 is cut off or by the user's command. May be turned on again after it is turned off.

As the electronic device 100 is restarted, the controller 180 may be set to maintain the virtual input mode. In this case, as the electronic device 100 is turned off and then restarted, the controller 180 can maintain the virtual input area 20 set or activated before being turned off.

Alternatively, as the electronic device 100 is restarted, the controller 180 maintains the virtual input mode, but may request a user to set a new virtual input area.

In addition, the virtual input area 20 set in various ways as described in the first to fourth embodiments described above may be managed by a separate database. The separate database may be stored in a separate memory that can be attached to or detached from the memory 160 or the electronic device 100.

For example, the database may manage the virtual input area 20 set by a user in various ways. For example, the virtual input area 20 may be databaseized and managed according to various management criteria such as user, time, and place.

The user may call the database by controlling the electronic device 100 through various input means. The user can select his or her desired virtual input area 20 from the called database without a separate setting process. For example, when the electronic device 100 is a DTV placed in a living room, the environment of the living room will not change significantly. In this case, the user may select a specific virtual input area among a plurality of virtual input areas stored in the database, and the controller 180 may determine whether the selected virtual input area exists or matches in the environment of the current living room. have.

In addition, the process of setting the various virtual input areas 20 as described in the first to fourth embodiments described above may be provided through the display unit 151.

For example, the controller 180 displays an image acquired through the camera 121 on the display unit 151 and other objects included in the virtual input area 20 in the acquired image. It can be displayed separately from.

In addition, the controller 180 may display the setting process of the virtual input area 20 on the display unit 151 while maintaining the original screen displayed before the setting of the virtual input area 20.

In this case, the controller 180 may overlap the region displaying the setting process of the virtual input region 20 and the region displaying the original screen, or display the divided region in a separate region.

When an area displaying the setting process and an area displaying the original screen overlap, the controller 180 may process one of the two areas semi-transparently.

When the camera 121 includes both a 2D camera and a 3D camera, the controller 180 displays an image acquired through the 2D camera in an area displaying the setting process, and obtains the image through the 3D camera. The image to be used may be used as an image for analysis necessary in the setting process.

Virtual input area 20 and Of the display unit 151 matching

The virtual input area 20 set in the first to fourth embodiments described above may be matched with a screen of the display unit 151 of the electronic device 100, or face-to-surface matching or point-to-point. Point-to-point matching can be made.

In order for a user to use the virtual input area 20 as an external touch device, a specific point of the virtual input area 20 needs to correspond to a specific point of the display unit 151.

18 to 26 are diagrams for describing matching between the virtual input area 20 and the display unit 151. 18 to 24 illustrate various methods of matching between the virtual input area 20 and the display unit 151. 25 and 26 are diagrams for describing a case in which no matching is required between the virtual input area 20 and the display unit 151.

Hereinafter, matching between the virtual input area 20 and the display unit 151 will be described with reference to FIGS. 18 to 26.

Referring to FIG. 18, when the virtual input area 20 is a quadrangle, the controller 180 selects each vertex of the virtual input area 20 and each vertex of the screen constituting the display unit 151. One-to-one matching

The controller 180 may use a reference point for the matching. For example, referring to FIG. 18, the controller 180 uses the center O1 of the display unit 151 as a reference point, and uses the center O1 of the display unit 151 as the virtual input area 20. Can be matched with the center or any point O2.

The controller 180 may perform matching of the remaining points or the entire screen by using the matching of the reference points.

Since the size and shape of the virtual input area 20 and the display unit 151 may be different, the controller 180 scales in matching the virtual input area 20 and the display unit 151. You can also perform scaling.

For example, when a user moves a finger in the virtual input area 20, a function of moving an indicator displayed on the display unit 151 may be performed in response to this. In this case, due to the difference in size and / or shape of the virtual input area 20 and the display unit 151, the movement distance of the finger and the movement distance of the indicator need to be set differently. To this end, the controller 180 may perform the scaling.

The scaling may be performed depending on the size and shape of each of the virtual input area 20 and the display unit 151.

For example, when the virtual input area 20 is smaller than the display unit 151, the distance between any two points within the virtual input area 20 is two points corresponding to the inside of the display unit 151. The scaling may be performed to be smaller than the distance between the two.

Also, for example, when the virtual input area 20 is larger than the display unit 151, the distance between any two points within the virtual input area 20 may correspond to two corresponding points within the display unit 151. The scaling may be performed to be greater than the distance between the points.

On the other hand, a situation in which the boundary of the virtual input area 20 does not exactly match the boundary of the display unit 151 may occur.

For example, as in the first or fourth embodiment described above, when the user draws the closed curve 11 in the air or on the desk with a finger, the movement path of the finger may not necessarily consist of straight lines. . The trajectory 11 to which the user's finger moves, shown in FIG. 19, does not consist of straight lines.

In this case, as shown in FIG. 19, the controller 180 considers the trajectory of the closed curve 11 so that the virtual input area 20 includes only straight lines. Can be set. The virtual input area 20 shown in FIG. 19 exists inside the closed curve 11 formed by the user's finger, and has four straight lines B2-A2, A2-D2, D2-C2 and C2-B2. It can be set to consist of a.

The controller 180 may set the virtual input area 20 in various ways such that the virtual input area 20 corresponds to the closed curve 11.

For example, as illustrated in FIG. 20, the controller 180 may set the virtual input area 20 to inscribe the virtual input area 20 to the closed curve 11.

For example, as illustrated in FIG. 21, the controller 180 may set the virtual input area 20 to allow the virtual input area 20 to circumscribe the closed curve 11.

For example, as shown in FIG. 22, the control unit 180 may have the virtual input area such that the virtual input area 20 has an intermediate state between a state of being circumscribed to the closed curve 11 and an inscribed state. 20 can be set.

For example, the controller 180 may set the virtual input area 20 so that the size of the virtual input area 20 has a constant value. 23 and 24, although the size of the closed curve 11 formed by the user is different from each other, the size of the virtual input area 20 is the same.

23 and 24, the size of the virtual input area 20 may be preset to several sizes having different values. The controller 180 compares the size of the closed curve 11 formed by the user with the finger and the plurality of preset sizes to determine the size closest to the size of the closed curve 11 among the plurality of preset sizes. The size of the virtual input area 20 may be determined.

25 and 26 illustrate examples in which the matching between the virtual input area 20 and the display unit 151 is not necessary.

Face-to-face matching or point-to-point matching between the virtual input area 20 and the display unit 151 may be performed based on absolute coordinates. In absolute coordinate-based matching, in theory, all points of the virtual input area 20 and all points between the display unit 151 may correspond to each other in a 1: 1 relationship.

However, information input through the virtual input area 20 may be performed based on relative coordinates. For example, according to information input based on relative coordinates, a first moving distance of a specific external object on the virtual input area 20 and a second moving distance of a cursor displayed on the display unit 151. Can match each other.

According to the matching based on the relative coordinates, the size of the virtual input area 20 and the size or resolution of the display unit 151 do not need to match 1: 1, but an external object made on the virtual input area 20. The specific moving distance of may be matched to the specific pixel distance on the display unit 151 or the specific moving distance of the cursor.

For example, referring to FIG. 25, when a specific object moves in the virtual input area 20 by a specific distance, the window 28 displayed on the display unit 151 is as much as the distance or the number of pixels corresponding to the specific distance. Can be moved.

Meanwhile, information input through the virtual input area 20 may be set to be performed only by a specific trajectory.

For example, as illustrated in FIG. 26, when the user performs a gesture of forming a specific trajectory with a finger in the virtual input area 20, the controller 180 is obtained through the camera 121. It may be set to recognize the performance of the gesture through the image, and to perform a specific function corresponding to the specific trajectory or the gesture.

In this case, matching between the display unit 151 and the virtual input area 20 may not be necessary. This is because the controller 180 can perform a specific function if only the specific trajectory or the shape of the gesture is recognized without detecting the specific trajectory or the size of the gesture according to the absolute coordinate or the relative coordinate. to be.

Movement of the virtual input area 20

27 is a flowchart of a method of performing a function of an electronic device according to a fifth embodiment of the present invention. 28 to 30 are diagrams for describing a method of performing a function of an electronic device according to a fifth embodiment of the present invention. Hereinafter, a method of performing a function of an electronic device according to a fifth embodiment of the present invention and an operation of the electronic device 100 for implementing the same will be described in detail with reference to necessary drawings.

The controller 180 may detect movement of the virtual input area 20 or movement of an external object related to the set virtual input area 20 according to the first to fourth embodiments described above. [S500]. The controller 180 may continuously acquire an external image through the camera 121 in the virtual input mode. Therefore, the controller 180 may always detect the movement of at least one object existing outside by analyzing the obtained external image.

FIG. 28 shows that when the surface of the book 15 is set to the virtual input area 20 according to the above-described second or third embodiment, a user or somebody views the book 15 in the virtual input area ( 20 shows an example of moving from the initially set position to another position.

When the controller 180 detects a movement of the virtual input area 20 or an external object related thereto, the controller 180 determines whether the virtual input mode is set to a fixed mode or to a tracking mode. It is determined whether it is set [S510].

The fixed mode refers to a mode in which the movement is not tracked when there is a movement of the virtual input area 20 or an external object related thereto. The tracking mode refers to a mode for tracking the movement.

If the virtual input mode is set to the fixed mode as a result of the determination in step S510, the controller 180 may request setting of a new virtual input area [S520].

FIG. 29 illustrates an example in which the control unit 180 provides the display unit 151 with a graphical user interface (GUI) 30 that requests setting of a new virtual input area when the fixed mode is set. .

Referring to FIG. 29, the user may control the indicator 22 to select “region setting” 31 and newly set the virtual input region according to the first to fourth embodiments. In addition, when the user selects "end" 32, the controller 180 may end the virtual input mode.

On the other hand, the controller 180 may initialize the virtual input mode without providing the GUI 30 when the virtual input mode is set as the fixed mode as a result of the determination in step S510. The user may newly set the virtual input mode by a separate input method (for example, through a specific gesture or a menu).

If the virtual input mode is set to the tracking mode, the controller 180 may update the virtual input area by tracking the movement and maintain the virtual input area [S530].

FIG. 30 illustrates that when the book 15 is moved from the first position 15a to the second position 15b in FIG. 28, the controller 180 senses the movement of the book 15 to detect the book ( The virtual input area 20 set as the surface of 15 may be tracked from the first position 20a to the second position 20b.

Accordingly, under the tracking mode, the user may continue to use the virtual input area 20 moved from the first location 20a to the second location 20b as a virtual input area for information input.

Under the tracking mode, when the virtual input area 20 or the related object disappears from the angle of view of the camera 121 and reappears within a predetermined time, the controller 180 maintains the virtual input area 20. Can be.

Use of the virtual input area 20

31 is a flowchart of a method of performing a function of an electronic device according to a sixth embodiment of the present invention. 32 is an example of a detailed flowchart of step S610 of FIG. 31. 33 to 45 are diagrams for describing a method of performing a function of an electronic device according to a sixth embodiment of the present invention. Hereinafter, a method of performing a function of an electronic device according to a sixth embodiment of the present invention and an operation of the electronic device 100 for implementing the same will be described in detail with reference to the accompanying drawings.

After the virtual input area 20 is set, the controller 180 can recognize a movement of the first external object rather than an external object related to the virtual input area 20 [S600]. For example, when a user moves a finger to input specific information in the set virtual input area 20, the controller 180 may detect movement of the finger.

The controller 180 may execute a specific function in response to the movement of the first external object [S610].

The step S610 may be performed in various ways and forms. Referring to FIG. 32 and necessary drawings, various examples of how the step S610 is performed will be described.

The controller 180 determines whether the detected movement of the first external object forms a predetermined trajectory [S611].

33 to 35 illustrate examples in which the movement of the first external object forms the predetermined trajectory. 33 to 35, the first trajectory 40, the second trajectory 41, and the third trajectory 42 may be stored in the memory 160 as the predetermined trajectory.

The controller 180 may perform step S611 in consideration of a preset time. For example, as will be described later, since the detected movement of the first external object may be for performing another function, the controller 180 may determine that the movement of the first external object is predetermined within the preset time. It is possible to determine whether or not to form a trajectory.

Accordingly, even if the movement of the first external object does not occur within the preset time even if the movement of the first external object forms the predetermined trajectory, the controller 180 determines that the movement of the first external object does not form the predetermined trajectory. can do.

As a result of the determination in step S611, when the movement forms the predetermined trajectory, the controller 180 may execute a function corresponding to the predetermined trajectory [S612].

For example, the first trajectory 40, the second trajectory 41, and the third trajectory 42 illustrated in FIGS. 33 to 35 may correspond to execution of a broadcast application, Internet access, and main menu display, respectively. have. Corresponding relationships between the first to third trajectories 40, 41, and 42 and the functions corresponding to the first to third trajectories 40, 41, and 42 may be stored in the memory 160.

For example, when the controller 180 detects the first trajectory 40, as illustrated in FIG. 33, the controller 180 may execute a broadcast application corresponding to the first trajectory 40.

For example, when the controller 180 detects the second trajectory 41, as illustrated in FIG. 34, the controller 180 may execute an internet connection corresponding to the second trajectory 41. In this case, the controller 180 may drive a web browser and access a predetermined website.

For example, when the controller 180 detects the third trajectory 42, as illustrated in FIG. 35, the display unit 151 displays a menu 44 corresponding to the third trajectory 40. ) Can be displayed.

In this case, the specific trajectory may overlap and match various functions. For example, in a first state displaying a standby screen, a second state outputting a broadcast, and a third state connected to the Internet, the controller 180 may control the first trajectory 40 illustrated in FIG. 33. When input through the virtual input area 20, different functions may be performed.

For example, when the first trajectory 40 is input in the standby screen state, the controller 180 can display a specific menu on the display unit 151.

In addition, when the first trajectory 40 is input in the second state, which is the broadcast output state, the controller 180 may display electronic program guide (EPG) information on the display unit 151.

In addition, when the first trajectory 40 is input in the third state, the controller 180 may perform user authentication using an ID and password previously stored in a website currently being accessed.

36 is a diagram illustrating an example in which a corresponding function varies depending on a current state with respect to a specific trajectory. The correspondence relationship shown as an example in FIG. 36 may be set or changed by a user.

If the movement does not form the predetermined trajectory as a result of the determination in step S611, the controller 180 determines whether the movement forms a predetermined pattern [S613].

If the predetermined trajectory corresponds to the movement of the first external object on the surface of the virtual input area 20, the predetermined pattern is a movement of the first external object regardless of the surface of the virtual input area 20. Corresponds to.

For example, the user may move the finger up and down once on the virtual input area 20, or move the finger up and down twice, such as an operation in which the user touches the touch screen with a finger. There is this.

37 and 38, an example of an operation in which a user moves a finger up and down on the virtual input area 20 is illustrated. The controller 180 may detect the movement of a finger shown in FIGS. 37 and 38 by analyzing an image acquired by the camera 121.

The controller 180 may select or execute an object displayed on the display unit 151 when the movement forms the predetermined pattern as a result of the determination in step S613 [S615].

For example, referring to FIG. 37, a plurality of icons 50 are displayed on the display unit 151. The user may move the indicator 22 by controlling the remote control 200 or by using a finger in the virtual input area 20.

As shown in FIG. 37, when the user moves the finger up or down one or two times while the indicator 22 indicates the specific icon 50c, the specific icon 50c is selected, or the specific icon is selected. The function corresponding to 50c may be executed.

The predetermined pattern may be formed on the virtual input area 20 or may pass through the virtual input area 20.

FIG. 38 illustrates an example in which the predetermined pattern is formed on the virtual input area 20, and FIG. 39 illustrates an example in which the predetermined pattern is formed through the virtual input area 20.

As a result of the determination in step S613, when the movement does not form the predetermined pattern, the controller 180 may move the indicator displayed on the display unit 151 in response to the movement [S614].

For example, referring to FIG. 40, the user may move the indicator 22 displayed on the display unit 151 by moving a finger on the virtual input area 20. In FIG. 40, when the user's finger moves from the state 10a indicating the point M to the state 10b indicating the point N, the indicator 22 indicates the point M of the display unit 151. It may be moved to the state 22b indicating the point N at 22a.

Here, the movement of the indicator 22 may be performed in different ways depending on whether it is set based on absolute coordinates or based on relative coordinates.

For example, on the basis of the absolute coordinate, the indicator 22 is located at the point of the display unit 151, corresponding to the point where the finger 10 is located in the virtual input area 20. Therefore, in the absolute coordinate base, when the user indicates an arbitrary point of the virtual input area 20 with the finger 10, the indicator 22 is connected to the arbitrary point indicated by the finger 10. You can move to the corresponding position.

For example, on the basis of the relative coordinates, the indicator 22 moves in correspondence with the movement distance of the finger 10 in the virtual input area 20. Therefore, on the basis of the relative coordinates, the indicator 22 may not move even if the user indicates an arbitrary point of the virtual input area 20 with the finger 10.

As described above, the virtual input area 10 exists outside the electronic device 100, but a user may use the virtual input area 10 as an input area as a part of the electronic device 100. have.

Other embodiments of controlling the electronic device 100 using the virtual input area 10 will be described.

FIG. 41 is a diagram illustrating an example of controlling sound output of the electronic device 100 through the virtual input area 10. The user may control the volume of the sound output from the electronic device 100 by moving the finger 10 in the virtual input area 10.

Referring to FIG. 41, when the left and right movement of an external object in the virtual input area 10 corresponds to the volume control function of the sound, the user's finger points to the point N in the state 10c indicating the point M. FIG. Moving to the instructed state 10d, the volume of the sound output from the electronic device 100 increases.

42 and 43 illustrate an example of controlling a broadcast channel through the virtual input area 10. The user may change the broadcast channel of the broadcast being output from the electronic device 100 by moving the finger 10 in the virtual input area 10.

Referring to FIG. 42, when the vertical movement of the external object in the virtual input area 10 corresponds to the broadcast channel changing function, the user's finger indicates the point K in a state 10e in which the user's finger indicates the point J. FIG. If it moves to the state 10f, it is changed to the broadcast channel corresponding to the increased number from the broadcast channel currently being output.

Referring to FIG. 43, when the movement trajectory of the external object inside the virtual input area 10 corresponds to a number, the controller 180 may output a broadcast channel corresponding to the number. In FIG. 43, the broadcast channel being output on the left screen is "06". When the user draws a number "9" in the virtual input area 10, the controller 180 detects this and outputs a broadcast received from the broadcast channel "09" corresponding to the input "9". do.

44 and 45 illustrate an example in which the virtual input area 10 is used as a virtual keyboard. 44 illustrates an example in which the virtual input area 10 is used as a virtual keyboard in a standby screen state, and FIG. 45 illustrates that the virtual input area 10 is used as a virtual keyboard while driving a specific application such as a broadcast screen. An example is shown.

The controller 180 may display the virtual keyboard 52 on the display unit 151. The controller 180 detects the movement of both hands of the user from the image acquired by the camera 121 and displays a hand-shaped object 53 reflecting the detected movement on the display unit 151. can do.

The hand-shaped object 53 may be a real external object captured from an image input through the camera 21 or an object stored in the memory 160 as an object to reflect the detected movement. It may be.

Of course, the shape of the object 53 does not necessarily have the shape of a hand. The controller 180 may provide various objects for reflecting the detected motion.

When the user moves the fingers through the virtual input area 20, the controller 180 may move the object 53 in response to the movement of the fingers.

In addition, the controller 180 may match a position where a plurality of keys included in the virtual keyboard 52 are displayed and the virtual input area 20. Accordingly, as the user moves his / her fingers in the virtual input area 20, a specific key may be selected on the virtual keyboard 52 to input information.

In the case of FIG. 45, at least one of the virtual keyboard 52 and the object 53 may be provided to be opaque or translucent.

In the above-described embodiments, it has been mainly described that the virtual input area is set to correspond to the closed curve when the movement trajectory of the external object forms the closed curve.

However, the technical idea disclosed in this document is not limited thereto. For example, if the movement trajectory of the external object itself does not have to form a closed curve, but the movement trajectory of the external object can correspond to the closed curve, the virtual input area may be set.

46 to 48 illustrate examples in which the movement trajectory of the external object may correspond to the closed curve.

Referring to FIG. 46, a user may take an operation of selecting four points P, Q, R, and S sequentially or in any order with a finger on an object such as a desk or in the air.

The controller 180 detects a motion of the user through an image acquired through the camera 121 and detects the closed curve 20 connecting the four points P, Q, R, and S. The virtual input area 20 may be set.

However, when the four points P, Q, R, and S are selected within a preset time, the closed curve connecting the four points P, Q, R, and S is the virtual input area 20. It may be set to).

Referring to FIG. 47, the user may take an operation of drawing the "X" -shaped trajectories 55 and 56 with a finger on an object such as a desk or in the air.

The controller 180 detects two trajectories 55 and 56 formed by the user's finger, and the closed curve 20 in which the two trajectories 55 and 56 are formed as shown in FIG. 47. ) May be set as the virtual input area 20.

Here, as in FIG. 46, the controller 180 may set the virtual input area 20 in consideration of the time at which the traces of the “X” shape are formed.

In addition, referring to FIG. 48, a user may take an operation of drawing a straight trajectory 57 with a finger on an object such as a desk or in the air.

The controller 180 detects one straight line trajectory 57 formed by the user's finger, and displays the start point E and the end point F of the trajectory 57 of the display unit 151. Each of the corners E and F may be matched to each other, and the closed curve 20 formed as shown in FIG. 48 may be set as the virtual input area 20.

The method for performing a function of an electronic device according to the present invention described above may be provided by recording on a computer-readable recording medium as a program for executing in a computer.

The method of performing a function of an electronic device according to the present invention can be executed through software. When implemented in software, the constituent means of the present invention are code segments that perform the necessary work. The program or code segments may be stored on a processor readable medium or transmitted by a computer data signal coupled with a carrier on a transmission medium or network.

Computer-readable recording media include all kinds of recording devices that store data that can be read by a computer system. Examples of the computer-readable recording device include ROM, RAM, CD-ROM, DVD ± ROM, DVD-RAM, magnetic tape, floppy disk, hard disk, optical data storage device, and the like. The computer-readable recording medium may also be distributed to networked computer devices so that computer readable code can be stored and executed in a distributed manner.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings. In addition, the embodiments described in this document may not be limitedly applied, but may be configured by selectively combining all or part of the embodiments so that various modifications may be made.

1 is a block diagram of an electronic device according to an embodiment of the present invention.

2 is a diagram illustrating a case in which the electronic device 100 according to an embodiment of the present invention is a DTV or an IPTV.

3 is a diagram illustrating a case in which the electronic device 100 according to another embodiment of the present invention is a mobile terminal.

4 is a flowchart of a method of performing a function of an electronic device according to a first embodiment of the present invention.

5 to 8 are diagrams for describing a method of performing a function of an electronic device according to a first embodiment of the present invention.

9 is a flowchart of a method of performing a function of an electronic device according to a second embodiment of the present invention.

10 is a diagram for describing a method of performing a function of an electronic device according to a second embodiment of the present disclosure.

11 is a flowchart of a method of performing a function of an electronic device according to a third embodiment of the present invention.

12 to 14 are diagrams for describing a method of performing a function of an electronic device according to a third embodiment of the present invention.

15 is a flowchart illustrating a method of performing a function of an electronic device according to a fourth embodiment of the present invention.

16 and 17 are diagrams for describing a method of performing a function of an electronic device according to a fourth embodiment of the present invention.

18 to 26 are diagrams for describing matching between the virtual input area 20 and the display unit 151.

27 is a flowchart of a method of performing a function of an electronic device according to a fifth embodiment of the present invention.

28 to 30 are diagrams for describing a method of performing a function of an electronic device according to a fifth embodiment of the present invention.

31 is a flowchart of a method of performing a function of an electronic device according to a sixth embodiment of the present invention.

32 is an example of a detailed flowchart of step S610 of FIG. 31.

33 to 45 are diagrams for describing a method of performing a function of an electronic device according to a sixth embodiment of the present invention.

46 to 48 illustrate examples in which the movement trajectory of the external object may correspond to the closed curve.

<Explanation of symbols for the main parts of the drawings>

100: electronic device 121: camera

151: display unit 160: memory

180: control unit 20: virtual input area

Claims (19)

A camera for capturing an external image; And A control unit for recognizing a movement of an external object that substantially forms a closed curve from the image acquired by the camera, and setting a plane substantially parallel to the closed curve as a virtual input area for inputting information Electronic device containing. The method of claim 1, wherein the control unit, And analyzing the image acquired by the camera and recognizing that the closed curve is formed when the starting point and the arriving point of the movement of the external object are within a predetermined distance. The method of claim 1, wherein the control unit, And the closed curve is set by analyzing a trajectory formed by the movement of the external object through the image acquired by the camera. The method of claim 1, wherein the control unit, And matching the coordinates of the display provided in the electronic device with the coordinates of the virtual input area. The method of claim 1, wherein the control unit, The external device moves the indicator displayed on the display provided in the electronic device in response to the movement in the virtual input area. The method of claim 1, wherein the control unit, And recognizing a gesture of the external object made in the virtual input area after setting the virtual input area, executing a function corresponding to the gesture in advance. A camera for capturing an external image; And A control unit for selecting a first external object among at least one external object included in the image acquired through the camera and setting a surface of the first external object as a virtual input area for inputting information; Electronic device containing. The method of claim 7, wherein the control unit, And displaying the at least one external object on a display provided in the electronic device, and selecting the first external object by receiving a selection signal for the first external object among the displayed at least one external object. Electronic devices. The method of claim 7, wherein the control unit, And selecting the first external object in response to a predetermined gesture of a second external object of the at least one external object. The method of claim 7, wherein the control unit, And update the virtual input area by tracking the movement of the first external object. The method of claim 7, wherein the control unit, And a second external object of the at least one external object moves an indicator displayed on a display provided in the electronic device in response to a movement in the virtual input area. The method of claim 7, wherein the control unit, When the gesture of the second external object is recognized among the at least one external object, a function corresponding to the gesture is executed in advance, The gesture of the second external object is made in the virtual input area. In the method for performing a function in an electronic device having a camera for capturing an external image, Recognizing a movement forming a substantially closed curve of an external object from an image acquired by the camera; And Setting a plane substantially parallel to the closed curve as a virtual input area for inputting information; Method of performing a function of an electronic device comprising. The method of claim 13, After setting the virtual input area, executing a predetermined function in response to the movement of the external object in the virtual input area; Method of performing a function of the electronic device further comprising. In the method for performing a function in an electronic device having a camera for capturing an external image, Receiving a selection signal for a first external object among at least one external object included in the image acquired through the camera; And Setting a surface of the selected first external object as a virtual input area for inputting information; Method of performing a function of an electronic device comprising. The method of claim 15, Displaying the at least one external object on a display provided in the electronic device; The receiving of the selection signal for the first external object may include receiving a selection signal for the first external object among the displayed at least one external object. The method of claim 15, wherein receiving a selection signal for the first external object comprises: And receiving a selection signal for the first external object by recognizing a predetermined gesture of a second external object among the at least one external object. The method of claim 15, And updating the virtual input area by tracking the movement of the first external object. The method of claim 15, After setting the virtual input area, executing a predetermined function in response to a movement of a second external object of the at least one external object, which is performed in the virtual input area; Method of performing a function of the electronic device further comprising.

Images