KR102088866B1 - Mobile terminal - Google Patents

Abstract

The mobile terminal according to the present invention can control the operation of the other electronic device based on the touch on the GUI image of the other electronic device displayed on the preview screen.

Description

Mobile terminal

The present invention relates to a mobile terminal, and more particularly, to a mobile terminal capable of controlling other electronic devices based on a touch on a preview screen.

Terminals such as personal computers, laptops, mobile phones, smartphones, etc. have diversified functions, and multimedia devices with complex functions such as taking pictures or videos, playing music or video files, receiving games and broadcasting, etc. It is implemented in the form of (multimedia player).

In order to support and increase the function of the terminal, it may be considered to improve the structural part and / or software part of the terminal. 2. Description of the Related Art Recently, various terminals, including mobile terminals, are complex and provide a variety of functions, so the menu structure is also complicated. In addition, in recent years, interest, research, and commercialization of mobile terminals capable of controlling other electronic devices connected through short-range wireless communication have been active.

Technical problem to be solved by the present invention relates to a mobile terminal capable of controlling another electronic device based on a touch through a preview screen associated with another electronic device.

Technical problems to be achieved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned above may be clearly understood by those skilled in the art from the following description. There will be.

A mobile terminal according to an embodiment of the present invention for solving the above technical problem, a wireless communication unit to form a short-range wireless communication link with other electronic devices; A camera that acquires a graphic user interface (GUI) image of the other electronic device; A touch screen that displays the GUI image as a preview screen; And a control unit. When a touch is received through the preview screen, the control unit determines a position in the GUI of the other electronic device corresponding to a location where the touch is received on the preview screen, and the determined position is directly in the GUI of the other electronic device. A control signal for controlling the other electronic device is generated to perform the same function as the selected one, and the generated control signal can be transmitted to the other electronic device through the short-range wireless communication link.

The control unit may enlarge and display a part of the GUI image on the preview screen, and perform an enlargement, reduction, and scroll function on the preview screen based on a touch received through the preview screen. The controller may determine a part of the GUI image to be enlarged and displayed on the preview screen based on the focus of the camera.

A mobile terminal according to another embodiment of the present invention includes a wireless communication unit that forms a short-range wireless communication link with other electronic devices; A camera that acquires an image of the other electronic device; A touch screen displaying a preview screen of the image of the other electronic device; And a control unit. The control unit receives a control signal for controlling the other electronic device to perform a specific predetermined function to correspond to a specific part of the other electronic device when a touch on a specific part of the other electronic device is received on the preview screen. It can be generated and transmitted through the short-range wireless communication link.

The controller may enlarge and display a part of the image of the other electronic device on the preview screen, and perform an enlargement, reduction, and scrolling function on the preview screen based on a touch received through the preview screen. .

The controller may determine a part of the image of the other electronic device to be enlarged and displayed on the preview screen based on the focus of the camera. The control unit may automatically enlarge a portion of the image of the other electronic device including the operation unit of the other electronic device to provide a preview screen. The control unit generates a control signal for controlling the other electronic device to perform the same function as the operation unit directly selected by the other electronic device when a touch to the operation unit of the other electronic device displayed on the preview screen is received. In addition, the generated control signal may be transmitted to the other electronic device through the short-range wireless communication link.

A mobile terminal according to another embodiment of the present invention, a wireless communication unit for forming a short-range wireless communication link with other electronic devices; A camera that acquires an image of a specific place;

A touch screen that displays a preview screen of the video of the specific place; And a control unit. The control unit generates a control signal for controlling the other electronic device to perform a predetermined specific function for the specific place when a touch for the specific place is received on the preview screen, and generates the generated control signal. It may be transmitted to the other electronic device through the short-range wireless communication link.

The controller may enlarge and display a part of the image of the specific place on the preview screen, and perform an enlargement, reduction, and scrolling function on the preview screen based on a touch received through the preview screen. The controller may determine a part of the image of the specific place to be enlarged and displayed on the preview screen based on the focus of the camera.

The user of the mobile terminal according to the present invention can control the other electronic device through a touch on the graphic user interface of the other electronic device displayed on the preview screen.

In addition, the user of the mobile terminal according to the present invention can control the other electronic device to perform a predetermined specific function for the specific part through a touch on a specific part of the other electronic device displayed on the preview screen.

In addition, the user of the mobile terminal according to the present invention can control other electronic devices to perform a specific function related to the specific place through a touch on the specific place displayed on the preview screen.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.
2 is a flowchart illustrating an example of a method for driving a mobile terminal according to the present invention.
FIG. 3 illustrates that a preview screen for a part of an image of a GUI of a TV is displayed on the mobile terminal according to the method of driving the mobile terminal illustrated in FIG. 2.
4 shows an example in which the preview screen of the mobile terminal is enlarged and reduced according to the method of driving the mobile terminal illustrated in FIG. 2.
5 is a diagram for explaining a method of matching a location on a preview screen displayed on a mobile terminal according to the present invention to a TV GUI image.
6 illustrates that the content selection function is performed on the TV based on the touch on the preview screen of the mobile terminal according to the method of driving the mobile terminal illustrated in FIG. 2.
7 is a flowchart illustrating another example of a method for driving a mobile terminal according to the present invention.
8 illustrates an example in which an air conditioner image is provided as a preview screen in a mobile terminal according to the method of driving the mobile terminal illustrated in FIG. 7.
9 shows an example in which the air conditioner operation is controlled based on the touch of the preview screen of the mobile terminal 100.
FIG. 10 illustrates that, according to the method of driving the mobile terminal illustrated in FIG. 8, a portion including an air conditioner control unit is automatically enlarged among all images of the air conditioner and is provided as a preview screen in the mobile terminal.
FIG. 11 illustrates controlling on / off of lighting according to the method of driving the mobile terminal illustrated in FIG. 8.
FIG. 12 illustrates controlling on / off of a TV according to the method of driving the mobile terminal illustrated in FIG. 8.
13 is a flowchart illustrating another example of a method for driving a mobile terminal according to the present invention.
14 illustrates an example of controlling the operation of the robot cleaner according to the method of driving the mobile terminal shown in FIG. 13.

The above-described objects, features and advantages of the present invention will become more apparent through the following detailed description in conjunction with the accompanying drawings. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Throughout the specification, the same reference numbers indicate the same components. In addition, when it is determined that a detailed description of known functions or configurations related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description is omitted.

Hereinafter, a mobile terminal according to the present invention will be described in more detail with reference to the drawings. The suffixes "module" and "unit" for components used in the following description are given or mixed in consideration of ease of specification, and do not have distinct meanings or roles.

The mobile terminal described in the present specification may include a mobile phone, a smart phone, a laptop computer, a terminal for digital broadcasting, a personal digital assistants (PDA), a portable multimedia player (PMP), navigation, a dedicated terminal, and the like. have. However, the scope of the present invention is not limited thereto.

1 is a block diagram of a mobile terminal 100 according to an embodiment of the present invention. Referring to FIG. 1, the mobile terminal 100 includes a wireless communication unit 110, an audio / video (A / V) input unit 120, a user input unit 130, a sensing unit 140, and an output unit 150, It may include a storage unit 160, the interface unit 170, the control unit 180 and the power supply unit 190. Since the components shown in FIG. 1 are not essential, the mobile terminal 100 may have more components or fewer components.

Hereinafter, the components will be described in turn.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast reception module 111, a mobile communication module 112, a wireless Internet module 113, a short-range communication module 114, and a location information module 115. .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits broadcast signals and / or broadcast-related information or a server that receives previously generated broadcast signals and / or broadcast-related information and transmits them to a terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, and may also include a TV broadcast signal or a radio broadcast signal combined with a data broadcast signal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, it may exist in the form of an Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or an Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives a broadcast signal using various broadcast systems. In particular, DMB-T (Digital Multimedia Broadcasting-Terrestrial), DMB-S (Digital Multimedia Broadcasting-Satellite), MediaFLO (Media Forward Link) Digital broadcast signals can be received using digital broadcasting systems such as Only), DVB-H (Digital Video Broadcast-Handheld), and ISDB-T (Integrated Services Digital Broadcast-Terrestrial). Of course, the broadcast receiving module 111 may be configured to be suitable for not only the above-described digital broadcast system but also other broadcast systems for providing broadcast signals.

The broadcast signal and / or broadcast-related information received through the broadcast receiving module 111 may be stored in the storage 160.

The mobile communication module 112 transmits and receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text / multimedia message.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 may be built in or external to the mobile terminal 100. Wireless Internet technology includes Wireless LAN (WLAN) (Wi-Fi), Wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and Long Term Evolution (LTE). You can.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), and ZigBee may be used as short-range communication technology.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. The location information module 115 may acquire location information using a Global Navigation Satellite System (GNSS). Here, the global satellite navigation system (GNSS) is a term used to describe radio navigation satellite systems that orbit the Earth to send reference signals that allow certain types of radio navigation receivers to determine their location on or near the surface. . The Global Satellite Navigation System (GNSS) includes Global Position System (GPS) operated in the United States, Galileo operated in Europe, GLONASS (Global Orbiting Navigational Satelite System) operated in Russia, COMPASS operated in China, and Japan. And QZSS (Quasi-Zenith Satellite System) operated by the company.

As a representative example of GNSS, the location information module 115 may be a Global Position System (GPS) module. The GPS module calculates information on a distance at which a point (object) is separated from three or more satellites, and information on a time when the distance information is measured, and then applies a trigonometry method to the calculated distance information to perform one hour. It is possible to calculate three-dimensional position information according to latitude, longitude, and altitude for a point (object). Furthermore, a method of calculating position and time information using three satellites and correcting the error of the calculated position and time information using another satellite is also used. The GPS module continuously calculates the current location in real time and uses it to calculate speed information.

Referring to FIG. 1, an audio / video (A / V) input unit 120 is for inputting an audio signal or a video signal, which may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving pictures obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display unit 151.

The image frames processed by the camera 121 may be stored in the storage unit 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be provided according to a configuration aspect of the terminal.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes it as electrical voice data. The processed voice data may be converted into a form transmittable to a mobile communication base station through the mobile communication module 112 and output. Various noise reduction algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 122.

The user input unit 130 generates input data for the user to control the operation of the terminal. The user input unit 130 may be configured with a key pad dome switch, a touch pad (static pressure / blackout), a jog wheel, a jog switch, or the like.

As illustrated in FIG. 1, the A / V input unit 120 and the user input unit 130 may be implemented as one input unit 116. Various input data generated from the input unit 116 are related to memos performed by the mobile terminal 100 according to the present invention, such as a memo writing function, a memo storage function, a memo insertion function, etc. It can be used to activate or deactivate a function. The input data may include an image received through the camera 121, a user's voice received through the microphone 122, a touch received through the display unit 151, and the like. However, the scope of the present invention is not limited thereto.

The sensing unit 140 moves, such as the open / closed state of the mobile terminal 100, the location of the mobile terminal 100, the presence or absence of user contact, the user's touch action on a specific part, the orientation of the mobile terminal, acceleration / deceleration of the mobile terminal, and the like. The current state of the terminal 100 is sensed to generate a sensing signal for controlling the operation of the mobile terminal 100. The sensing signal is transmitted to the control unit 180, so that the control unit 180 may be a basis for performing a specific function.

The sensing unit 140 includes a touch sensor for detecting whether a user is touched, a vibration sensor for detecting vibration generated based on the user's touch, and a gyro for detecting rotation of the mobile terminal 100. It may include a sensor, an acceleration sensor, a geomagnetic sensor, and the like. However, the scope of the present invention is not limited thereto.

In addition, as illustrated in FIG. 1, the sensing unit 140 may include a posture detection sensor including at least one of the various types of sensors described above. Here, the posture may mean the posture of the user as well as the posture of the mobile terminal 100.

For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, the sensing function related to whether the power supply unit 190 is supplied with power or whether the interface unit 170 is coupled with an external device may be in charge. Meanwhile, the sensing unit 140 may include a proximity sensor.

The output unit 150 is for generating output related to vision, hearing, or tactile sense, and may include a display unit 151, an audio output unit 152, an alarm unit 153, and a haptic module 154. have.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in a call mode, a UI (User Interface) or a GUI (Graphic User Interface) related to the call is displayed. When the mobile terminal 100 is in a video call mode or a shooting mode, it displays a photographed or / and received video, UI, or GUI.

The display unit 151 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a three-dimensional display ( 3D display).

Some of these displays may be of a transparent type or a light transmissive type so that the outside can be seen through them. This may be called a transparent display, and a typical example of the transparent display is a transparent LCD. The rear structure of the display unit 151 may also be configured as a light transmissive structure. With this structure, the user can see an object located behind the terminal body through an area occupied by the display unit 151 of the terminal body.

Two or more display units 151 may exist depending on the implementation form of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display units may be spaced apart or integrally disposed on one surface, or may be disposed on different surfaces.

When the display unit 151 and a sensor that senses a touch operation (hereinafter referred to as a “touch sensor”) form a mutual layer structure (hereinafter, referred to as a “touch screen”), the display unit 151 may be used in addition to an output device. It can also be used as an input device. The touch sensor may have, for example, a form of a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display unit 151 or capacitance generated at a specific portion of the display unit 151 into an electrical input signal. The touch sensor may be configured to detect not only the touched position and area, but also pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and then transmits corresponding data to the controller 180. Accordingly, the control unit 180 can know which area of the display unit 151 has been touched, and the like.

A proximity sensor may be disposed in an inner area of the mobile terminal surrounded by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without mechanical contact using electromagnetic force or infrared rays. Proximity sensors have a longer lifespan and higher utilization than contact sensors.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor.

When the touch screen is capacitive, it is configured to detect the proximity of the cursor by a change in electric field according to the proximity of the cursor. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

The proximity sensor detects a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The audio output unit 152 may output audio data received from the wireless communication unit 110 or stored in the storage unit 160 in a call signal reception, call mode or recording mode, voice recognition mode, broadcast reception mode, or the like. The sound output unit 152 outputs sound signals related to a function (for example, a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The sound output unit 152 may include a receiver, a speaker, and a buzzer. Also, the sound output unit 152 may output sound through the earphone jack 116. The user can hear the sound output by connecting the earphone to the earphone jack 116.

The alarm unit 153 may output a signal for notifying the occurrence of an event in the mobile terminal 100. Examples of events generated in the mobile terminal include call signal reception, message reception, key signal input, and touch input. The alarm unit 153 may output a signal for notifying the occurrence of an event by other forms, such as vibration, in addition to a video signal or an audio signal. The video signal or the audio signal may also be output through the display unit 151 or the sound output unit 152.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 are controllable. For example, different vibrations may be synthesized and output or sequentially output.

In addition to vibration, the haptic module 154 can be applied to effects caused by stimulation by pin arrays that vertically move with respect to the contact skin surface, effects caused by stimulation through air or the inhalation force of air through a spray or inlet, and stimulation through a skin surface. Various tactile effects can be generated, such as an effect, an effect caused by stimulation through contact of an electrode, an effect caused by stimulation using electrostatic force, and an effect of reproducing a feeling of cold and warm using an element capable of absorbing heat or generating heat.

The haptic module 154 may not only transmit the tactile effect through direct contact, but may also implement the haptic effect through muscle sensation such as a user's finger or arm. Two or more haptic modules 154 may be provided according to a configuration aspect of the portable terminal 100.

The storage 160 may store a program for the operation of the controller 180, and may temporarily store input / output data (eg, a phone book, a message, a still image, a video, etc.). The storage unit 160 may store data related to various patterns of vibration and sound output when a touch is input on the touch screen.

The storage unit 160 includes a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD memory), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM) magnetic memory, magnetic It may include a storage medium of at least one type of disk, optical disk. The mobile terminal 100 may operate in connection with a web storage that performs a storage function of the storage unit 160 on the Internet.

The interface unit 170 serves as a passage with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or receives power and transmits it to each component inside the mobile terminal 100 or allows data inside the mobile terminal 100 to be transmitted to the external device.

For example, wired / wireless headset port, external charger port, wired / wireless data port, memory card port, port for connecting devices equipped with an identification module, audio input / output (I / O) port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The interface unit 170 may include a card slot into which the card type user identification module can be mounted. Then, the user identification module may be connected to the mobile terminal 100 through the card slot. At this time, a plurality of user identification modules may be connected to the interface unit 170.

When the mobile terminal 100 is connected to an external cradle, the interface unit 170 becomes a passage through which power from the cradle is supplied to the mobile terminal 100, or various commands input by the user from the cradle. The signal may be a passage to the mobile terminal. Various command signals or power inputted from the cradle may be operated as signals for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, it performs related control and processing for voice calls, data communication, video calls, and the like. The controller 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the controller 180, or may be implemented separately from the controller 180.

The controller 180 may perform pattern recognition processing capable of recognizing handwriting input or picture drawing input performed on the touch screen as text and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 to supply power required for the operation of each component.

The various embodiments described herein can be implemented in a computer- or similar device-readable recording medium using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. It may be implemented by the controller 180.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code can be implemented by software applications written in the appropriate programming language. In addition, the software code is stored in the storage unit 160 and may be executed by the control unit 180.

The configuration of the mobile terminal 100 according to the present invention has been described above with reference to FIG. 1. Hereinafter, other electronic device control features through the preview screen touch of the mobile terminal 100 according to the present invention will be described in more detail with reference to FIGS. 2 to 14.

2 is a flowchart illustrating an example of a method for driving a mobile terminal according to the present invention. Hereinafter, a method of driving the mobile terminal will be described with reference to necessary drawings.

First, the controller 180 controls the camera 121 to obtain a graphic user interface (hereinafter referred to as 'GUI') image of the TV (S100). Here, the TV 200 is an example of an image output device according to the present invention. However, the scope of the present invention is not limited thereto. Then, the controller 180 displays the image of the GUI of the TV 200 on the touch screen 151 as a preview screen (S110).

That is, acquiring the image of the GUI of the TV 200 in the mobile terminal 100 according to the present invention does not mean that the image of the GUI of the TV 200 is captured and stored in the memory 160. It means displaying on the screen through a simple image data buffering process.

Then, a touch is received through the preview screen (S120). Here, the touch may be received by a user's finger or other body part, a tool such as a stylus pen. Then, the controller 180 determines a location on the GUI of the actual TV 200 corresponding to the location on the preview screen where the touch is received (S130).

Then, the control unit 180 generates a control signal for allowing the position determined on the GUI of the TV 200 to perform the same function as directly selected by the user (S140), and then the generated control signal to the TV It can be transmitted to (200). Transmission of the generated control signal may be performed through a short-range wireless communication link formed between the mobile terminal 100 and the TV 200.

FIG. 3 illustrates that a preview screen for a part of the image of the GUI of the TV 200 is displayed on the mobile terminal 100 according to the method of driving the mobile terminal illustrated in FIG. 2. The mobile terminal 100 according to the present invention stores the entire data corresponding to the image of the GUI of the TV 200 therein, but only a part thereof can be selected and provided as a preview screen.

Also, the mobile terminal 100 may enlarge a part of the GUI image of the TV 200 and provide it as a preview screen. Here, a part of the GUI image to be enlarged may be through a simple screen magnification (digital zoom), not through an actual lens zoom. That is, since the entire GUI image is acquired at a predetermined resolution and a part of the GUI image is enlarged, the resolution of the enlarged part may be lowered.

Meanwhile, a part of the GUI image displayed on the preview screen may be selected based on the focus of the camera 121. Also, a part of the GUI image displayed on the preview screen may be a part of a predetermined TV 200 screen. For example, the mobile terminal 100 may enlarge and display a part of the center of the screen of the TV 200, may enlarge and display a part of the upper left, or may enlarge and display a part of the upper right. However, the scope of the present invention is not limited thereto.

4 shows an example in which the preview screen of the mobile terminal 100 is enlarged and reduced according to the method of driving the mobile terminal illustrated in FIG. 2. As shown in (a) of FIG. 4, when a touch in which the two fingers of the user are getting closer and closer is received on the preview screen, the controller 180 displays the preview screen as shown in FIG. 4 (b). Shrink it. On the contrary, when a touch in which two fingers of a user are gradually moving away is received through the preview screen in the state of FIG. 4B, the controller 180 may enlarge the preview screen to the state of FIG. 4A.

Meanwhile, although not illustrated in FIG. 4, when a touch in a specific direction is received through the preview screen, the controller 180 may scroll the preview screen in a specific direction. Also, the controller 180 may enlarge, reduce, and scroll the preview screen based on a predetermined hardware key or software key operation.

5 is a view for explaining a method of matching the location of the preview screen displayed on the mobile terminal 100 according to the present invention to the GUI image of the TV 200. As illustrated in FIG. 5, the mobile terminal 100 calculates an actual location in the GUI of the TV 200 corresponding to the touch location on the preview screen of the mobile terminal 100 in consideration of the enlargement / reduction ratio. You can.

In the equation shown in FIG. 5, P represents coordinates in the TV 200, Po represents coordinates in the mobile terminal 100, and z represents an enlargement / reduction magnification. Meanwhile, the location matching between the mobile terminal 100 and the TV 200 illustrated in FIG. 5 is only an example of a location matching method that can be performed by the mobile terminal 100 according to the present invention, and the scope of the present invention is limited thereto. It does not work.

6 illustrates that the content selection function in the TV 200 is performed based on the touch on the preview screen of the mobile terminal 100 according to the method of driving the mobile terminal illustrated in FIG. 2. First, a user touches a location corresponding to an item on the preview screen displayed on the mobile terminal 100.

Then, the mobile terminal 100 transmits a control signal for controlling the TV 200 to the TV 200 so that the content item in the GUI of the TV 200 corresponding to the touch position operates as a direct selection. . Then, the TV 200 performs a content selection function based on the control signal.

According to the mobile terminal driving method described above with reference to FIGS. 2 to 6, the user may use the mobile terminal 100 as a remote controller of the TV 200 driven based on a touch operation on the preview screen. .

7 is a flowchart illustrating another example of a method for driving a mobile terminal according to the present invention. Hereinafter, a method of driving the mobile terminal will be described with reference to necessary drawings.

First, the controller 180 controls the camera 121 to obtain an image of an air conditioner (S200). Then, the controller 180 displays the image of the air conditioner as a preview screen on the touch screen 151 (S210).

Then, a touch of a specific portion of the air conditioner is received in the preview screen (S220). Then, the control unit 180 may generate a control signal for performing a predetermined function to correspond to the specific portion (S230), and then transmit the generated control signal to the air conditioner. The generated control signal may be transmitted through a short-range wireless communication link formed between the mobile terminal 100 and the air conditioner. The air conditioner receiving the control signal may perform a specific function corresponding to the control signal.

8 shows an example in which the air conditioner 300 image is provided as a preview screen in the mobile terminal 100 according to the method of driving the mobile terminal illustrated in FIG. 7. 9 shows an example in which the operation of the air conditioner 300 is controlled based on the touch of the preview screen of the mobile terminal 100.

The mobile terminal 100 enlarges some of the prerequisite images for the air conditioner 300 and provides it as a preview screen. The portion to be enlarged may be selected based on the focus of the camera 121. In addition, a part to be enlarged may be a specific part of the predetermined air conditioner 300 to correspond to a specific function of the air conditioner. In addition, a part to be enlarged may be a part including the operation unit 310 of the air conditioner 300. However, the scope of the present invention is not limited to this.

The air conditioner 300 image may also be enlarged, reduced, or scrolled based on the user's preview screen touch. In this case, the enlargement / reduction of the image of the air conditioner 300 may be performed based on simple digital zooming, not lens zooming.

In FIG. 8, an image of the air conditioner 300 including the shutter 310 portion of the predetermined air conditioner 300 is provided as a preview screen in the mobile terminal 100 to correspond to a function of terminating the operation of the air conditioner 300. As shown in FIG. 8, the user moves the touch of the shutter portion down the screen.

Then, the mobile terminal 100 generates a control signal for terminating the operation of the air conditioner 300 and transmits it to the air conditioner 300. The air conditioner 300 receives a control signal and ends the air conditioner accordingly. Referring to FIG. 9, it can be seen that the shutter 310 of the air conditioner 300 at which the operation ends is lowered.

FIG. 10 illustrates that, according to the method of driving the mobile terminal illustrated in FIG. 8, a portion including the operation unit of the air conditioner 300 is automatically enlarged from the entire image of the air conditioner and is provided as a preview screen in the mobile terminal 100. As illustrated in FIG. 10, a user touches a portion of the air volume reduction operation unit on the preview screen.

Then, the mobile terminal 100 generates a signal for controlling the air conditioner 300 to reduce the air volume and transmits the signal to the air conditioner 300. Then, the air conditioner 300 may receive a control signal and perform an operation of reducing the air volume accordingly.

FIG. 11 illustrates controlling on / off of lighting according to the method of driving the mobile terminal illustrated in FIG. 8. First, an image of the lighting 400 obtained by the camera 121 of the mobile terminal 100 is provided through a preview screen. Then, the user touches the position corresponding to the lighting on the preview screen. Then, the mobile terminal 100 generates a control signal for turning off the light and transmits it to the light control device 500.

Contrary to that shown in FIG. 11, if the lighting is off, the mobile terminal 100 may generate a control signal for turning on the lighting based on the user's touch and transmit it to the lighting control device 500.

12 illustrates controlling on / off of the TV 200 according to the method of driving the mobile terminal illustrated in FIG. 8. First, an off TV 200 image obtained by the camera 121 of the mobile terminal 100 is provided through a preview screen (FIG. 12 (a)). Then, the user touches the location corresponding to the TV 200 on the preview screen. Then, the mobile terminal 100 generates a control signal for turning on the TV 200 and transmits it to the TV 200. Then, as shown in Fig. 12B, it is turned on in response to the control signal.

Contrary to that shown in FIG. 12, when the TV 200 is turned on, the mobile terminal 100 may generate a control signal for turning off the TV 200 based on the user's touch and transmit the control signal to the TV 200. have.

According to the mobile terminal driving method described above with reference to FIGS. 7 to 12, the user may operate the mobile terminal 100 based on a touch operation on the preview screen, such as air conditioner, TV 200, and lighting 1400. It can be used as a remote control.

13 is a flowchart illustrating another example of a method for driving a mobile terminal according to the present invention. Hereinafter, a method of driving the mobile terminal will be described with reference to necessary drawings.

First, the controller 180 controls the camera 121 to obtain an image of a specific place (S300). Then, the controller 180 displays the image of the specific place on the touch screen 151 as a preview screen (S310).

Then, a touch for a specific place in the preview screen is received (S320). Then, the control unit 180 generates a control signal for performing a predetermined function (that is, cleaning for a specific place) to correspond to the specific place, and then transmits the generated control signal to the robot cleaner ( S330).

Transmission of the generated control signal may be performed through a short range wireless communication link formed between the mobile terminal 100 and the robot cleaner. The robot cleaner receiving the control signal may perform a cleaning function for a specific place in response to the control signal. Location information for a specific place may be stored in advance.

14 shows an example of controlling the operation of the robot cleaner 600 according to the method of driving the mobile terminal illustrated in FIG. 13. 14A illustrates that a user touches a preview screen while a preview image for a specific place is displayed on the mobile terminal 100.

Then, the mobile terminal 100 generates a control signal for cleaning a specific place and transmits it to the robot cleaner 600. Referring to (b) of FIG. 14, the robot cleaner 600 may clean a specific place in response to a control signal.

In the above, according to the mobile terminal driving method described with reference to FIGS. 13 and 14, the user can wirelessly control the robot cleaner 600 based on the touch operation on the preview screen.

As described above, each of the mobile terminal driving methods according to the present invention is implemented in a program form that can be executed through various computer means and can be recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The programs recorded on the medium may be specially designed and configured for the present invention or may be known and usable by those skilled in computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine code such as that produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

As described above, although the present invention has been described by limited embodiments and drawings, the present invention is not limited to the above embodiments, and various modifications and variations from these descriptions will be made by those skilled in the art to which the present invention pertains. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims to be described later, but also by the claims and equivalents.

100: mobile terminal 110: wireless communication unit
120: A / V input 121: Camera
130: user input unit 140: sensing unit
150: output unit 151: display unit, touch screen
152: sound output unit 160: storage unit
170: interface unit 180: control unit
190: power supply unit 200; TV
300: air conditioning 400: lighting
500: lighting controller 600: robot cleaner

Claims (11)

A wireless communication unit that forms a short-range wireless communication link with other electronic devices;
A camera that acquires a graphic user interface (GUI) image of the other electronic device;
A touch screen displaying the GUI image in real time in a preview screen or selecting and displaying at least a portion of all data corresponding to the GUI image stored therein; And
When a touch is received through the preview screen,
Determine a location in the GUI of the other electronic device corresponding to the location where the touch is received on the preview screen,
A control signal for controlling the other electronic device is generated to perform a function such that the determined position is directly selected in the GUI of the other electronic device,
And a control unit that transmits the generated control signal to the other electronic device through the short-range wireless communication link. According to claim 1, The control unit,
A part of the GUI image is enlarged and displayed on the preview screen,
A mobile terminal characterized by performing an enlargement, reduction, and scrolling function on the preview screen based on a touch received through the preview screen. According to claim 2, The control unit,
And a part of the GUI image to be enlarged and displayed on the preview screen based on the focus of the camera. A wireless communication unit that forms a short-range wireless communication link with other electronic devices;
A camera that acquires an image of the other electronic device;
A touch screen displaying a preview screen by enlarging a specific predetermined portion to correspond to a specific function among all images of the other electronic devices; And
When a touch on a specific portion of the other electronic device is received on the preview screen, a short-range is generated by generating a control signal for controlling the other electronic device to perform a predetermined specific function to correspond to a specific part of the other electronic device. A mobile terminal comprising a control unit transmitting through a wireless communication link. The method of claim 4, wherein the control unit,
A part of the image of the other electronic device is enlarged and displayed on the preview screen,
A mobile terminal characterized by performing an enlargement, reduction, and scrolling function on the preview screen based on a touch received through the preview screen. The method of claim 5, wherein the control unit,
And a part of the image of the other electronic device to be enlarged and displayed on the preview screen based on the focus of the camera. The method of claim 5, wherein the control unit,
A mobile terminal characterized in that a portion including an operation unit of the other electronic device is automatically enlarged and provided as a preview screen among the images of the other electronic device. The method of claim 7, wherein the control unit,
When a touch to the operation portion of the other electronic device displayed on the preview screen is received, a control signal for controlling the other electronic device is generated to perform the same function as that directly selected by the operation unit in the other electronic device,
And a control unit that transmits the generated control signal to the other electronic device through the short-range wireless communication link. A wireless communication unit forming a short-range wireless communication link with the robot cleaner;
A camera that acquires an image of a specific place;
A touch screen that displays a preview screen of the video of the specific place; And
When a touch for the specific place is received on the preview screen, a control signal for controlling the robot cleaner is generated to perform cleaning for the specific place,
And a control unit transmitting the generated control signal to the robot cleaner through the short-range wireless communication link. The method of claim 9, wherein the control unit,
A part of the image of the specific place is enlarged and displayed on the preview screen,
A mobile terminal characterized by performing an enlargement, reduction, and scrolling function on the preview screen based on a touch received through the preview screen. The method of claim 10, wherein the control unit,
And determining a part of the image of the specific place to be enlarged and displayed on the preview screen based on the focus of the camera.

Images