KR20110026350A - A remote conroll device for electronic devices and a remote controlling method using it - Google Patents

Abstract

PURPOSE: A remote control device for electronic devices and a remote controlling method using the same are provided to receive and display various kinds of information through interactive communication. CONSTITUTION: A camera(12) photographs an image for a target device, and a communication module(13) transfers a control command to the target device and communicates with the target device. A display unit(14) displays an actual image or virtual image. A control unit(11) synthesizes the actual image with the virtual image and displays the synthesized image.

Description

A REMOTE CONROLL DEVICE FOR ELECTRONIC DEVICES AND A REMOTE CONTROLLING METHOD USING IT}

The present invention relates to a remote control device and method.

More specifically, the present invention relates to a remote control device equipped with a display and a camera, and a control method using the same. The present invention relates to a remote control device capable of displaying an image, transmitting various commands according to a user's gesture, and receiving and displaying data, and a control method using the same.

Conventionally, remote controllers for controlling home appliances such as TVs, DVD players, game consoles, and audio exist. These conventional remote controllers use infrared light to transmit binary codes to home appliances, thereby controlling simple menus such as volume, channel, and the like. As the type and number of home appliances have increased, an integrated remote controller capable of controlling a plurality of these home appliances has been released. However, these remote controllers also offer little convenience other than integrating the commands for various home appliances into a single device.

There is a need for a new remote control device that can provide a user with a more intuitive and versatile control method.

It is an object of the present invention to provide a remote control apparatus and method using an intuitive user interface to a user.

An object of the present invention is to provide a remote control device and method capable of receiving and displaying various information through two-way communication with a target device to be controlled.

An object of the present invention is to provide a remote control device and method that can intuitively control the target device using a user gesture.

Remote control apparatus according to an embodiment of the present invention, the camera for taking an image of the target device; A communication module for transmitting a control command to the target device and communicating with the target device; A display for displaying a photographed actual image or a virtual image associated with the target device; And a controller configured to generate a control signal for controlling the target device, generate a virtual image related to the target device, and display the synthesized image and the virtual image.

Remote control method according to an embodiment of the present invention, the step of taking an image of the target device; Generating a virtual image associated with the target device; Synthesizing the virtual image and the captured real image of the target device; And displaying the synthesized real image and the virtual image.

According to the present invention, an intuitive user interface can be provided to a user.

According to the present invention can provide a remote control device and method that can receive and display a variety of information through the two-way communication with the target device to control.

The present invention can provide a remote control device and method that can intuitively control the target devices using the user's gesture.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 shows a configuration of a remote control apparatus according to an embodiment of the present invention.

As shown in FIG. 1, the remote control apparatus 10 according to an exemplary embodiment of the present invention transmits a control command to the camera 12 and the target apparatus to capture an image of the target apparatus, and communicates with the target apparatus. A short range communication module 13, a display unit 14 displaying a photographed real image of the target device or a virtual image related thereto, a control signal for controlling the target device, and a virtual image associated with the target device And a controller 11 which controls to synthesize and display the real image and the virtual image.

The target device is an electronic device controlled by a remote control device according to the present invention, and may be a broadcast receiving device such as a TV or an image display device equipped with a display unit. In addition, according to an exemplary embodiment, the target device may be a DVD player, a Blu-ray player, a VCR, a PC, an MP3 player, an audio device, a DVix player, a game device, or an external device connected to the broadcast receiving device or a video display device.

The camera 12 captures an image of the target device and transfers the image to the controller 11. The camera may be a camera module mounted on a conventional mobile terminal such as a mobile phone.

The short range communication module 13 communicates with the target device, transmits a command of the remote control device 10, and receives data from the target device. The short range communication module 13 may use any communication module such as a Bluetooth module, an infrared communication module, a zigbee module, a NFC (Near Field Communication) module, an RF communication module, or a combination thereof.

The display unit 14 displays an image captured by the camera 12. According to an exemplary embodiment, an image obtained by synthesizing the actual image of the target device photographed by the camera 12 and the virtual image generated by the controller 11 under the control of the controller 11 is output.

The display unit 14 may be any display device capable of displaying information such as an LCD, an OLED, a VFD, an electrophoretic device, and the like.

In addition, the display unit 14 may be a touch screen on which a touch panel is mounted on the front surface. The touch panel may be a touch panel of any type well known, such as an electric resistive film type or a capacitive type.

When the controller 11 controls the operations of the above-described components, the controller 11 generates an image obtained by synthesizing the virtual image associated with the actual image captured by the camera 12, among others. As will be described later, the virtual image is an image related to the captured target device. When the target device is a TV, the virtual image may be a channel browsing image for a plurality of channels or an image including information of external devices connected to the TV. When the target device is an external device such as a DVD player, a Blu-ray player, a VCR, a PC, an MP3 player, an audio, a DVix player, a game machine, or the like, the target device may be an image including information on content that the external devices can provide.

In addition, according to the exemplary embodiment, a motion sensor for detecting a user's gesture applied to the remote control device 10 may be mounted inside the remote control device 10. As the motion sensor, any sensor capable of detecting the movement of the remote control device 10 such as an acceleration sensor, a gyro sensor, and a geomagnetic sensor may be used.

As described later, a gesture applied to the remote control apparatus 10 may be detected and various commands may be mapped to the gesture. By inputting a command by the gesture, a user may be provided with a more intuitive and easy user interface.

2A and 2B show an appearance of a remote control apparatus 10 according to an embodiment of the present invention. 2A is a perspective view including the front of the remote control device 10, and FIG. 2B is a perspective view including the back side. According to an embodiment, the front side of the remote control device 10 may face the user, and the rear side may face the target device to be controlled.

As shown in FIG. 2A, the display unit 14 is disposed in front of the remote control apparatus 10. The display unit 14 may be a touch screen.

Other control buttons may be disposed in the remaining portions except for the display unit 14. The control button may include a power button 21, a channel control button 24, a volume control button 25, a mute button 23, a previous channel button 22, and the like. In addition, the control button may include various buttons according to the type of the target device, and may vary according to embodiments. According to the exemplary embodiment, the touch screen may be used as the display unit 14, only one or two of the control buttons may be left, and the remaining buttons may be displayed on the touch screen.

Referring to FIG. 2B, the camera 12 is disposed at the rear of the remote control apparatus 10. The camera 12 may face the direction of photographing the target device during operation. Preferably, the direction in which the camera 12 and the display unit 14 face may be opposite to each other. That is, the camera 12 may face the target device to be controlled, and the display 14 may face the user.

According to the embodiment, the actuator 12 is connected to the camera 12 may be configured to be able to change the direction in the up and down 27 or left and right (28) direction.

3 and 4 illustrate a network configuration diagram between the remote control device 10 and an external device according to an embodiment of the present invention.

As shown in FIG. 3, the remote control apparatus 10 may directly communicate with one of the target devices, for example, the TV 31, and the TV 31 may communicate with other external devices 32 and 33 by wire or wirelessly. . Instead of the TV 31, a video display device without a tuner may be used.

In the present embodiment, the remote control device 10 communicates with the TV 31. The TV transmits a command to the external devices 32 and 33 or receives data from the external devices 32 and 33 according to the command received from the remote control device 10 and transmits the data to the remote control device 10. do. The short-range wireless communication module capable of communicating with the remote control device 10 should be mounted on the TV 31, and the external devices 32 and 33 may be equipped with a wired or wireless communication module with the TV 31.

The TV 31 or the image display device may receive and display content from the external devices 32 and 33. For example, the DVD player 32 plays the DVD title and transmits it to the TV 31, and the TV 31 displays the content included in the DVD title. In addition, when the game title is played back on the game machine 33, the game screen is displayed on the TV 31. FIG. When the external device is an MP3 player, only sound may be output through the TV 31.

According to one embodiment of the invention, as shown in Figure 4, the remote control device 10 may be configured to enable direct communication with not only the TV 31, but also other external devices (32, 33). . In this case, the external devices 32 and 33 should be equipped with a short range wireless communication module for communicating with the remote control device 10 in addition to the wired or wireless communication module with the TV 31.

As described above, the short range wireless communication module mounted on the remote control device 10, the TV 31, or the external devices 32 and 33 may be a Bluetooth module, an infrared communication module, a zigbee module, or a near field communication (NFC). Module), an RF communication module, or any short range wireless communication module.

As shown in FIGS. 3 and 4, the user 34 may direct the camera toward the target device to control the remote control device 10 and then control the target device in various ways.

In various embodiments described below, the user can change the operation mode by operating the remote control device 10, and embodiments described below can be performed when the remote control device 10 is in a specific operation mode. The specific operation mode may include a control mode, a menu operation mode, a screen synchronization mode, and the like.

According to one embodiment of the present invention, the content displayed on the target device to which the remote control device 10 is directed may be configured to be displayed on the display 14 of the remote control device 10. This operation may be performed when the remote control device is in a screen synchronization mode, for example.

3 or 4, when the remote control device 10 is turned on to the TV 31, the content currently being played on the TV 31 screen, that is, the screen being played on the TV broadcast program or the DVD player, is remote. It may be displayed on the screen of the control device 10 as it is.

When the tuner or TV is an IPTV or a broadband TV, the TV 31 may display a broadcast program received through an internet cable or may display content received from external devices 32 and 33.

When the remote control device 10 faces the TV 31, the control unit 11 of the remote control device 10 determines that the remote control device 10 is currently facing the TV 31 through the camera 12. As shown in FIG. 5, the control unit may receive the content currently being played from the TV 31 and display it on the display 14.

The TV 31 transmits the displayed image data to the remote control apparatus 10 using the above-mentioned short range communication module, and the remote control apparatus 10 displays the image data on the display 14.

Once the operation of displaying the content being played on the TV 31 on the remote control device 10 is started, there may be other input from the user, even if the remote control device 10 (or the camera 12) faces away. The content can continue to be received from the TV 31 until it is displayed.

According to an embodiment of the present invention, in addition to the actual image of the target device, a virtual image related thereto may be generated and displayed together with the actual image. This embodiment may be performed when the remote control device 10 is in the control mode.

6 illustrates a target device control method of a remote control apparatus according to another embodiment of the present invention.

Assuming that the remote control device 10 faces the TV 31, the camera 12 captures an image of the TV 31, and the TV is displayed on the display 14 of the remote control device 10.

In this state, when a user inputs a predetermined command, for example, a "view channel information command", as shown in FIG. 6, information of other channels other than the channel currently tuned and displayed on the TV is displayed on the left and right around the TV image 41. Can be displayed.

The virtual command may be input using a button mounted on the outside of the remote control apparatus 10. When the display unit 14 is a touch screen, the virtual command may display a virtual button for the user to input a command on the screen. It may be input using virtual buttons.

According to an embodiment of the present invention, the channel information is a virtual image generated by the controller 11. The channel information may be thumbnail images 42 and 43 of channels other than the current channel obtained by continuously scanning a plurality of channels (ie, channel browsing operations). Channel browsing can be performed by a variety of known methods.

The channel browsing operation may be performed after receiving a predetermined command from the user, or may perform channel browsing in advance to store channel information and display the same as shown in FIG. 6 after receiving the predetermined command from the user.

According to an exemplary embodiment, channel information displayed to the left and right of the TV image 41 may be displayed in the order of the channel numbers. For example, in FIG. 6, channel information is displayed in order of decreasing channel number toward the left side 42 of the TV image 41 on the basis of the current channel, and in order of decreasing channel number toward the right side 43. Channel information can be displayed.

For example, if the current channel is 10, the left 42 may display channel information in the order of CH9, CH8, CH7, ..., and the right 43 may display CH11, CH12, CH13, ... Channel information may be displayed in the order of.

In some embodiments, the channel information may be displayed in the up and down directions 44 and 45 of the TV image 41.

When the display 14 is a touch screen, when one of the channels 42 and 43 other than the current channel is touched while being displayed as shown in FIG. 6, the channel may be changed to the channel.

According to an embodiment of the present invention, the TV may be photographed and the actual image of the TV and information of an external device connected to the TV may be displayed as a virtual image. This embodiment may also be performed when the remote control device 10 is in a control mode.

3 or 4, when the remote control device 10 faces the TV 31, the controller of the remote control device 10 is connected to the external devices 32 connected to the TV 31 from the TV 31. 33 may be received and displayed as a virtual image as shown in FIG. 7. In this case, the images 52 and 53 of the external devices can be displayed by unfolding up and down the actual image 41 of the TV 31. In this way, the user can intuitively grasp which external devices are connected to the TV 31.

According to an embodiment, the images 52 and 53 of the external devices may be unfolded and displayed to the left and right of the actual image 41 of the TV.

When the display 14 is a touch screen, when one of the external devices 52 and 53 is touched in the state as shown in FIG. 7, the output of the touched external device may be displayed on the TV 31.

Further, according to the embodiment, in combination with the embodiment of Figs. 6 and 7, when the remote control device 10 faces the TV 31, the virtual image representing the channel information of the TV 31 is an image of the TV. The left and right displays of 41 may be displayed, and information of an external device connected to the TV 31 may be displayed up and down. The top, bottom, left and right of the TV image 41 is merely an embodiment, and the channel information or the image of the external device may be displayed at any position around the actual image 41 of the TV.

According to an embodiment of the present invention, when the remote control device 10 faces an external device, information on the external device may be generated and displayed as a virtual image. This embodiment may also be performed when the remote control device 10 is in a control mode.

As shown in FIG. 8, assume that the remote control device 10 faces an external device, such as a DVD player 32. The camera of the remote control apparatus 10 captures an image of the DVD player 32, and the controller analyzes the captured image to determine that the remote control apparatus 10 is facing the DVD player 32.

The controller of the remote control apparatus 10 may receive a list of playable contents stored in the DVD player 32 from the DVD player 32 and display the virtual image 61 around the image 62 of the DVD player. . For example, when the mermaid and the kung fu panda are included in the DVD title inserted into the DVD player 32, the mermaid and the kung fu panda title may be received from the DVD player 32 to generate a virtual image. . Then, as shown in FIG. 9, the virtual image 61 may be displayed around the actual image 62 of the DVD player. From the image displayed in this way, the user can easily grasp the list of contents playable on the DVD player 31.

In addition, by displaying the virtual image 61 of the reproducible content list around the DVD player image 62, the user can intuitively grasp the contents.

The virtual image 61 of the content list may include text such as titles of contents, a cover letter image of a DVD title, a character representing contents, or an advertisement image.

According to an embodiment of the present invention, the remote control device may be used to control a menu of a target device to which the remote control device is directed. This embodiment can be performed when the remote control device 10 is in the menu operation mode, for example.

3 or 4, assume that the remote control device 10 faces the TV 31, that is, the camera of the remote control device 10 faces the TV 31. As shown in FIG. With the remote control device 10 facing the TV 31, as shown in FIG. 10, when the user presses a button 65 mounted on one side of the remote control device 10, the display of the remote control device 10 ( 14 may display a menu 66 for controlling the TV 31.

The control menu of the TV 31 may include screen adjustment, reservation recording, volume adjustment, time adjustment, and the like.

The above operation may be similarly performed on the TV 31 as well as other external devices 32 and 33.

The button 65 of the remote control device 10 may allow the user to place the remote control device 10 at the position of a shooting button of a conventional camera. The user can display a menu of the target device being photographed by pressing the button 65 in the same way as pressing the photographing button of the camera while facing the remote control device 10, similarly to photographing the target device with the camera. Can be.

With the menu 66 displayed, the user can touch the display to select the menu and control the menu of the TV 31.

According to an embodiment of the present invention, when the locations of the target devices are input to the remote control device 10, the target devices may be controlled even when the camera of the remote control device 10 does not face the target device. In this case, the motion sensor is mounted on the remote control device 10.

As shown in FIG. 11, the user 34 causes the camera of the remote control device 10 to scan the target devices, namely the TV 31 and the external devices 32, 33. The relative positions and images of the scanned target devices are stored in a memory in the remote control device 10. The remote control device 10 may display an image of the target devices and the location thereof on the display.

In this state, when the user moves the remote control device 10 side to side, the motion sensor in the remote control device 10 detects the movement and displays it on the display of the remote control device 10.

That is, even if the remote control device 10 does not face the target device to be controlled, the movement sensor can detect a movement applied by the user to the remote control device 10 using a movement sensor, and is displayed according to the movement. You can change the device. According to the displayed target device, the same operations as those of the above-described embodiments may be implemented.

In the present embodiment, once the remote control apparatus 10 photographs the target apparatuses and stores the relative positions and images of the target apparatuses, the remote control apparatus 10 does not face the target apparatus, or the remote control apparatus 10 of the remote control apparatus 10. Even when the camera is disabled, the operations of the aforementioned various embodiments can be implemented.

According to this embodiment, the user does not need to point the target device to control the remote control device in order to control the target devices. The remote control device only needs to be within a wireless communication range with the target devices to be controlled.

As the remote control apparatus 10 according to the embodiment of the present invention, any electronic device equipped with a camera, a display, and a short-range communication module may be used as long as appropriate software is installed. For example, when appropriate software is attached to a mobile terminal such as a mobile phone, it can be used as the remote control device 10. In particular, recently released touch screen phone equipped with a touch screen is suitable for use as a remote control device (10).

12 is a block diagram of a mobile terminal 100 that can be used as the remote control apparatus 10 according to an embodiment of the present invention. The mobile terminal 100 may include not only a mobile phone but also a smart phone, an MP3 player, a portable multimedia player (PMP), a digital broadcasting terminal, a notebook computer, a navigation device, and the like.

The illustrated mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, The interface unit 170 may include a controller 180, a power supply unit 190, and the like. 1 shows a mobile terminal having various components. However, not all illustrated components are required. The mobile terminal may be implemented by more components than the illustrated components, or the mobile terminal may be implemented by fewer components.

Hereinafter, the components will be described in turn.

The wireless communication unit 110 may include one or more components for wireless communication between the mobile terminal 100 and the wireless communication system or wireless communication between the mobile terminal 100 and a network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcast signals using various broadcasting systems, and in particular, digital multimedia broadcasting-terrestrial (DMB-T), digital multimedia broadcasting-satellite (DMB-S), and media forward link (MediaFLO). Digital broadcast signals can be received using digital broadcasting systems such as only), digital video broadcast-handheld (DVB-H), integrated services digital broadcast-terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcasting systems that provide broadcasting signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

In addition, the mobile communication module 112 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The wireless Internet module 113 is a module for wireless Internet access, and the wireless Internet module 113 can be built in or externally.

The short range communication module 114 refers to a module for short range communication. The short range communication module 114 may be used as the short range communication module 13 of FIG. 1.

In addition, the location information module 115 is a module for checking or obtaining the location of the mobile terminal. One example is the Global Position System (GPS) module. The GPS module receives position information from a plurality of satellites. Here, the location information may include coordinate information represented by latitude and longitude. For example, the GPS module can measure the exact time and distance from three or more satellites and accurately calculate the current position by triangulating three different distances. A method of obtaining distance and time information from three satellites and correcting the error with one satellite may be used. In particular, the GPS module can obtain not only the location of latitude, longitude, and altitude but also accurate time together with three-dimensional speed information from the location information received from the satellite.

Meanwhile, the A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes an image frame such as a still image or a moving image obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be provided according to the configuration aspect of the terminal. The camera 121 may be used as the camera 12 of FIG. 1.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. The processed voice data may be converted into a form transmittable to the mobile communication base station through the mobile communication module 112 and output in the call mode. The microphone 122 may implement various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for the user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like. In FIG. 1, the user input unit 130 may be a touch screen in which a touch panel is mounted on the display unit 14.

The sensing unit 140 detects a current state of the mobile terminal 100 such as an open / closed state of the mobile terminal 100, a location of the mobile terminal 100, presence or absence of a user contact, orientation of the mobile terminal, acceleration / deceleration of the mobile terminal, and the like. To generate a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, the power supply unit 190 is responsible for sensing functions related to whether or not the power supply unit 190, the interface unit 170 is coupled to the external device.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, wired / wireless headset ports, external charger ports, wired / wireless data ports, memory card ports, ports for connecting devices with identification modules, audio input / output (I / O) ports, Video I / O (Input / Output) port, earphone port, etc. may be included.

Here, the identification module is a chip that stores various types of information for authenticating the use authority of the mobile terminal 100, and includes a user identification module (UIM) and a subscriber identify module (SIM). ), A Universal Subscriber Identity Module (“USIM”), and the like. In addition, the device equipped with the identification module (hereinafter, 'identification device') may be manufactured in the form of a smart card. Accordingly, the identification device can be connected to the terminal 100 through the port. The interface unit 170 receives data from an external device or receives power and transmits the data to each component inside the mobile terminal 100 or transmits the data inside the mobile terminal 100 to an external device.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 may include a display unit 151, a sound output module 152, an alarm unit 153, and the like.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in a call mode, a user interface (UI) or a graphic user interface (GUI) related to a call is displayed. When the mobile terminal 100 is in a video call mode or a photographing mode, the mobile terminal 100 displays a photographed and / or received image, a UI, or a GUI. The display unit 151 may be the display unit 14 of FIG. 1.

Meanwhile, as described above, when the display unit 13 and the touch pad form a mutual layer structure to form a touch screen, the display unit 151 may be used as an input device in addition to the output device. The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, or a three-dimensional display. 3D display). In addition, two or more display units 151 may exist according to the implementation form of the mobile terminal 100. For example, an external display unit (not shown) and an internal display unit (not shown) may be simultaneously provided in the mobile terminal 100.

The sound output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. In addition, the sound output module 152 outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying occurrence of an event of the mobile terminal 100. Examples of events occurring in the mobile terminal include call signal reception, message reception, and key signal input. The alarm unit 153 may output a signal for notifying occurrence of an event in a form other than an audio signal or a video signal. For example, the signal may be output in the form of vibration. When a call signal is received or a message is received, the alarm unit 153 may output a vibration to inform this. Alternatively, when a key signal is input, the alarm unit 153 may output a vibration in response to the key signal input. Through the vibration output as described above, the user can recognize the occurrence of the event. Of course, the signal for notification of event occurrence may be output through the display unit 151 or the voice output module 152.

The memory 160 may store a program for processing and controlling the controller 180 and may provide a function for temporarily storing input / output data (for example, a phone book, a message, a still image, a video, etc.). It can also be done.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), RAM Random Access Memory (RAM) Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM) magnetic memory, It may include at least one type of storage medium of the optical disk. In addition, the mobile terminal 100 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The controller 180 typically controls the overall operation of the mobile terminal. For example, perform related control and processing for voice calls, data communications, video calls, and the like. In addition, the control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the controller 180 or may be implemented separately from the controller 180. The controller 180 may be the controller 11 of FIG. 1.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

The various embodiments described herein may be implemented in a computer readable recording medium using, for example, software, hardware or a combination thereof.

According to a hardware implementation, embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing the functions. It may be implemented by the controller 180.

In a software implementation, embodiments such as procedures or functions may be implemented with separate software modules to perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. In addition, the software codes may be stored in the memory 160 and executed by the control unit 180. [

1 shows a configuration of a remote control apparatus according to an embodiment of the present invention.

2A and 2B show an appearance of a remote control apparatus 10 according to an embodiment of the present invention.

3 and 4 illustrate a network configuration diagram between the remote control device 10 and an external device according to an embodiment of the present invention.

5 to 7 show a TV control method according to an embodiment of the present invention.

8 and 9 illustrate an external device control method according to an embodiment of the present invention.

10 illustrates a target device control method according to an embodiment of the present invention.

11 is a diagram for a method of controlling a remote control device without facing a target device according to one embodiment of the present invention.

12 illustrates a configuration of a mobile terminal that can be used as a remote control device according to an embodiment of the present invention.

Claims (29)

A camera for taking an image of the target device; A communication module for transmitting a control command to the target device and communicating with the target device; A display for displaying a photographed actual image or a virtual image associated with the target device; And And a controller configured to generate a control signal for controlling the target device, generate a virtual image related to the target device, and display the synthesized image and the virtual image. The method of claim 1, And the communication module receives content information provided by the target device from the target device. The method of claim 1, And the target device is a broadcast receiving device. The method of claim 1, The target device is a broadcast receiving device, and the virtual image is a remote control device including channel information of the broadcast receiving device. The method of claim 1, And the target device is an image display device, and the virtual image includes information of an external device connected to the image display device. The method of claim 1, And the virtual image includes an external device and content information that can be provided by the external device. The method of claim 1, And the display is a touch screen. The method of claim 1, The virtual image is displayed on the periphery of the image of the target device. The method of claim 1, And the camera and the display face different surfaces. The method of claim 1, The controller is configured to control to periodically synthesize and display the image of the target device and the virtual image. The method of claim 1, Further comprising a memory for storing the target device and its associated image, And the control unit controls to output an image of a target device stored in the memory or an image related thereto when the remote control device is not in a control mode. The method of claim 11, When not in the control mode, the remote control device for receiving and displaying the same image as the image that the target device is output. The method of claim 1, In the menu operation mode of the target device, a menu displayed on the target device is operatively displayed on the display. The method of claim 13, The display is a touch screen, and the menu is operable by the touch screen. The method of claim 13, And the menu is one of screen adjustment, scheduled recording, volume adjustment, and time adjustment of the target device. The method of claim 4, wherein The channel information is displayed on the left and right or up and down of the channel currently displayed information of each channel is displayed on the remote control device. The method of claim 16, The channel information is displayed in the order of the channel number to the left and right or up and down of the currently displayed channel. The method of claim 1, The remote control device, wherein the information on the external device connected to the target device is displayed to the left and right or up and down of the target device. The method of claim 1, A remote control device further comprising a motion sensor. The method of claim 1, A remote control device further comprising a mobile communication module. The method of claim 13, When the zoom-in operation command of the camera is input, the menu enters the menu operation mode and the menu is displayed. The method of claim 1, A remote control device that displays a virtual image with the target device or the external device, regardless of whether the target device faces the target device or the external device. Photographing an image of the target device; Generating a virtual image associated with the target device; Synthesizing the virtual image and the captured real image of the target device; And Displaying the synthesized real image and the virtual image. 24. The method of claim 23, Receiving information related to the target device via a communication module. 24. The method of claim 23, The target device is a broadcast receiving device, and the virtual image remote control method including channel information of the broadcast receiving device. 24. The method of claim 23, Further comprising a memory for storing the image of the target device and the image associated with it, And when the remote control apparatus is not in a control mode, controlling the controller to output an image of the target apparatus stored in the memory or an image related thereto. The method of claim 26, When not in the control mode, the remote control method further comprises the step of receiving and displaying the same image as the image being output. 24. The method of claim 23, And in a menu operation mode of the target device, operatively displaying a menu displayed on the target device on the display. The method of claim 28, Wherein the display is a touch screen and the menu is operable by the touch screen.

Images