KR101608990B1 - A remote conroll device for electronic devices and a remote controlling method using it - Google Patents

Abstract

The present invention relates to a remote control device equipped with a display and a camera, and a control method using the remote control device. The remote control device captures a target device to be controlled by a camera, displays an actual image of the target device and additional information related to the target device as a virtual image , A remote control device capable of transmitting various commands according to a user's gesture, receiving and displaying data, and a control method using the same.

Remote control device, remote control, camera, external device, TV, motion, virtual image

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a remote control apparatus for an electronic apparatus and a remote control method using the remote control apparatus.

The present invention relates to a remote control apparatus and method.

More particularly, the present invention relates to a remote control device equipped with a display and a camera, and a control method using the remote control, and more particularly, To a remote control device capable of displaying various kinds of commands according to a gesture of a user, receiving and displaying data, and a control method using the same.

Conventionally, there is a remote controller for controlling household appliances such as a TV, a DVD player, a game machine, and audio. These conventional remote controllers transmit binary codes to household appliances using infrared rays, and thus can control simple menus such as volume and channel. As the number and type of home appliances have increased, an integrated remote control has been released that can control a single remote control with many household appliances. However, these remote controllers do not offer much convenience except to integrate commands for various consumer products into one device.

There is a need for a new remote control device that is more intuitive to the user and capable of providing various control methods.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a remote control apparatus and method using an intuitive user interface.

An object of the present invention is to provide a remote control device and method capable of receiving and displaying various information through bidirectional communication with a target device to be controlled.

It is an object of the present invention to provide a remote control apparatus and method that can intuitively control target devices using a user's gesture.

A remote control device according to an embodiment of the present invention includes: a camera that captures an image of a target device; A communication module for transmitting a control command to the target device and communicating with the target device; A display that displays a photographed actual image of the target device or a virtual image associated therewith; And a control unit for generating a control signal for controlling the target apparatus, generating a virtual image associated with the target apparatus, and controlling the display unit to synthesize and display the actual image and the virtual image.

A remote control method according to an embodiment of the present invention includes: capturing an image of a target device; Generating a virtual image associated with the target device; Synthesizing the virtual image with the photographed actual image of the target device; And displaying the synthesized real image and the virtual image.

According to the present invention, an intuitive user interface can be provided to a user.

The present invention can provide a remote control apparatus and method capable of receiving and displaying various information through bidirectional communication with a target apparatus to be controlled.

The present invention can provide a remote control apparatus and method that can intuitively control target devices using a user's gesture.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 shows a configuration of a remote control apparatus according to an embodiment of the present invention.

1, a remote control apparatus 10 according to an embodiment of the present invention includes a camera 12 for photographing an image of a target apparatus, a control unit 12 for transmitting a control command to the target apparatus, A display section (14) for displaying a photographed actual image of the target device or a virtual image associated therewith, a control section for generating a control signal for controlling the target device, and a virtual image And a control unit (11) for controlling the display unit to synthesize and display the real image and the virtual image.

The target device may be an electronic device controlled by the remote control device according to the present invention, such as a TV receiver, or a video display device equipped with a display unit. The target device may be an external device connected to the broadcasting receiver or the video display device such as a DVD player, a Blu-ray player, a VCR, a PC, an MP3 player, an audio player, a DVix player and a game machine.

The camera 12 captures an image of the target device and transfers it to the control unit 11. [ The camera may be a camera module mounted on a conventional mobile terminal such as a mobile phone.

The short-range communication module 13 communicates with the target device, conveys the command of the remote control device 10, and receives data from the target device. The short-range communication module 13 may be any communication module such as a Bluetooth module, an infrared communication module, a zigbee module, an NFC (Near Field Communication) module, an RF communication module, or a combination thereof.

The display unit 14 displays an image taken by the camera 12. [ According to the embodiment, the control unit 11 outputs the combined image of the actual image of the target device photographed by the camera 12 and the virtual image generated by the control unit 11 under the control of the control unit 11. [

The display unit 14 may be any display device capable of displaying information such as an LCD, an OLED, a VFD, and an electrophoresis apparatus.

The display unit 14 may be a touch screen having a touch panel mounted on the front surface thereof. The touch panel may be any known well-known touch panel such as an electric resistance film type or a capacitive type.

When the control unit 11 controls the operation of the above-described components, the control unit 11 generates an image in which a virtual image related to the actual image captured by the camera 12 is synthesized. As will be described later, the virtual image may be an image related to the photographed target device, or an image including information of a channel browsing image for a plurality of channels or information of external devices connected to the TV if the target device is a TV. In the case where the target device is an external device such as a DVD player, a Blu-ray player, a VCR, a PC, an MP3 player, an audio player, a DVix player, or a game machine, the target device may be an image including information on contents that the external devices can provide.

Further, according to the embodiment, a motion sensor for detecting a user's gesture applied to the remote control device 10 may be mounted inside the remote control device 10. [ As the motion sensor, any sensor capable of detecting the motion of the remote control device 10 such as an acceleration sensor, a gyro sensor, a geomagnetic sensor, or the like can be used.

As will be described later, the gesture applied to the remote control apparatus 10 can be detected and various commands can be mapped to the gesture. By inputting the command by the gesture, a more intuitive and easy user interface can be provided to the user.

2A and 2B show an appearance of a remote control apparatus 10 according to an embodiment of the present invention. FIG. 2A is a perspective view including the front face of the remote control device 10, and FIG. 2B is a perspective view including a rear face. According to an embodiment, the front of the remote control device 10 can face the user, and the backside can face the target device to be controlled.

As shown in Fig. 2A, a display portion 14 is disposed on the front surface of the remote control device 10. [ The display unit 14 may be a touch screen.

Other control buttons may be disposed in the remaining portions except for the display portion 14. The control button may include a power button 21, a channel adjustment button 24, a volume adjustment button 25, a mute button 23, a previous channel button 22, and the like. In addition, the control button may include various buttons depending on the type of the target device, and may vary according to the embodiment. In some embodiments, the display 14 may use a touch screen, leaving only one or two of the control buttons, and displaying the remaining buttons on the touch screen.

Referring to FIG. 2B, a camera 12 is disposed on the rear surface of the remote control device 10. [ The camera 12 may be oriented in the direction of imaging the target device in operation. Preferably, the direction in which the camera 12 and the display unit 14 are oriented may be opposite to each other. That is, the camera 12 faces the target device to be controlled, and the display unit 14 faces the user.

According to the embodiment, the camera 12 may be structured such that an actuator is connected to the camera 12 so that the camera 12 can be changed in the direction of the up and down 27 or the left and right 28 directions.

3 and 4 show a network configuration diagram between the remote control apparatus 10 and an external apparatus according to an embodiment of the present invention.

3, the remote control apparatus 10 can directly communicate with one of the target apparatuses, for example, the TV 31, and the TV 31 can communicate with other external apparatuses 32 and 33 either in a wired or wireless manner . A video display device without a tuner may be used instead of the TV 31. [

In the present embodiment, the remote control device 10 communicates with the TV 31. [ The TV transmits a command to the external devices 32 and 33 or receives data from the external devices 32 and 33 and transmits the command to the remote control device 10 in accordance with the command received from the remote control device 10 do. A local wireless communication module capable of communicating with the remote control device 10 should be mounted on the TV 31 and a wired or wireless communication module with the TV 31 may be mounted on the external devices 32 and 33.

The TV 31 or the video display device can receive and display the content from the external devices 32 and 33. [ For example, the DVD player 32 reproduces the DVD title and transmits it to the TV 31, and the content included in the DVD title is displayed on the TV 31. When a game title is played back on the game machine 33, the game screen is displayed on the TV 31. [ If the external device is an MP3 player, only the sound may be outputted through the TV 31. [

4, the remote control device 10 can be configured to be capable of directly communicating with not only the TV 31 but also other external devices 32 and 33 . In this case, the external devices 32 and 33 must be equipped with a local wireless communication module for communicating with the remote controller 10 in addition to the wired or wireless communication module with the TV 31.

As described above, the short-range wireless communication module mounted on the remote control device 10, the TV 31, or the external devices 32 and 33 may be a Bluetooth module, an infrared communication module, a zigbee module, an NFC ) Module, an RF communication module, and the like.

As shown in FIGS. 3 and 4, the user 34 may direct the remote control device 10 toward the target device to be controlled, and then control the target device in various ways.

In various embodiments to be described later, the user can operate the remote control device 10 to change the operation mode, and the embodiments described below can be performed when the remote control device 10 is in the specific operation mode. The specific operation mode may include a control mode, a menu operation mode, a screen synchronization mode, and the like.

According to an embodiment of the present invention, the content displayed on the target device to which the remote control device 10 is directed can be configured to be displayed on the display 14 of the remote control device 10. [ Such an operation may be performed when the remote control device is in a screen synchronization mode, for example.

3 or 4, when the remote control device 10 is turned on and faces the TV 31, the content currently being reproduced on the TV 31 screen, that is, the TV program or the screen reproduced by the DVD player, It can be displayed on the screen of the control device 10 as it is.

When the tuner or the TV is an IPTV or a broadband TV, the TV 31 may display the broadcast program received via the Internet cable or may display the content received from the external devices 32 and 33.

When the remote control device 10 faces the TV 31, the control unit 11 of the remote control device 10 determines that the remote control device 10 is currently pointing to the TV 31 through the camera 12 , As shown in Fig. 5, the content currently being reproduced is received from the TV 31 and displayed on the display 14.

The TV 31 transmits the displayed image data to the remote control device 10 using the above-described short distance communication module, and the remote control device 10 displays the image data on the display 14.

Once the operation of displaying the content being reproduced on the TV 31 to the remote control device 10 is started, even if the remote control device 10 (or the camera 12) is pointing elsewhere, The content can be continuously received from the TV 31 and displayed.

According to one embodiment of the present invention, in addition to the actual image of the target device, a virtual image associated therewith can be generated and displayed together with the actual image. This embodiment can be performed when the remote control apparatus 10 is in the control mode.

6 shows a method of controlling a target device of a remote control device according to another embodiment of the present invention.

Assuming that the remote control device 10 is facing the TV 31, the camera 12 takes an image of the TV 31 and the TV is displayed on the display 14 of the remote control device 10.

In this state, when the user inputs a predetermined command, for example, "channel information view command ", information of channels other than the channel currently tuned and displayed on the TV is displayed on the left and right around the TV image 41 Can be displayed.

The virtual command may be input using a button mounted on the outside of the remote control apparatus 10 or may be displayed on the screen when the display unit 14 is a touch screen by displaying a virtual button for the user to input a command, Or may be input using virtual buttons.

According to an embodiment of the present invention, the channel information is a virtual image generated by the control unit 11. The channel information may be thumbnail images 42 and 43 of channels other than the current channel obtained by continuously scanning a plurality of channels (i.e., a channel browsing operation). Channel browsing can be performed by various known methods.

The channel browsing operation may be performed after receiving a predetermined command from the user, or may be performed by previously performing channel browsing to store the channel information, and may display a message as shown in FIG. 6 after receiving a predetermined command from the user.

According to the embodiment, channel information displayed on the left and right of the TV image 41 may be displayed in the order of channel numbers. For example, in FIG. 6, the channel information is displayed in decreasing order of the channel number toward the left side (42) of the TV image 41 on the basis of the current channel, and the channel number is decremented toward the right side (43) Channel information can be displayed.

For example, if the current channel is 10 times, the left side 42 can display channel information in the order of CH9, CH8, CH7, ..., and the right side 43 can display CH11, CH12, CH13, ... The channel information can be displayed in this order.

Further, according to the embodiment, the channel information can be displayed in the up and down directions 44 and 45 of the TV image 41.

When the display 14 is a touch screen, if one of the channels 42 and 43 other than the current channel is touched in a state shown in FIG. 6, the channel may be changed to the channel.

According to an embodiment of the present invention, a TV can be photographed, and an actual image of a TV and information of an external device connected to the TV can be displayed as a virtual image. This embodiment can also be performed when the remote control apparatus 10 is in the control mode.

3 or 4, when the remote control device 10 faces the TV 31, the control unit of the remote control device 10 controls the operation of the external devices 32, 33, and display it as a virtual image as shown in FIG. At this time, the images (52, 53) of the external devices can be displayed by spreading up and down the actual image (41) of the TV (31). According to this method, the user can intuitively understand what external devices are connected to the TV 31. [

In some embodiments, the images 52,53 of the external devices may be displayed on the left and right sides of the actual image 41 of the TV.

When the display 14 is a touch screen, the output of the touched external device may be displayed on the TV 31 by touching one of the external devices 52 and 53 in the state of FIG.

6 and 7, when the remote control device 10 is directed to the TV 31, the virtual image representing the channel information of the TV 31 corresponds to the image of the TV 31. In other words, And the information of the external device connected to the TV 31 may be displayed on the upper and lower sides. The TV image 41 is only an embodiment, and the channel information or the image of the external device can be displayed at an arbitrary position around the actual image 41 of the TV.

According to an embodiment of the present invention, when the remote controller 10 is pointing to an external device, information of the external device can be generated and displayed as a virtual image. This embodiment can also be performed when the remote control apparatus 10 is in the control mode.

As shown in Fig. 8, it is assumed that the remote control device 10 is directed to an external device, for example, the DVD player 32. [ The camera of the remote control device 10 photographs the image of the DVD player 32 and the control unit analyzes the photographed image to determine that the remote control device 10 is heading toward the DVD player 32. [

The control unit of the remote control apparatus 10 can receive the list of reproducible contents stored in the DVD player 32 from the DVD player 32 and display it in the virtual image 61 around the image 62 of the DVD player . For example, if a DVD title inserted in the DVD player 32 includes a mermaid and a kung fu panda, a title of the mermaid and a kung fu panda may be received from the DVD player 32 to generate a virtual image . Then, the virtual image 61 can be displayed around the actual image 62 of the DVD player as shown in FIG. From the image displayed in this manner, the user can easily grasp the list of the content reproducible on the DVD player 31. [

In addition, by displaying the virtual image 61 of the list of reproducible contents around the DVD player image 62, the user can grasp the contents intuitively.

The virtual image 61 of the content list may include text such as the title of the contents, a cover image of the DVD title, a character representing contents, or an advertisement image.

According to one embodiment of the present invention, a remote control device can be used to control the menu of the target device to which the remote control device is directed. This embodiment can be performed when the remote control device 10 is in a menu operation mode, for example.

Assuming that the remote control device 10 is directed to the TV 31, that is, the camera of the remote control device 10 is facing the TV 31, as shown in Fig. 3 or Fig. 10, when the user presses a button 65 mounted on one side of the remote control device 10, the display (not shown) of the remote control device 10 14 can display a menu 66 for controlling the TV 31. [

The control menu of the TV 31 may include screen adjustment, scheduled recording, volume adjustment, time adjustment, and the like.

The above operation can be performed not only for the TV 31 but also for other external devices 32 and 33 as well.

The button 65 of the remote control device 10 can be placed at the position of the photographing button of the normal camera by the user to the remote control device 10. [ Similarly to the case where the user is shooting the target device with the camera, in the state of facing the remote control device 10, when the button 65 is pressed in the same manner as pressing the photographing button of the camera, the menu of the target device being photographed is displayed .

With the menu 66 displayed, the user can touch the display to select the menu and control the menu of the TV 31. [

According to one embodiment of the present invention, by inputting the position of the target devices to the remote control device 10, the camera of the remote control device 10 can control the target devices without facing the target device. In this case, a motion sensor is mounted on the remote control device 10.

11, the user 34 causes the camera of the remote control device 10 to scan the target devices, i.e., the TV 31 and the external devices 32 and 33. [ The relative positions and images of the scanned target devices are stored in the memory in the remote control device 10. [ The remote control device 10 can display an image of the target devices and its position on the display.

In this state, when the user moves the remote control device 10 from side to side, the motion sensor in the remote control device 10 detects the movement and displays it on the display of the remote control device 10.

That is, even if the remote control device 10 is not aimed at the target device to be controlled, the movement sensor can be used to detect a movement applied by the user to the remote control device 10, You can change the device. Depending on the displayed target device, the same operations as those of the above-described embodiments can be implemented.

The present embodiment is advantageous in that when the remote control device 10 does not face the target device or after the remote control device 10 has taken a picture of the target devices, The operation of the various embodiments described above can be implemented even when the camera is disabled.

According to the present embodiment, the user does not need to direct the target device to control the remote control device in order to control the target devices. However, it is only necessary that the remote control device is within a wireless communicable range with the target devices to be controlled.

The remote control device 10 according to the embodiment of the present invention can be any electronic device equipped with a camera, a display, and a local communication module as long as it is equipped with appropriate software. For example, when appropriate software is installed in a mobile terminal such as a mobile phone, the remote control device 10 can be used. Especially, recently, the touch screen phone equipped with the touch screen is suitable for use as the remote control device 10.

12 shows a configuration diagram of a mobile terminal 100 that can be used as a remote control device 10 according to an embodiment of the present invention. The mobile terminal 100 may include not only a mobile phone but also a smart phone, an MP3 player, a portable multimedia player (PMP), a digital broadcasting terminal, a notebook computer, navigation and the like.

The mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, An interface unit 170, a control unit 180, a power supply unit 190, and the like. 1 shows a mobile terminal having various components. However, not all illustrated components are required. A mobile terminal may be implemented by more components than the illustrated components, or a mobile terminal may be implemented by fewer components.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more components that allow wireless communication between the mobile terminal 100 and the wireless communication system or wireless communication between the mobile terminal 100 and a network on which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcasting systems that provide broadcasting signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

In addition, the mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data depending on a voice call signal, a video call signal, or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and the wireless Internet module 113 can be built in or externally.

The short-range communication module 114 refers to a module for short-range communication. The short range communication module 114 may be used as the short range communication module 13 of FIG.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. One example is the GPS (Global Position System) module. The GPS module receives position information from a plurality of satellites. Here, the location information may include coordinate information indicated by latitude and longitude. For example, the GPS module can accurately calculate the current location according to the triangulation method by measuring the precise time and distance from three or more satellites and measuring three different distances. A method of obtaining distance and time information from three satellites and correcting the error by one satellite may be used. In particular, the GPS module can acquire latitude, longitude and altitude as well as three-dimensional velocity information and accurate time from the position information received from the satellite.

An audio / video (A / V) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes an image frame such as a still image or a moving image obtained by the image sensor in the video communication mode or the photographing mode. Then, the processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be equipped with two or more cameras according to the configuration of the terminal. The camera 121 can be used as the camera 12 in Fig.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. The microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like. The user input unit 130 may be a touch screen having a touch panel mounted on the display unit 14 in FIG.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. Also, it is responsible for sensing the power supply of the power supply unit 190, whether or not the interface unit 170 is coupled to an external device, and the like.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, Video input / output (I / O) ports, earphone ports, and the like.

Here, the identification module is a chip that stores various information for authenticating the use right of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identity module (SIM) ), A Universal Subscriber Identity Module (" USIM "), and the like. In addition, an apparatus having an identification module (hereinafter referred to as 'identification device') can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to each component in the mobile terminal 100 or to transmit data in the mobile terminal 100 to an external device.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 may include a display unit 151, an audio output module 152, an alarm unit 153, and the like.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or GUI (Graphic User Interface) related to the call is displayed. If the mobile terminal 100 is in the video communication mode or the photographing mode, the captured image and / or the received image or UI and GUI are displayed. The display unit 151 may be the display unit 14 of FIG.

Meanwhile, as described above, when the display unit 13 and the touch pad have a mutual layer structure to constitute a touch screen, the display unit 151 can be used as an input device in addition to the output device. The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display). In addition, there may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, the mobile terminal 100 may be provided with an external display unit (not shown) and an internal display unit (not shown) at the same time.

The audio output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 outputs an acoustic signal related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include receiving a call signal, receiving a message, and inputting a key signal. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than an audio signal or a video signal. For example, it is possible to output a signal in a vibration mode. When the call signal is received or a message is received, the alarm unit 153 can output a vibration to notify it. Alternatively, when the key signal is input, the alarm unit 153 can output the vibration by the feedback to the key signal input. The user can recognize the occurrence of an event through the vibration output as described above. Of course, a signal for notifying the occurrence of an event may also be output through the display unit 151 or the audio output module 152.

The memory 160 may store a program for processing and controlling the control unit 180 and may store a function for temporarily storing input / output data (e.g., a phone book, a message, a still image, .

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory) SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read- And an optical disc. In addition, the mobile terminal 100 may operate a web storage for storing the memory 150 on the Internet.

The control unit 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. In addition, the control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [ The control unit 180 may be the control unit 11 of FIG.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a computer-readable recording medium using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, And may be implemented by the control unit 180.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. In addition, the software codes may be stored in the memory 160 and executed by the control unit 180. [

1 shows a configuration of a remote control apparatus according to an embodiment of the present invention.

2A and 2B show an appearance of a remote control apparatus 10 according to an embodiment of the present invention.

3 and 4 show a network configuration diagram between the remote control apparatus 10 and an external apparatus according to an embodiment of the present invention.

5 to 7 show a TV control method according to an embodiment of the present invention.

8 and 9 show an external device control method according to an embodiment of the present invention.

10 shows a method of controlling a target device according to an embodiment of the present invention.

11 shows a method of controlling a remote control device in a state in which it is not facing a target device, according to an embodiment of the present invention.

12 shows a configuration of a mobile terminal that can be used as a remote control device according to an embodiment of the present invention.

Claims (29)

In the remote control device, display; A camera for capturing an image of the target device; A communication module for transmitting a control command to the target device and receiving content output from the target device from a target device corresponding to the captured image; And And a control unit for generating a control signal for controlling the target apparatus and controlling the display to display the content output by the target apparatus when the remote control apparatus operates in a screen synchronization mode. The method according to claim 1, Wherein the control unit generates a virtual image associated with the target device and controls the display to display the composite image of the captured target device image and the virtual image when the remote control device operates in the control mode. The method according to claim 1, Wherein the target device is a broadcast receiving device. 3. The method of claim 2, Wherein the target device is a broadcast receiving device, and the virtual image includes channel information of the broadcast receiving device. 3. The method of claim 2, Wherein the target device is an image display device, and the virtual image includes information of an external device connected to the image display device. 3. The method of claim 2, Wherein the virtual image includes content information that can be provided to an external apparatus and an external apparatus. The method according to claim 1, Wherein the display is a touch screen. 3. The method of claim 2, Wherein the virtual image is displayed around the image of the target device. The method according to claim 1, Wherein the camera and the display are directed to different planes. delete 3. The method of claim 2, Further comprising a memory for storing the target device image and the virtual image, Wherein the control unit controls the display to synthesize and display the target device image stored in the memory and the virtual image when the remote control device operates in the control mode. delete The method according to claim 1, Wherein when the remote control device operates in a menu operation mode, a menu displayed on the target device is operably displayed on the display. 14. The method of claim 13, Wherein the display is a touch screen and the menu is operable by the touch screen. 14. The method of claim 13, Wherein the menu is one of a screen adjustment, a reserved recording, a volume adjustment, and a time adjustment of the target device. 5. The method of claim 4, Wherein the channel information is displayed in such a manner that information of each channel is displayed on the left or right or top and bottom of a currently displayed channel. 17. The method of claim 16, Wherein the channel information is displayed on the left or right or top and bottom of the currently displayed channel in the order of the channel number. The method according to claim 1, Wherein information of an external device connected to the target device is displayed on the left or right or top and bottom of the target device. The method according to claim 1, Further comprising a motion sensor. delete 14. The method of claim 13, The control unit Wherein when the zoom-in operation command of the camera is received, the remote controller enters the menu operation mode and displays the menu. The method according to claim 1, The camera scans the target device and external devices connected to the target device, Further comprising a memory for storing images and relative positions of the scanned target device and external devices, And displays an image and a relative position of the target device and the external devices on the display regardless of whether the remote control device faces the target device or the external devices. A remote control method for a remote control device, Capturing an image of the target device; Receiving a content output from the target device from a target device corresponding to the photographed image; And And displaying the content output by the target device when the remote control device operates in a screen synchronization mode. 24. The method of claim 23, Generating a virtual image associated with the target device; And And synthesizing and displaying the photographed target device image and the virtual image when the remote control device operates in the control mode. 25. The method of claim 24, Wherein the target apparatus is a broadcast receiving apparatus, and the virtual image includes channel information of the broadcast receiving apparatus. 25. The method of claim 24, Further comprising a memory for storing the image of the target device and the virtual image, Further comprising synthesizing and displaying the target device image stored in the memory and the virtual image when the remote control device operates in the control mode. delete 24. The method of claim 23, Wherein the menu displayed on the target device is operably displayed on a display of the remote control device when the remote control device operates in a menu operation mode. 29. The method of claim 28, Wherein the display is a touch screen and the menu is operable by the touch screen.

Images