KR20110028185A - Remote control device and method for remote controlling using the same - Google Patents

Abstract

PURPOSE: A remote control device and a method thereof are provided to supply convenient user interface. CONSTITUTION: A camera unit(12) photographs a target device. A communication unit(13) transmits a control signal. A display unit(14) displays an synthesis image. A user input unit receives a user input. A control unit(11) controls a virtual image by operating and displaying the image.

Description

REMOTE CONTROL DEVICE AND METHOD FOR REMOTE CONTROLLING USING THE SAME}

The present invention relates to a remote control device and method. More specifically, the present invention relates to a remote control apparatus and method for synthesizing and displaying a virtual image of a real image photographing a target device and content provided by the target device.

There are remote controls that control home appliances such as televisions, DVD players, game consoles, and audio. The remote control transmits a binary code to the home appliance using infrared rays, and controls the volume, channel, operation mode, etc. of the home appliance accordingly.

Recently, as the type and number of home appliances provided in a home increase, an integrated remote controller capable of controlling a plurality of home appliances with a single remote controller has been released.

However, the integrated remote controller does not provide any convenience other than integrating a remote controller for a plurality of home appliances into a single device.

There is a need for new remote control devices that can more conveniently and intuitively control home appliances.

An object of the present invention is to provide a remote control apparatus and method for providing a user interface more intuitive and easy to use.

An object of the present invention is to provide a remote control apparatus and method for more intuitively and easily browsing a plurality of contents provided by a target apparatus, and selecting and executing predetermined contents.

An object of the present invention is to provide a remote control apparatus and method for selecting an external device connected to a target device more intuitively and easily, and executing content provided by the external device on the target device.

Remote control apparatus according to an embodiment of the present invention includes a camera unit for photographing the target device; A communication unit for receiving information of contents provided by the target device and transmitting a control signal to the target device; A display unit configured to display a composite image obtained by synthesizing a real image of the target device and a virtual image of the content; A user input unit configured to receive a user input for manipulating the virtual image; And a controller configured to control to display and manipulate the virtual image according to the received user input.

Remote control method according to an embodiment of the present invention comprises the steps of photographing the target device; Receiving information on content provided by the target device; Generating a virtual image of the content based on the received information of the content; And synthesizing and displaying the actual image of the target device and the virtual image of the content.

According to the present invention can provide a user interface more intuitive and easy to use.

According to the present invention, a plurality of contents provided by the target apparatus can be viewed more intuitively and easily, and predetermined contents can be executed on the target apparatus.

According to the present invention, it is possible to select an external device connected to the target device more intuitively and easily, and to execute content provided by the external device on the target device.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 shows a configuration of a remote control apparatus according to an embodiment of the present invention.

As shown in FIG. 1, the remote control apparatus 10 according to an exemplary embodiment of the present invention transmits a control command to the camera 12 and the target apparatus to capture an image of the target apparatus, and communicates with the target apparatus. A short range communication module 13, a display unit 14 displaying a photographed real image of the target device or a virtual image related thereto, a control signal for controlling the target device, and a virtual image associated with the target device And a controller 11 which controls to synthesize and display the real image and the virtual image.

The target device is an electronic device controlled by a remote control device according to the present invention, and may be a broadcast receiving device such as a TV or an image display device equipped with a display unit. In addition, according to an embodiment, the target device may be a DVD player, a Blu-ray player, a VCR, a PC, an MP3 player, an audio device, a DVix player, a game device, or the like.

The camera 12 captures an image of the target device and transfers the image to the controller 11. The camera may be a camera module mounted on a conventional mobile terminal such as a mobile phone.

The short range communication module 13 communicates with the target device, transmits a command of the remote control device 10, and receives data from the target device. The short range communication module 13 may use any communication module such as a Bluetooth module, an infrared communication module, a zigbee module, a NFC (Near Field Communication) module, an RF communication module, or a combination thereof.

The display unit 14 displays an image captured by the camera 12. According to an exemplary embodiment, an image obtained by synthesizing the actual image of the target device photographed by the camera 12 and the virtual image generated by the controller 11 under the control of the controller 11 is output.

The display unit 14 may be any display device capable of displaying information such as an LCD, an OLED, a VFD, an electrophoretic device, and the like.

In addition, the display unit 14 may be a touch screen on which a touch panel is mounted on the front surface. The touch panel may be a touch panel of any type well known, such as an electric resistive film type or a capacitive type.

When the controller 11 controls the operations of the above-described components, the controller 11 generates an image obtained by synthesizing the virtual image associated with the actual image captured by the camera 12, among others. As will be described later, the virtual image is an image related to the captured target device. When the target device is a TV, the virtual image may be a channel browsing image for a plurality of channels or an image including information of external devices connected to the TV. When the target device is an external device such as a DVD player, a Blu-ray player, a VCR, a PC, an MP3 player, an audio, a DVix player, a game machine, or the like, the target device may be an image including information on content that the external devices can provide.

In addition, according to the exemplary embodiment, a motion sensor for detecting a user's gesture applied to the remote control device 10 may be mounted inside the remote control device 10. As the motion sensor, any sensor capable of detecting the movement of the remote control device 10 such as an acceleration sensor, a gyro sensor, and a geomagnetic sensor may be used.

As described later, a gesture applied to the remote control apparatus 10 may be detected and various commands may be mapped to the gesture. By inputting a command by the gesture, a user may be provided with a more intuitive and easy user interface.

2A and 2B show an appearance of a remote control apparatus 10 according to an embodiment of the present invention. 2A is a perspective view including the front of the remote control device 10, and FIG. 2B is a perspective view including the back side. According to an embodiment, the front side of the remote control device 10 may face the user, and the rear side may face the target device to be controlled.

As shown in FIG. 2A, the display unit 14 is disposed in front of the remote control apparatus 10. The display unit 14 may be a touch screen.

Other control buttons may be disposed in the remaining portions except for the display unit 14. The control button may include a power button 21, a channel control button 24, a volume control button 25, a mute button 23, a previous channel button 22, and the like. In addition, the control button may include various buttons according to the type of the target device, and may vary according to embodiments. According to the exemplary embodiment, the touch screen may be used as the display unit 14, only one or two of the control buttons may be left, and the remaining buttons may be displayed on the touch screen.

Referring to FIG. 2B, the camera 12 is disposed at the rear of the remote control apparatus 10. The camera 12 may face the direction of photographing the target device during operation. Preferably, the direction in which the camera 12 and the display unit 14 face may be opposite to each other. That is, the camera 12 may face the target device to be controlled, and the display 14 may face the user.

According to the embodiment, the actuator 12 is connected to the camera 12 may be configured to be able to change the direction in the up and down 27 or left and right (28) direction.

3 and 4 illustrate a network configuration diagram between the remote control device 10 and an external device according to an embodiment of the present invention.

As shown in FIG. 3, the remote control apparatus 10 may directly communicate with one of the target devices, for example, the TV 31, and the TV 31 may communicate with other external devices 32 and 33 by wire or wirelessly. . Instead of the TV 31, a video display device without a tuner may be used.

In the present embodiment, the remote control device 10 communicates with the TV 31. The TV transmits a command to the external devices 32 and 33 or receives data from the external devices 32 and 33 according to the command received from the remote control device 10 and transmits the data to the remote control device 10. do. The short-range wireless communication module capable of communicating with the remote control device 10 should be mounted on the TV 31, and the external devices 32 and 33 may be equipped with a wired or wireless communication module with the TV 31.

The TV 31 or the image display device may receive and display content from the external devices 32 and 33. For example, the DVD player 32 plays the DVD title and transmits it to the TV 31, and the TV 31 displays the content included in the DVD title. In addition, when the game title is played back on the game machine 33, the game screen is displayed on the TV 31. FIG. When the external device is an MP3 player, only sound may be output through the TV 31.

According to one embodiment of the invention, as shown in Figure 4, the remote control device 10 may be configured to enable direct communication with not only the TV 31, but also other external devices (32, 33). . In this case, the external devices 32 and 33 should be equipped with a short range wireless communication module for communicating with the remote control device 10 in addition to the wired or wireless communication module with the TV 31.

As described above, the short range wireless communication module mounted on the remote control device 10, the TV 31, or the external devices 32 and 33 may be a Bluetooth module, an infrared communication module, a zigbee module, or a near field communication (NFC). Module), an RF communication module, or any short range wireless communication module.

As shown in FIGS. 3 and 4, the user 34 may direct the camera toward the target device to control the remote control device 10 and then control the target device in various ways.

In various embodiments described below, the user can change the operation mode by operating the remote control device 10, and embodiments described below can be performed when the remote control device 10 is in a specific operation mode. The specific operation mode may include a control mode, a menu operation mode, a screen synchronization mode, and the like.

According to one embodiment of the present invention, the content displayed on the target device to which the remote control device 10 is directed may be configured to be displayed on the display 14 of the remote control device 10. This operation may be performed when the remote control device is in a screen synchronization mode, for example.

3 or 4, when the remote control device 10 is turned on to the TV 31, the content currently being played on the TV 31 screen, that is, the screen being played on the TV broadcast program or the DVD player, is remote. It may be displayed on the screen of the control device 10 as it is.

When the tuner or TV is an IPTV or a broadband TV, the TV 31 may display a broadcast program received through an internet cable or may display content received from external devices 32 and 33.

When the remote control device 10 faces the TV 31, the control unit 11 of the remote control device 10 determines that the remote control device 10 is currently facing the TV 31 through the camera 12. As shown in FIG. 5, the control unit may receive the content currently being played from the TV 31 and display it on the display 14.

The TV 31 transmits the displayed image data to the remote control apparatus 10 using the above-mentioned short range communication module, and the remote control apparatus 10 displays the image data on the display 14.

Once the operation of displaying the content being played on the TV 31 on the remote control device 10 is started, there may be other input from the user, even if the remote control device 10 (or the camera 12) faces away. The content can continue to be received from the TV 31 until it is displayed.

The remote control apparatus 10 according to an embodiment of the present invention includes a camera unit 12 photographing a target device, a communication unit receiving information of contents provided by the target device, and transmitting a control signal to the target device ( 13), a display unit 14 for displaying a composite image obtained by synthesizing the actual image of the target device and the virtual image of the content, a user input unit for receiving a user input for manipulating the virtual image, and the received image. The controller 11 may be configured to control to display and manipulate the virtual image according to a user input.

Hereinafter, the operation of the remote control apparatus 10 according to an embodiment of the present invention will be described with reference to FIGS. 6 to 12.

FIG. 6A illustrates a remote control apparatus 10 for displaying a composite image obtained by synthesizing a real image 71 of the target apparatus and a virtual image 72 of content provided by the target apparatus according to an exemplary embodiment of the present invention. It is an illustration.

As shown in FIG. 6A, the camera unit 12 of the remote control apparatus 10 photographs a target device and provides an actual image of the target device to the controller 11.

As described above, the target device may be a broadcast receiving device such as a TV, but the target device is not limited thereto, a display device for outputting a predetermined image or video, an audio device for outputting a predetermined voice, and the broadcast reception. It may be an external device such as a DVD player, a Blu-ray player, a game machine, a portable terminal, a USB memory, a VCR, a computer, an MP3 player, etc., which is connected to the device by wire or wirelessly to provide predetermined content.

The communication unit 13 receives information of content provided by the target device. According to an embodiment of the present invention, if the target device is the broadcast receiving device as described above, the content may be a broadcast channel provided by the broadcast receiving device, and the information of the content may be broadcast channel information. That is, the content information may include a channel number of a channel provided by the broadcast reception device, a broadcast station name, a representative image, and the like.

According to an embodiment, if the target device is a DVD player, the content may be a DVD title inserted into the DVD player, and the information of the content may include a title, genre, play time, viewing rating, representative image, It can include snapshots of key scenes.

According to an embodiment, if the target device is a computer, the content may be video, audio, image, and text data stored in the computer, and the information of the content may be a title, a representative image, or the like of the content.

The received information of the content is provided to the control unit 11, the control unit 11 generates a virtual image of the content based on the information of the content.

In detail, the controller 11 generates a virtual image of the content provided by the target device based on the received information of the content.

Here, the virtual image is an image representing an object that does not exist in reality unlike the actual image photographed by the camera unit 12. The virtual image is a graphic, an image, a video, etc. generated by the controller 11. Can be.

The controller 11 generates a virtual image of the content provided by the target device. According to an embodiment, the virtual image for the content may include content information such as a title of the content and a representative image.

For example, as illustrated in FIG. 6A, the virtual image 72 of the content may include a channel number of a broadcast channel provided by the target device, that is, the broadcast reception device. According to an exemplary embodiment, the virtual image 72 may further include a representative image of a program broadcast on a corresponding channel.

The controller 11 synthesizes the generated virtual image 72 with the actual image 71 of the target device photographed by the camera unit 12 to generate a composite image, and the display unit 14 displays the synthesized image. You can control to display a composite image.

That is, in FIG. 6A, the image 71 of the broadcast reception device displayed on the display unit 14 of the remote control device 10 is an actual image captured by the camera unit 12, and the image 71 of the broadcast reception device. The image 72 positioned around the circumference is a virtual image generated by the controller 11. Each virtual image 72 corresponds to a channel provided by the broadcast reception device, and a user can check a list of channels provided by the broadcast reception device through the virtual image 72 displayed around the real image 71. Can be.

According to an embodiment of the present invention, the content may be a plurality, and the display unit 14 may display a plurality of virtual images corresponding to the plurality of contents in a predetermined pattern.

For example, as illustrated in FIG. 6A, the content provided by the target device, that is, the channel provided by the broadcast reception device may be a plurality of channels, and the display unit 14 may include a plurality of channels corresponding to the plurality of contents. The virtual image 72 may be displayed by being arranged in a panorama pattern around a predetermined pattern, for example, the actual image 71 of the target device.

In FIG. 6A, the virtual image 72 is arranged to the left and right about the real image 71 of the target device. According to an embodiment, the virtual image 72 is vertically or vertically centered on the real image 71. It can also be arranged diagonally.

According to an embodiment, as shown in FIG. 6B, the virtual image 72 of the content is arranged in a circle around the actual image 71 of the target device so that a user can view the virtual image 72 of the plurality of contents. ) At a glance.

6A and 6B illustrate a case in which the target device is a broadcast receiving device. As shown in FIG. 6C, when the target device is a computer, the remote control device 10 may display an actual image of the computer. 73 and a virtual image 74 of content provided by the computer, for example, content stored in the computer, may be synthesized and displayed.

Similarly, the content stored in the computer may be plural. In this case, the display unit 14 assigns a plurality of virtual images 74 corresponding to the plurality of content to the periphery of the actual image 73 of the computer. It can be arranged in a pattern.

The user input unit may receive a user input for manipulating the displayed virtual image, and the controller 11 may control to manipulate and display the virtual image according to the received user input.

According to an embodiment of the present invention, the user input unit may be at least one of a touch screen, a key button, and a motion sensor.

According to an embodiment of the present invention, when the user input unit is a touch screen, the touch screen detects a user's drag gesture, and the controller 11 responds to the detected drag gesture to the virtual image 72. Can be moved in the drag direction.

For example, as illustrated in FIG. 7A, the remote control apparatus 10 may include a touch screen as a user input unit, and when the touch screen detects a user's drag gesture, the controller 11 may determine the touch screen. The displayed virtual image 72 may be moved in the drag direction in response to the detected drag gesture.

As a result, the plurality of virtual images 72 corresponding to the plurality of channels may be moved in the drag direction so that the plurality of channels may be continuously scanned (that is, channel browsing). That is, in FIG. 6A, the remote control device 10 displays the virtual image 72 for the channels CH 7 to CH 13, but the remote control device 10 displays the CH 7 by the drag gesture as shown in FIG. 7A. In addition to CH 13, other channels such as CH 2 to CH 6 may be continuously browsed.

Accordingly, the user can more easily and intuitively check the list of contents provided by the target device by inputting the user input as described above.

According to an embodiment of the present invention, when the user input unit is a key button and the user presses a predetermined key button, the controller 11 continuously displays the virtual images 72 in response to the key button input. You can browse.

According to an embodiment of the present invention, when the user input unit is a motion sensor, the motion sensor detects a movement of the remote control device, and the control unit 11 moves in the direction of movement of the remote control device 10. You can also move the virtual image.

FIG. 7B is an exemplary diagram illustrating a movement of a virtual image according to a movement direction of a remote control apparatus detected by a motion sensor according to an embodiment of the present invention, wherein (i) is a front view of the remote control apparatus 10. (Ii) is a plan view of the remote control device 10 and the target device 31.

As shown in FIG. 7B, when the user tilts the remote control device 10 in a predetermined direction, for example, counterclockwise, a motion sensor embedded in the remote control device 10 may be connected to the remote control device 10. In response to the tilting of the controller 11, the controller 11 may move the virtual image 72 to the tilting direction of the remote control apparatus 10, that is, to the right in response to the detected tilting.

As a result, similar to FIG. 7A, the plurality of virtual images 72 are continuously scanned to display virtual images for other contents not displayed on the display unit 14, so that the user can view a plurality of contents. You can see a list of at a glance.

According to an embodiment, the motion sensor may include a sensor capable of detecting a movement direction, a rotation direction, a tilting direction, and the like of the remote control device 10 such as an inertial sensor, a geomagnetic sensor, an acceleration sensor, and the like.

In FIG. 7B, the remote control device 10 is tilted counterclockwise, and the virtual image 72 is moved to the right to browse the contents. In contrast, the remote control device 10 is tilted clockwise. The motion sensor may detect tilting in the clockwise direction.

Specifically, when the remote control device 10 is tilted in the clockwise direction, the motion sensor built into the remote control device 10 detects the tilting direction of the remote control device 10 and detects the detected tilting direction. In response, the controller 11 may browse the contents by moving the displayed virtual image 72 to the tilting direction, that is, to the left.

In this case, while the virtual images 72 for the channel are moved to the left, the virtual images for the other channels of CH 14 or more besides CH 7 to CH 13 shown in FIG. 6A are displayed, and as a result, the user can select the channel list. You can see at a glance.

According to an embodiment of the present invention, the remote control apparatus 10 does not receive a separate user input through a user input unit, and analyzes a change pattern of an actual image photographed by the camera unit 12 to analyze the virtual image. Can manipulate them.

Specifically, as shown in FIG. 7C, when the remote control apparatus 10 is moved to the right by a user, the actual image 71 of the target apparatus photographed by the camera unit 12 is displayed on the display. Will move left on portion 14.

In this case, the controller 11 may determine a moving direction of the remote control apparatus 10 by analyzing a change pattern of an image captured by the camera unit 12, for example, an actual image of a target device. That is, in FIG. 7C, the controller 11 determines that the actual image 71 of the target device is moved to the left among the images captured by the camera unit 12, so that the remote control device 10 moves to the right. You may decide to move.

Then, the controller 11 may move the virtual image in the determined movement direction. That is, in FIG. 7C, the controller 11 may browse the channel by moving the virtual image 72 to the right side, which is the moving direction of the remote control apparatus 10.

As such, the remote control apparatus 10 of the present invention may move the virtual image 72 by analyzing an image photographed by the camera unit 12 without a user input through a user input unit.

According to an embodiment of the present invention, the user input unit receives a user input for selecting a virtual image for a predetermined content, and the controller 11 causes the target device to execute content corresponding to the selected virtual image. A control signal can be generated.

For example, as shown in FIG. 7D, when the user input unit is a touch screen, the touch screen may detect a touch gesture of a user who touches a predetermined virtual image among the virtual images 72.

In this case, the controller 11 may generate a control signal for executing the content corresponding to the touched virtual image, for example, CH 12 in the target device, in response to the detected touch gesture.

Then, the communication unit 13 may transmit the generated control signal to the target device, and as a result, the target device may output a channel selected by the user.

According to an embodiment, the user input for selecting the virtual image may be a gesture for placing the virtual image to be selected at a predetermined position of the display unit 14.

For example, as shown in FIG. 7D, instead of touching a predetermined virtual image (ie, a virtual image of CH 12), the predetermined virtual image may be dragged and dropped to give an actual image of the target device ( 71, the control unit 11 may generate a control signal for controlling the target device to execute content corresponding to the corresponding virtual image.

According to an embodiment, in FIG. 7B, when the motion sensor no longer detects the tilting of the remote control device 10, that is, when the remote control device 10 returns to before the tilting, the control unit ( 11) may generate a control signal for stopping the movement of the virtual images 72 and controlling the target device to execute content corresponding to the virtual image located at a predetermined position of the display unit.

In other words, when the remote control device 10 returns to before the tilting, the control unit 11 is located at a predetermined position of the display unit, for example, at a position where the actual image 71 of the target device is displayed. The virtual image overlapped with the image 71 may be selected. Thereafter, a control signal for executing the content corresponding to the selected virtual image on the target device may be generated and transmitted to the target device.

As a result, the user may not only browse a plurality of contents provided by the target device, but also select and execute specific content on the target device.

According to an embodiment of the present disclosure, the communication unit 13 may further receive information on an external device connected to the target device, and the display unit 14 may further display a virtual image of the external device. .

For example, as shown in FIG. 8A, the remote control apparatus 10 of the present invention may include a virtual image 81 of an external device connected to the target device in addition to the virtual image 72 of the content provided by the target device. ) Can be provided further.

According to an embodiment, the virtual image 72 for the content and the virtual image 81 for the external device may be displayed together, but the present invention is not limited thereto, and each virtual image may be displayed separately.

The communication unit 13 receives information about an external device connected to the target device. For example, when the target device is a broadcast receiving device, the communication unit 13 may receive information about an external device, such as a DVD player, a game machine, a computer, a USB memory, or the like, which is wirelessly or wired connected to the broadcast receiving device. have. The received information about the external device may include the type of the external device, the title, genre, playback time, viewing grade, representative image, and the like of the content provided by the external device.

Then, the controller 11 generates a virtual image 81 for the external device based on the received information of the external device. Then, the controller 11 controls the display unit to synthesize the virtual image 81 of the generated external device with the actual image 71 of the target device.

As a result, the user can easily grasp the external device connected to the target device through the remote control device 10.

According to an embodiment of the present disclosure, when there are a plurality of external devices, the display unit 14 may arrange and display a plurality of virtual images corresponding to the plurality of external devices in a predetermined pattern.

For example, as illustrated in FIG. 8A, the display unit 14 displays a plurality of virtual images 81 corresponding to the plurality of external devices in a vertical direction with respect to the actual image 71 of the target device. Can be arranged in a panoramic manner.

According to an embodiment, similar to FIGS. 6A and 6B, the plurality of virtual images 81 may be arranged in left and right directions around the actual image 71 of the target device in addition to the vertical direction. It may be arranged in a circle around the actual image 71 of.

According to an embodiment, when the virtual image 72 for the content provided by the target device and the virtual image 81 for the external device are simultaneously provided, as illustrated in FIG. 8A, the remote control device ( 10 may be displayed by differently arranging an arrangement pattern of the virtual image 72 for the content (for example, a panorama in the left and right direction) and an arrangement pattern of the virtual image 81 for the external device (for example, a panorama in the vertical direction). Can be.

According to an embodiment of the present invention, the user input unit receives a user input for moving the virtual image 81 for the external device, and the control unit 11 responds to the user input for the virtual device for the external device. Image 81 can be moved.

The movement of the virtual image 81 relative to the external device is similar to the movement of the virtual image 72 of the content described with reference to FIGS. 7A-7C.

For example, as illustrated in FIG. 8B, when the user input unit is a touch screen, when the touch screen detects a drag gesture in a predetermined direction, the controller 11 responds to the detected drag gesture. The virtual image 82 for the external device may be moved in the drag direction.

In addition, as illustrated in FIG. 8C, when the user input unit is a motion sensor, when the motion sensor detects a movement of the remote control device 10, the controller 11 may detect the detected remote control device 10. ), The virtual image 82 of the external device may be moved in the direction of movement of the remote control device 10 in response to the movement of the external device.

For example, as shown in FIG. 8C, when the user tilts the remote control device 10 downward, the motion sensor built in the remote control device 10 detects the tilting of the remote control device 10. In addition, the controller 11 may move the virtual image 81 of the external device downward in a tilting direction in response to the detected tilting.

Similarly, when the remote control device 10 is tilted upward, the motion sensor detects the tilting of the remote control device 10, and the controller 11 responds to the detected tilting of the external device. The virtual image 81 for may be moved upward in a tilting direction.

According to an embodiment, the remote control apparatus 10 may receive a moving direction of the virtual image 81 through a key button, and the controller 11 inputs the virtual image 81 of the external device. It can be moved in the direction of movement received.

According to an embodiment, the remote control device 10 determines the moving direction of the remote control device 10 by analyzing a change pattern of the image captured by the camera unit 12 similarly to FIG. 7C. The virtual image 81 may be moved in the moving direction.

As a result, the remote control apparatus 10 of the present invention can browse while moving a plurality of virtual images 81 corresponding to a plurality of external devices continuously, so that a user can view a plurality of external devices connected to the target device. I can grasp it at a glance.

According to an embodiment of the present invention, the user input unit receives a user input for selecting a virtual image for a predetermined external device, and the control unit 11 outputs an output signal of the external device corresponding to the selected virtual image. The external input switching signal may be generated to control the output from the target device.

For example, as illustrated in FIG. 9A, when detecting a touch input of touching a predetermined virtual image 81, for example, a virtual image for a computer, through the touch screen, the controller 11 may detect the detected touch input. In response, an external input switching signal for controlling an output signal of an external device corresponding to the selected virtual image, that is, a computer, may be output from the target device.

The generated external input switching signal is transmitted by the communication unit 13 to the target device, and the target device receives an external input switching signal to output an output signal of the selected external device, that is, a computer. You can switch.

Similar to the selection of the virtual image 72 for the above-described content, the virtual image 81 for the external device may be selected by drag and drop, may be selected via a key button, the remote control device When 10 returns to before the tilting, the external input switching signal may be generated by selecting the virtual image 81 of the external device located at a predetermined position of the display unit.

According to an embodiment of the present invention, when the external input switching signal is transmitted to the target device, the communication unit 13 may receive information of content provided by the external device.

Then, the controller 11 may generate a virtual image of the content provided by the external device based on the received information of the content. Then, the controller 11 may control to synthesize and display the actual image 71 of the target device and the virtual image of the content provided by the external device.

For example, as illustrated in FIG. 9B, when the external input switching signal is transmitted to the target device, the communication unit 13 receives information of the content provided by the computer in the selected external device, that is, the computer in FIG. 9A. The controller 11 may generate a virtual image 82 of the content provided by the computer based on the received information of the content, and control to display the synthesized image with the actual image 71 of the target device. Can be.

As a result, instead of displaying the virtual image of the content provided by the target device, that is, the virtual image 72 for the channel, the virtual image 82 of the content provided by the computer is displayed around the target device. The user may browse the contents provided by the computer through the inputs of FIGS. 7A to 7 and execute the desired contents on the target device.

According to an embodiment of the present invention, the user input unit further receives a user input for continuously displaying the composite image, that is, the image obtained by combining the real image 71 and the virtual image 72, and the controller 11. ) May control to continuously display the composite image regardless of the direction in which the camera unit 12 faces.

For example, as shown in FIG. 10, the remote control device 10 has a 'lock' button 16 on the outside, and from the user the actual image 71 of the target device and a virtual image of its contents ( User input to continuously display 72).

That is, as shown in FIG. 10, when the camera unit 12 captures an image receiving apparatus, the control unit 11 controls the actual image 71 of the image receiving apparatus and the contents provided by the image receiving apparatus. When the user presses the 'lock' button 16, the control unit 11 controls the actual image 71 regardless of the direction in which the camera unit 12 faces. ) And the virtual image 72 may be continuously displayed.

As a result, even if the user presses the 'lock' button 16 and points the remote control device 10 in any direction, the remote control device 10 does not display the actual image that was displayed when the lock button 16 was pressed. The virtual image can be displayed continuously.

Even in this case, the communication unit 13 may receive the content information from the target device and transmit a control signal to the target device.

Therefore, the user can browse the contents provided by the target device by inputting the inputs of FIGS. 7A to 7D without pointing the remote control device 10 to the target device, and select and execute the desired content. have.

According to an embodiment of the present invention, the user input unit receives a zoom command for zooming in or out of the composite image, and the controller 11 controls to zoom in or zoom out of the composite image according to the zoom command. Can be.

According to an embodiment, a zoom command for zooming in or out of the composite image may be input through a key button. For example, as shown in FIG. 11, when the zoom button 24 provided in the remote control apparatus 10 is pressed, for example, when the '-' button is pressed, the real image 71 and the virtual image ( The synthesized image 72 may be zoomed out and reduced.

Conversely, when the '+' button is pressed, the composite image may be zoomed in and enlarged to be displayed.

According to an embodiment, the zoom command may be multi-touch on a touch screen. For example, if a user touches two fingers to the touch screen and increases the distance between the two fingers, the touch screen detects the corresponding multi-touch gesture, and the controller 11 responds to the multi-touch gesture. To zoom in on the composite image.

Unlike the embodiment in which the external device is selected through the virtual image 81 of the external device as illustrated in FIG. 9A, according to another embodiment of the present invention, the external device may be selected through the actual image of the external device. .

According to another embodiment of the present invention, the user input unit receives an 'external input change command' for transmitting and outputting an output signal of an external device to a target device. Then, the control unit 11 sets the remote control device 10 to the 'external input switching mode' in response to the external input switching command.

In this case, the user places the remote control device 10 at the external device to be selected in order to select an external device to transmit an output signal to the target device.

For example, if a user wants to send an output signal of a computer to the target device, the user can position the remote control device 10 to face the computer.

In this case, the camera unit 12 may photograph the external device to be selected, that is, the computer, and as shown in FIG. 12A, the display unit 14 displays an actual image 73 of the photographed computer. can do.

The message displayed at the top of FIG. 12A is a message for notifying the user that the remote control apparatus 10 is currently in an 'external input switching mode'.

Then, the user may receive a user input for selecting the actual image 73 of the photographed external device.

For example, as shown in FIG. 12B, when a user touches an actual image 73 of a computer displayed on a touch screen, the touch screen detects a touch input of the actual image 73, and the controller (11) transmits an output signal of an external device corresponding to the touched actual image 73, that is, a computer, to the target device in response to the touch input to generate an external input switching signal for outputting from the target device, The external input switch signal may be transmitted to the target device through the communication unit 13.

The target device may receive the external input switching signal, and accordingly set the selected computer as an external device to receive and output an output signal output from the computer.

According to an embodiment, when the camera unit 12 photographs the external device to be selected, that is, the computer for a preset time, the controller 11 automatically selects the photographed external device to select the external device. The external signal may be generated by transmitting an output signal to the target device and outputting the output signal.

That is, according to the above-described embodiment, the output signal of the photographed external device may be transmitted to the target device and output by photographing the external device to be selected for a predetermined time without a separate user input for selecting the external device.

According to an embodiment, the remote control apparatus 10 receives a user input of dragging and dropping the actual image 73 of the external device to be selected to the point where the actual image 71 of the target device is displayed, and outputting the external device. The signal may be transmitted to the target device and output.

For example, as illustrated in FIG. 12C, the user may adjust the remote control apparatus 10 such that both the external device and the target device to be selected are displayed on the display unit 14. In this case, the user may enlarge or reduce the image displayed on the display unit 14 by inputting the above-described zoom command.

When the display unit 14 displays real images 73 and 71 of the external device and the target device, the user drags and drops the real image 73 of the external device to the real image 71 of the target device. You can.

In this case, the touch screen detects the drag and drop gesture, and the controller 11 transmits and outputs an output signal of the external device, for example, a computer, to the target device in response to the detected drag and drop gesture. You can.

13 is a flowchart illustrating a remote control method according to an embodiment of the present invention.

Step S91 photographs the target device. That is, the camera unit 12 included in the remote control apparatus 10 of the present invention photographs the target device to provide an actual image of the target device.

According to an embodiment, the target device may be a broadcast receiving device that provides a broadcast program.

Step S92 receives information of content provided by the target device. That is, the communication unit 13 receives information of content provided by the target device. According to an embodiment, when the target device is a broadcast receiving device, the content may be a broadcast program of a plurality of channels provided by the broadcast receiving device. have.

According to an embodiment, the information of the content may include a channel number of the channel, a broadcasting station, a representative image of a broadcast program provided in the channel, and the like.

In operation S93, a virtual image of the content is generated based on the information of the received content. That is, the controller 11 generates a virtual image of the content provided by the target device based on the information of the received content.

In operation S94, the actual image captured by the target device and the virtual image are synthesized and displayed. That is, the controller 11 controls to display the generated virtual image and the actual image of the target device photographed by the camera unit 12 and display them on the display unit.

According to an embodiment, when there are a plurality of contents, a plurality of virtual images corresponding to the plurality of contents may be arranged and displayed in a predetermined pattern.

Step S95 receives a user input for manipulating the virtual image. For example, the user may move the virtual image through a user input unit or receive a user input of zooming in or out of an image obtained by combining the virtual image with the actual image.

According to an embodiment, the user input unit may include at least one of a touch screen, a key button, and a motion sensor.

Step S96 manipulates and displays the virtual image according to the received user input. That is, the controller 11 may move the virtual image or zoom in or zoom out the image displayed on the display unit according to the user input.

As a result, the user can easily check a list of contents provided by the target device, for example, a plurality of channels provided by the broadcast receiving device.

Step S97 receives a user input for selecting a virtual image. That is, after the user browses the channels provided by the broadcast receiving apparatus by moving the displayed virtual images, the user inputs a user input for selecting a channel to be watched.

Step S98 generates a control signal for executing the content corresponding to the selected virtual image on the target device. That is, when the virtual image of the desired channel is selected, the controller 11 generates a control signal for executing the channel corresponding to the selected virtual image in the broadcast receiving apparatus.

Step S99 transmits the control signal to the target device.

According to the present invention, the user can intuitively and easily browse contents provided by the target device, for example, channels provided by the broadcast receiving device, through the remote control device 10 of the present invention.

In addition, the channel of the broadcast reception device may be switched to a desired channel by selecting a virtual image of the channel displayed on the remote control device 10.

According to an embodiment of the present invention, when the remote control device 10 faces an external device, information on the external device may be generated and displayed as a virtual image. This embodiment may also be performed when the remote control device 10 is in the control mode.

As shown in FIG. 14, it is assumed that the remote control device 10 faces an external device, such as a DVD player 32. The camera of the remote control apparatus 10 captures an image of the DVD player 32, and the controller analyzes the captured image to determine that the remote control apparatus 10 is facing the DVD player 32.

The controller of the remote control apparatus 10 may receive a list of playable contents stored in the DVD player 32 from the DVD player 32 and display the virtual image 61 around the image 62 of the DVD player. . For example, when the mermaid and the kung fu panda are included in the DVD title inserted into the DVD player 32, the mermaid and the kung fu panda title may be received from the DVD player 32 to generate a virtual image. . Then, as shown in FIG. 15, the virtual image 61 may be displayed around the actual image 62 of the DVD player. From the image displayed in this way, the user can easily grasp the list of contents playable in the DVD player 31.

In addition, by displaying the virtual image 61 of the reproducible content list around the DVD player image 62, the user can intuitively grasp the contents.

The virtual image 61 of the content list may include text such as titles of contents, a cover letter image of a DVD title, a character representing contents, or an advertisement image.

According to an embodiment of the present invention, the remote control device may be used to control a menu of a target device to which the remote control device is directed. This embodiment can be performed when the remote control device 10 is in the menu operation mode, for example.

3 or 4, assume that the remote control device 10 faces the TV 31, that is, the camera of the remote control device 10 faces the TV 31. As shown in FIG. With the remote control device 10 facing the TV 31, as shown in FIG. 16, when the user presses a button 65 mounted on one side of the remote control device 10, the display of the remote control device 10 ( 14 may display a menu 66 for controlling the TV 31.

The control menu of the TV 31 may include screen adjustment, reservation recording, volume adjustment, time adjustment, and the like.

The above operation may be similarly performed on the TV 31 as well as other external devices 32 and 33.

The button 65 of the remote control device 10 may allow the user to place the remote control device 10 at the position of a shooting button of a conventional camera. The user can display a menu of the target device being photographed by pressing the button 65 in the same way as pressing the photographing button of the camera while facing the remote control device 10, similarly to photographing the target device with the camera. Can be.

With the menu 66 displayed, the user can touch the display to select the menu and control the menu of the TV 31.

According to an embodiment of the present invention, when the locations of the target devices are input to the remote control device 10, the target devices may be controlled even when the camera of the remote control device 10 does not face the target device. In this case, the motion sensor is mounted on the remote control device 10.

As shown in FIG. 17, the user 34 causes the camera of the remote control device 10 to scan the target devices, namely the TV 31 and the external devices 32, 33. The relative positions and images of the scanned target devices are stored in a memory in the remote control device 10. The remote control device 10 may display an image of the target devices and the location thereof on the display.

In this state, when the user moves the remote control device 10 side to side, the motion sensor in the remote control device 10 detects the movement and displays it on the display of the remote control device 10.

That is, even if the remote control device 10 does not face the target device to be controlled, a motion sensor can be used to detect a motion that the user applies to the remote control device 10 and is displayed according to the motion. Can be changed. According to the displayed target device, the same operations as those of the above-described embodiments may be implemented.

In the present embodiment, once the remote control apparatus 10 photographs the target apparatuses and stores the relative positions and images of the target apparatuses, the remote control apparatus 10 does not face the target apparatus, or the remote control apparatus 10 of the remote control apparatus 10. Even when the camera is disabled, the operations of the aforementioned various embodiments can be implemented.

According to this embodiment, the user does not need to point the target device to control the remote control device in order to control the target devices. The remote control device only needs to be within a wireless communication range with the target devices to be controlled.

As the remote control apparatus 10 according to the embodiment of the present invention, any electronic device equipped with a camera, a display, and a short-range communication module may be used as long as appropriate software is installed. For example, when appropriate software is attached to a mobile terminal such as a mobile phone, it can be used as the remote control device 10. In particular, recently released touch screen phone equipped with a touch screen is suitable for use as a remote control device (10).

18 is a block diagram of a mobile terminal 100 usable as the remote control apparatus 10 according to an embodiment of the present invention. The mobile terminal 100 may include not only a mobile phone but also a smart phone, an MP3 player, a portable multimedia player (PMP), a digital broadcasting terminal, a notebook computer, a navigation device, and the like.

The illustrated mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, The interface unit 170 may include a controller 180, a power supply unit 190, and the like. 1 shows a mobile terminal having various components. However, not all illustrated components are required. The mobile terminal may be implemented by more components than the illustrated components, or the mobile terminal may be implemented by fewer components.

Hereinafter, the components will be described in turn.

The wireless communication unit 110 may include one or more components for wireless communication between the mobile terminal 100 and the wireless communication system or wireless communication between the mobile terminal 100 and a network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcast signals using various broadcasting systems, and in particular, digital multimedia broadcasting-terrestrial (DMB-T), digital multimedia broadcasting-satellite (DMB-S), and media forward link (MediaFLO). Digital broadcast signals can be received using digital broadcasting systems such as only), digital video broadcast-handheld (DVB-H), integrated services digital broadcast-terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcasting systems that provide broadcasting signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

In addition, the mobile communication module 112 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The wireless Internet module 113 is a module for wireless Internet access, and the wireless Internet module 113 can be built in or externally.

The short range communication module 114 refers to a module for short range communication. The short range communication module 114 may be used as the short range communication module 13 of FIG. 1.

In addition, the location information module 115 is a module for checking or obtaining the location of the mobile terminal. One example is the Global Position System (GPS) module. The GPS module receives location information from a plurality of satellites. Here, the location information may include coordinate information represented by latitude and longitude. For example, the GPS module can measure the exact time and distance from three or more satellites and accurately calculate the current position by triangulating three different distances. A method of obtaining distance and time information from three satellites and correcting the error with one satellite may be used. In particular, the GPS module can obtain not only the location of latitude, longitude, and altitude but also accurate time together with three-dimensional speed information from the location information received from the satellite.

Meanwhile, the A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes an image frame such as a still image or a moving image obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be provided according to the configuration aspect of the terminal. The camera 121 may be used as the camera 12 of FIG. 1.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. The processed voice data may be converted into a form transmittable to the mobile communication base station through the mobile communication module 112 and output in the call mode. The microphone 122 may implement various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for the user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like. In FIG. 1, the user input unit 130 may be a touch screen in which a touch panel is mounted on the display unit 14.

The sensing unit 140 detects a current state of the mobile terminal 100 such as an open / closed state of the mobile terminal 100, a location of the mobile terminal 100, presence or absence of a user contact, orientation of the mobile terminal, acceleration / deceleration of the mobile terminal, and the like. To generate a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, it is responsible for sensing functions related to whether the power supply unit 190 is supplied with power or whether the interface unit 170 is coupled to an external device.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, wired / wireless headset ports, external charger ports, wired / wireless data ports, memory card ports, ports for connecting devices with identification modules, audio input / output (I / O) ports, Video I / O (Input / Output) port, earphone port, etc. may be included.

Here, the identification module is a chip that stores various types of information for authenticating the use authority of the mobile terminal 100, and includes a user identification module (UIM) and a subscriber identify module (SIM). ), A Universal Subscriber Identity Module (“USIM”), and the like. In addition, the device equipped with the identification module (hereinafter, 'identification device') may be manufactured in the form of a smart card. Accordingly, the identification device can be connected to the terminal 100 through the port. The interface unit 170 receives data from an external device or receives power and transmits the data to each component inside the mobile terminal 100 or transmits the data inside the mobile terminal 100 to an external device.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 may include a display unit 151, a sound output module 152, an alarm unit 153, and the like.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in a call mode, a user interface (UI) or a graphic user interface (GUI) related to a call is displayed. When the mobile terminal 100 is in a video call mode or a photographing mode, the mobile terminal 100 displays a photographed and / or received image, a UI, or a GUI. The display unit 151 may be the display unit 14 of FIG. 1.

Meanwhile, as described above, when the display unit 13 and the touch pad form a mutual layer structure to form a touch screen, the display unit 151 may be used as an input device in addition to the output device. The display unit 151 may include a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a three-dimensional display. 3D display). In addition, two or more display units 151 may exist according to the implementation form of the mobile terminal 100. For example, an external display unit (not shown) and an internal display unit (not shown) may be simultaneously provided in the mobile terminal 100.

The sound output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. In addition, the sound output module 152 outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying occurrence of an event of the mobile terminal 100. Examples of events occurring in the mobile terminal include call signal reception, message reception, and key signal input. The alarm unit 153 may output a signal for notifying occurrence of an event in a form other than an audio signal or a video signal. For example, the signal may be output in the form of vibration. When a call signal is received or a message is received, the alarm unit 153 may output a vibration to inform this. Alternatively, when a key signal is input, the alarm unit 153 may output a vibration in response to the key signal input. Through the vibration output as described above, the user can recognize the occurrence of the event. Of course, the signal for notification of event occurrence may be output through the display unit 151 or the voice output module 152.

The memory 160 may store a program for processing and controlling the controller 180 and may provide a function for temporarily storing input / output data (for example, a phone book, a message, a still image, a video, etc.). It can also be done.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), RAM Random Access Memory (RAM) Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM) magnetic memory, It may include at least one type of storage medium of the optical disk. In addition, the mobile terminal 100 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The controller 180 typically controls the overall operation of the mobile terminal. For example, perform related control and processing for voice calls, data communications, video calls, and the like. In addition, the control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the controller 180 or may be implemented separately from the controller 180. The controller 180 may be the controller 11 of FIG. 1.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

The various embodiments described herein may be implemented in a computer readable recording medium using, for example, software, hardware or a combination thereof.

According to a hardware implementation, embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing the functions. It may be implemented by the controller 180.

In a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that allow at least one function or operation to be performed. The software code may be implemented by a software application written in a suitable programming language. In addition, the software codes may be stored in the memory 160 and executed by the control unit 180. [

1 shows a configuration of a remote control apparatus according to an embodiment of the present invention.

2A and 2B show an appearance of a remote control apparatus 10 according to an embodiment of the present invention.

3 and 4 illustrate a network configuration diagram between the remote control device 10 and an external device according to an embodiment of the present invention.

5 to 12 are exemplary views illustrating the operation of a remote control apparatus according to an embodiment of the present invention.

13 is a flowchart illustrating a remote control method according to an embodiment of the present invention.

14 and 15 illustrate an external device control method according to an embodiment of the present invention.

16 illustrates a target device control method according to an embodiment of the present invention.

FIG. 17 illustrates a method of controlling a remote control device without facing the target device according to an embodiment of the present invention.

18 shows a configuration of a mobile terminal that can be used as a remote control device according to an embodiment of the present invention.

Claims (26)

A camera unit for photographing the target device; A communication unit for receiving information of contents provided by the target device and transmitting a control signal to the target device; A display unit configured to display a composite image obtained by synthesizing a real image of the target device and a virtual image of the content; A user input unit configured to receive a user input for manipulating the virtual image; And A control unit which controls to display and manipulate the virtual image according to the received user input; Remote control device comprising a. The method of claim 1, The target device is a broadcast receiving device, and the information on the content is the remote control device channel information of the broadcast receiving device. The method of claim 1, The plurality of contents, the display unit remote control device for displaying a plurality of virtual images corresponding to the plurality of contents arranged in a predetermined pattern. The method of claim 3, wherein The control unit is a remote control device for moving the plurality of virtual images in accordance with the user input. The method of claim 1, And the user input unit is at least one of a touch screen, a key button, and a motion sensor. The method of claim 5, wherein the user input unit comprises a touch screen, The touch screen detects a user's drag gesture, and the controller moves the virtual image according to the drag gesture. The method of claim 5, wherein the user input unit comprises a motion sensor, The motion sensor detects a movement of the remote control device, and the controller moves the virtual image according to the detected movement. The method of claim 1, The user input unit receives a user input for selecting a virtual image for a predetermined content, and the control unit generates and transmits a control signal for displaying the content corresponding to the selected virtual image on the target device. The method of claim 1, The controller is configured to analyze the actual image of the target device to be photographed to determine the movement of the remote control device, and to move the virtual image according to the determined movement. The method of claim 1, The communication unit further receives information about an external device connected to the target device, and the display unit further displays a virtual image for the external device. The method of claim 10, And a plurality of external devices, the display unit to display a plurality of virtual images corresponding to the plurality of external devices arranged in a predetermined pattern. The method of claim 10, The user input unit receives a user input for selecting a virtual image for a predetermined external device, and the controller controls an external input switching signal for controlling an output signal of an external device corresponding to the selected virtual image to be output from the target device. Remote control device to produce. The method of claim 12, When the external input switching signal is transmitted to the target device, the communication unit receives information on the content provided by the external device, and the control unit is configured to the actual image photographing the target device and the content provided by the external device. A remote control device that controls the display of the composite image for the display. The method of claim 1, The user input unit further receives a user input to continuously display the composite image, The control unit is a remote control device for controlling to continuously display the composite image regardless of the direction facing the camera unit. The method of claim 1, The user input unit receives a zoom command for zooming in or out of the composite image, And the control unit controls to display the composite image by zooming in or out according to the zoom command. In the remote control method executed in the remote control device, Photographing the target device; Receiving information on content provided by the target device; Generating a virtual image of the content based on the received information of the content; Synthesizing and displaying the actual image of the target device and the virtual image of the content; Remote control method comprising a. The method of claim 16, Receiving a user input for manipulating the virtual image; And Manipulating and displaying the virtual image according to the received user input; Remote control method further comprising. The method of claim 16, The target device is a broadcast receiving device, and the information on the content is channel information. The method of claim 16, The content is a plurality, and the display step comprises the step of arranging and displaying a plurality of virtual images corresponding to the plurality of content in a predetermined pattern. The method of claim 17, The receiving of the user input includes detecting a user's drag gesture through a touch screen, Manipulating and displaying the virtual image comprises moving the virtual image according to the drag gesture. The method of claim 17, The receiving of the user input includes detecting a movement of the remote control apparatus through a motion sensor, Manipulating and displaying the virtual image comprises moving the virtual image according to the detected movement. The method of claim 16, Receiving a user input for selecting a virtual image for predetermined content; Generating a control signal for executing content corresponding to the selected virtual image on the target device; And Transmitting the control signal to the target device; Remote control method further comprising. The method of claim 16, Analyzing the actual image to determine movement of the remote control device; And Moving the virtual image according to the determined movement; Remote control method further comprising. The method of claim 16, Receiving information of an external device connected to the target device; And Displaying a virtual image of the external device based on the received information of the external device; Remote control method further comprising. The method of claim 24, Receiving a user input for selecting a virtual image for a predetermined external device; Generating an external input switching signal for controlling an output signal of an external device corresponding to the selected virtual image to be output from the target device; And Transmitting the external input switching signal to the target device; Remote control method further comprising. 26. The method of claim 25, Receiving information of content provided by the external device; And Synthesizing and displaying a virtual image of the actual image photographing the target device and the content provided by the external device; Remote control method further comprising.

Images