KR20150004052A - Method for controlling remote controller and multimedia device - Google Patents

Abstract

The present invention relates to a method for controlling a remote controller and a multimedia device. According to an embodiment of the present invention, the method for controlling the remote controller for controlling the multimedia device and at least one external device includes the steps of: detecting a motion from the outside; turning on an image sensor in case of detecting the motion from the outside; photographing an object and generating an image from the image sensor; transmitting the generated image to the multimedia device; receiving a code value corresponding to the transmitted image from the multimedia device; and transmitting a control signal corresponding to the received code value.

Description

[0001] METHOD FOR CONTROLLING REMOTE CONTROLLER AND MULTIMEDIA DEVICE [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote controller and a control method for a multimedia device, and more particularly, to a technique for controlling a multimedia device and an external device using a single remote controller.

With the development of electronic technology, more and more kinds of electronic devices are being developed and popularized. Recently, electronic devices have a function of receiving a signal of a remote controller for the convenience of the user. Accordingly, a user can control various electronic devices such as a TV, an audio player, a DVD player, an air conditioner, a fan, and the like using a remote controller.

Meanwhile, a recently developed multimedia device (for example, a TV, etc.) has a function of receiving various information by being connected to the Internet network. In order to control such a multimedia device and an electronic device (for example, STB, DB player, DVD player, PC, etc.) connected thereto, there is a problem that a remote controller is required for the number of devices.

Of course, a technology discussion for controlling a plurality of devices using a single remote controller has been made, but most of the problems are limited to one communication standard. In addition, when a plurality of devices are controlled by an infrared communication method using a remote controller, when a device such as an STB is located at a different place from the TV, the control signal of the remote controller is difficult to reach the device due to the linearity of the infrared signal Problems also exist.

An embodiment of the present invention provides a solution for controlling a plurality of devices using a single remote controller by utilizing a gesture recognition function by a multimedia device.

According to another aspect of the present invention, there is provided a method of using a remote controller with a built-in camera as a security device.

According to another aspect of the present invention, there is provided a method of controlling a remote controller for controlling a multimedia device and at least one external device, the method comprising: sensing an external motion; A step of receiving a code value corresponding to the transmitted image from the multimedia device, a step of receiving the code value corresponding to the transmitted image from the multimedia device, , And transmitting a control signal corresponding to the received code value.

The image may be transmitted to the multimedia device according to a first communication standard and the control signal may be transmitted in accordance with a second communication standard.

The control signal transmitting step may transmit the control signal toward a plurality of directions.

A remote controller for controlling at least one external device and a multimedia device according to an embodiment of the present invention includes a motion detection sensor for detecting an external motion and turning on an image sensor when an external motion is detected, A first communication module for transmitting the generated image to the multimedia device and receiving a code value corresponding to the transmitted image from the multimedia device, and a second communication module for receiving the code value corresponding to the received code value And a second communication module for transmitting a control signal.

The image may be transmitted to the multimedia device according to a first communication standard and the control signal may be transmitted in accordance with a second communication standard.

The second communication module may transmit the control signal toward a plurality of directions.

According to another aspect of the present invention, there is provided a method of controlling a multimedia device and a remote controller for controlling at least one external device, the method comprising: setting a specific mode; sensing an external motion; The method comprising the steps of: turning on an image sensor when the image sensor is in a first state, generating an image by photographing a subject in the image sensor, transmitting a control signal for controlling the multimedia device according to a first communication standard, And transmitting the image to the multimedia device according to the second communication standard.

The control signal may include a first control signal for turning on the multimedia device and a second control signal for storing an image transmitted from the remote controller.

The control signal may further include a third control signal for displaying an image transmitted from the remote controller.

A method of controlling a multimedia device connected to at least one external device according to an embodiment of the present invention includes receiving an image from a remote controller, extracting image information from the received image, Recognizing a gesture from the recognized gesture; detecting a code value matched to the recognized gesture; and transmitting the detected code value to a remote controller.

The method of controlling a multimedia device according to an embodiment of the present invention further includes building a database by classifying code values matched to a gesture into the at least one external device.

The image information may include at least one of outline information of a subject, motion vector information, or color information of an image.

The gesture recognition step may recognize the locus of the subject motion.

The gesture recognition step may recognize a shape of a part of a subject.

According to an embodiment of the present invention, a plurality of devices can be controlled using a single remote controller by utilizing a gesture recognition function by a multimedia device.

In addition, according to the embodiment of the present invention, a remote controller with a built-in camera can be utilized as a security device.

FIG. 1 is a view schematically showing a system including a remote controller and a plurality of devices controlled by the remote controller according to an embodiment of the present invention.
2 is a block diagram illustrating the configuration modules of a remote controller according to an embodiment of the present invention.
3 is a block diagram illustrating a configuration module of a multimedia device according to an exemplary embodiment of the present invention.
4 is a block diagram illustrating the controller of the multimedia device shown in FIG. 3 in greater detail.
5 is a diagram illustrating an example of a database of code values stored in the multimedia device according to an embodiment of the present invention.
6 is a flowchart illustrating an example of a method of controlling a device by gesture recognition using a remote controller according to an embodiment of the present invention.
7 is a schematic view illustrating a system including a remote controller and a plurality of devices controlled by the remote controller according to an embodiment of the present invention.
8 is a diagram for explaining an embodiment in which a plurality of remote controllers are present.
9 is a flowchart showing an example of a method of utilizing a remote controller according to an embodiment of the present invention as a security device.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffix "module" and " part "for components used in the following description are given merely for ease of description, and the" module "and" part "

Meanwhile, the multimedia device described in the present specification is an intelligent network TV that adds a computer support function to a broadcast reception function, for example, and is equipped with an Internet function while being faithful to a broadcast reception function, A remote controller and the like can be provided. With the support of wired or wireless internet function, it can be connected to internet and computer and can perform functions such as e-mail, web browsing, banking or game. A standardized general-purpose OS can be used for these various functions.

Furthermore, since the multimedia device described in the present invention can freely add or delete various applications, for example, on a general-purpose OS kernel, various user-friendly functions can be performed.

Furthermore, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings and the accompanying drawings, but the present invention is not limited to or limited by the embodiments.

As used herein, terms used in the present invention are selected from general terms that are widely used in the present invention while taking into account the functions of the present invention, but these may vary depending on the intention or custom of a person skilled in the art or the emergence of new technologies. In addition, in certain cases, there may be a term arbitrarily selected by the applicant, in which case the meaning thereof will be described in the description of the corresponding invention. Therefore, it is intended that the terminology used herein should be interpreted based on the meaning of the term rather than on the name of the term, and on the entire contents of the specification.

FIG. 1 is a view schematically showing a system including a remote controller and a plurality of devices controlled by the remote controller according to an embodiment of the present invention. Hereinafter, a system including a remote controller and a plurality of devices controlled by the remote controller according to an embodiment of the present invention will be described with reference to FIG.

1, the remote controller 100 according to an exemplary embodiment of the present invention can control the multimedia device 300 and other external devices 200. As shown in FIG.

The multimedia device 300 is, for example, an intelligent network TV that adds a computer support function to a broadcast reception function, and is an apparatus added to the broadcast reception function while adding Internet functions. For example, the multimedia device 300 corresponds to a network TV, a smart TV, a Hybrid Brodacast Broadband Television (HBBTV), an Internet TV, a Web TV, an IPTV (Internet Protocol Television)

The external device 200 refers to any electronic device other than the multimedia device 300. The external device 200 may be an electronic device connected to the multimedia device 300 in a wired or wireless manner (e.g., a DVD player, a BD player, Or an electronic device (air conditioner, washing machine, etc.) functioning independently of the multimedia device 300, but is not limited thereto. Of course, even when the external device 200 is an electronic device such as an air conditioner or a washing machine, the external device 200 may be connected to the multimedia device 300 via wire or wirelessly, depending on the function of the external device 200. [

The DVD player 210, the BD (Blu-ray Disc) player 220, and the sound bar 230 are exemplified as the external device 200 controlled by the remote controller 100, but the present invention is not limited thereto, May correspond to the external device 200 controlled by the remote controller 100. [

In this specification, the multimedia device 300 and the external device 200 may be collectively referred to as a device without distinguishing between them.

The remote controller 100 incorporates an image sensor capable of photographing a subject. The user can control the multimedia device 300 and the external device 200 by taking a gesture to control the function of any device toward the remote controller 100. [

The remote controller 100 is designed to transmit the image generated by the image sensor to the multimedia device 300 and to receive the IR code value corresponding to the image from the multimedia device 300. The multimedia device 300 is designed to recognize the gesture by analyzing the image transmitted from the remote controller 100.

Data transmission / reception between the remote controller 100 and the multimedia device 300 is performed through the first communication module. According to an embodiment, in the first communication module, image transmission from the remote controller 100 to the multimedia device 300 is performed according to a first communication standard (e.g., WLAN) Transmission of the IR code value to the controller 100 may be made according to the second communication standard (for example, Bluetooth).

The IR code value is mapped to a specific function of a specific device. The remote controller 100 controls the external device 200 by transmitting an IR signal corresponding to the received IR code value. The IR signal of the remote controller 100 is transmitted through the second communication module. Within the second communication module, transmission of the IR signal may be in accordance with a third communication standard (e.g., infrared communication).

Of course, if the device to be controlled by the user is the multimedia device 300 itself, the multimedia device 300 analyzes the image transmitted from the remote controller 100 to recognize the gesture, and generates a control signal matched to the gesture To control the configuration modules in the multimedia device 300.

In addition, when the remote controller 100 is set in the security mode by the user, the remote controller 100 may be utilized as a security device in the absence of the user.

2 is a block diagram illustrating the configuration modules of a remote controller according to an embodiment of the present invention. Hereinafter, the configuration modules of the remote controller according to an embodiment of the present invention will be described with reference to FIG. However, FIG. 2 is an embodiment for convenience of description, and a configuration in which a configuration module other than the configuration modules shown in FIG. 2 is included, or in which some configuration modules are omitted belong to the scope of the present invention.

2, a remote controller 100 according to an exemplary embodiment of the present invention includes an audio input unit 110, a voice recognition unit 120, a controller 130, a motion detection sensor 140, an image sensor 150, a communication unit 160, a storage unit 170, and a power unit (not shown). The remote controller 100 may further include a display unit for displaying the current state of the remote controller 100 and may not include the voice input unit 110 and the voice recognition unit 120 according to the embodiment.

The motion detection sensor 140 detects an external motion. The motion detection sensor 140 may include, for example, an infrared sensor and / or a thermal sensor, and may sense any movement outside the remote controller 100.

The motion detection sensor 140 may be designed to turn on the image sensor 150 when an external motion is detected. This is because unnecessary standby power is consumed when the image sensor 150 is always in the ON state even when the user is absent. Alternatively, the motion sensor 140 may be configured to inform the controller 130 that an external motion has been detected, and the image sensor 150 may be turned on under the control of the controller 130.

The image sensor 150 converts an optical signal obtained from a subject into an electrical signal to generate an image. The generated image includes a still image or a moving image.

The remote controller 100 may include an audio input unit 110 and a voice recognition unit 120 when the remote controller 100 is equipped with a voice recognition function. The speech recognition unit 120 may be implemented to be included in the control unit 130. [

The voice input unit 110 receives a voice uttered by the user. The voice input unit 110 includes a microphone for sensing a user's voice and converting the voice into an electrical voice signal. A plurality of voice input units 110 may be provided.

The speech recognition unit 120 removes the ambient noise signal from the speech signal obtained by the speech input unit 110, and then converts the speech signal from which the noise signal is removed into text. The voice recognition unit 120 extracts the feature information from the waveform of the voice signal, and can convert the voice signal into text based on the extracted feature information. Examples of the feature information to be extracted include pause information included in the input voice, information on the number of syllables of the input voice, frequency information of the inputted voice, high-level information of the voice to be input, and the like.

For example, when the user has uttered a voice called " TV channel up ", the voice recognition unit 120 can convert the voice signal into a text called " TV channel up " Then, a control signal matching the converted text is detected. The storage unit 170 may previously store mapping data of a text and a control signal.

The image sensor 150 may be designed to be turned on by the user's voice recognized by the voice input unit 110 and the voice recognition unit 120. [

The communication unit 160 is designed to be capable of communicating with arbitrary devices according to a wireless communication protocol. The communication unit 160 includes an RF module 162 designed to perform data communication with the multimedia device 300 and an IR module 164 designed to perform infrared communication with other external devices 200. When the multimedia device 300 is controlled by the remote controller 100, the multimedia device 300 may communicate with the multimedia device 300 through the IR module 164 as well. According to an embodiment, the communication unit 140 may be designed to transmit and receive data to and from an external server or the like via an Internet network or the like.

The storage unit 170 may store a program for various signal processing of the control unit 130, a program, an algorithm, an application, and the like necessary for converting the received voice signal into text. In addition, the storage unit 170 stores various OSs, middleware, and platforms, for example. The storage unit 170 may also store or temporarily store the image generated by the image sensor 150 and the voice signal generated by the voice input unit 110. The storage unit 170 may be designed to be included in the control unit 130.

The controller 130 controls the overall operation of the remote controller 100. The control unit 130 may include a control unit 130 including a voice input unit 110, a voice recognition unit 120, a motion detection sensor 140, an image sensor 150, a communication unit 160, a storage unit 170, The operation of the configuration modules can be controlled.

In particular, an embodiment in which the remote controller 100 is set in the security mode will be described as follows.

The remote controller 100 can be set to the security mode by the user. When the remote controller 100 is set to the security mode, the motion sensor 140 detects an external motion and turns on the image sensor 150 when motion is detected. According to the embodiment, the image sensor 150 may be turned on by the control of the controller 130, which has been notified of the motion detection from the motion detection sensor 140.

The image sensor 150 photographs a subject to generate an image. The generated image may be a still image or a moving image according to predetermined conditions. When the generated image is a still image, it may be implemented such that a plurality of still images are generated by photographing the subject at predetermined intervals so as to be faithful to the crime prevention function.

The control unit 130 controls the communication unit 160 to transmit a control signal for controlling the multimedia device 300. [ The control signal may be transmitted through the IR module 164 of the communication unit 160.

In addition, the controller 130 controls the communication unit 160 to transmit the generated image to the multimedia device 300. The generated image may be transmitted through the RF module 164 of the communication unit 160.

The multimedia device 300 is designed to store the image transmitted from the remote controller 100 and to display the image as the case may be.

The control signal for controlling the multimedia device 300 includes a first control signal for turning on the multimedia device 300 and a second control signal for storing an image transmitted from the remote controller 100. Optionally, the control signal may comprise a third control signal for displaying an image transmitted from the remote controller 100.

The multimedia device 300 turns on the power according to the first control signal, receives the image from the remote controller 100, and stores the received image in the storage unit according to the second control signal. Also, the received image may be displayed through the video output unit according to the third control signal. When the multimedia device 300 displays an image according to the third control signal, there may be an effect of imposing a psychological burden on the intruder.

The remote controller 100 transmits the control signal and transmits the generated image to the multimedia device 300 after the external motion is no longer detected or after the subject has been photographed for a preset time by the user Can be designed.

Alternatively, the remote controller 100 may be designed to transmit the control signal and transmit the image generation completed portion to the multimedia device 300 while continuing to photograph the subject. In this case, the remote controller 100 transmits an image to the multimedia device 300 a plurality of times until the object imaging is completed, and the multimedia device 300 can receive and store the image a plurality of times.

An embodiment for controlling a device through gesture recognition using the remote controller 100 will be described below.

When the user turns on the image sensor 150 using a local key or the like provided in the remote controller 100 or when the image sensor 150 is turned on by detecting the motion in the motion sensor 140, The sensor 150 photographs a subject and generates an image.

The control unit 130 controls the communication unit 160 to transmit the generated image to the multimedia device 300. The generated image may be transmitted through the RF module 164 of the communication unit 160.

The multimedia device 300 may be designed to detect the gesture by analyzing the image transmitted from the remote controller 100 and to detect the IR code value matched to the recognized gesture and transmit it to the remote controller 100. [

The control unit 130 controls the communication unit 160 to transmit the IR signal corresponding to the IR code value. The IR signal may be transmitted through the IR module 164 of the communication unit 160. Since the IR code value is mapped to a specific function of a specific device, the specific device receives the IR signal and a specific function is executed accordingly.

In addition, according to an embodiment of the present invention, when a user makes a video chat or video call with a third party through the multimedia device 300, the image of the user generated in the image sensor 150 in the remote controller 100 To the multimedia device 300 so that the user can freely move the remote controller 100 to perform a video chat or a video call.

In addition, according to the remote controller 100 according to the embodiment of the present invention, a plurality of image sensors 150 may be provided. When a plurality of image sensors 150 are provided, for example, one image sensor 150 is designed to acquire, as an image, distance information and / or skeleton information of the subject, (150) may be designed to acquire the color information of the object as an image.

3 is a block diagram illustrating a configuration module of a multimedia device according to an exemplary embodiment of the present invention. Hereinafter, referring to FIG. 3, the configuration modules of the multimedia device according to an embodiment of the present invention will be described. However, FIG. 3 is an embodiment for convenience of description, and a configuration in which a configuration module other than the configuration modules shown in FIG. 3 is included, or in which some configuration modules are omitted is also within the scope of the present invention.

3, the multimedia device 300 according to an exemplary embodiment of the present invention includes a broadcast receiving module 301, a demodulator 302, a network interface 303, an external device interface 304, A storage unit 305, a control unit 306, a video output unit 307, an audio output unit 308, a power supply unit 309, a user interface 311, and the like. The broadcast receiving module 301 may be designed, for example, as an RF tuner or an interface for receiving broadcast data from an external device such as an STB.

The broadcast receiving module 301 can receive an RF broadcast signal of a single carrier according to an Advanced Television System Committee (ATSC) scheme or an RF broadcast signal of a plurality of carriers according to a DVB (Digital Video Broadcasting) scheme.

The demodulation unit 302 receives the digital IF signal DIF converted by the broadcast reception module 301 and performs a demodulation operation.

For example, when the digital IF signal output from the broadcast receiving module 301 is of the ATSC scheme, the demodulator 302 performs 8-VSB (8-Vestigal Side Band) demodulation, for example. Also, the demodulator 302 may perform channel decoding.

The external device interface unit 304 is an interface that enables data communication between the external device and the multimedia device 300. The external device interface unit 304 can be connected to an external device such as a DVD (Digital Versatile Disk), a Blu ray, a game device, a camera, a camcorder, a computer (notebook) The external device 200 may be connected to the multimedia device 300 via an external device interface 304. The external device interface unit 304 transfers the video, audio, or data signals input from the connected external device to the control unit 306. Also, the control unit 306 can output the processed video, audio, or data signal to an external device.

The external device interface unit 304 may include a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, an S-video terminal (analog), a DVI ) Terminal, an RGB terminal, a D-SUB terminal, and the like. The external device interface unit 304 may be connected to a wireless network such as Bluetooth, WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access) HSDPA (High Speed Downlink Packet Access) communication standard can be used. The external device interface unit 304 is connected to the remote controller 100 through a wireless network and can transmit and receive data to and from the remote controller 100.

The network interface unit 303 provides an interface for connecting the multimedia device 300 to a wired / wireless network including the Internet network. The network interface unit 303 may include an Ethernet terminal or the like for connection to a wired network and may be a WLAN (Wireless LAN) (Wi- Fi wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), and HSDPA (High Speed Downlink Packet Access) communication standards.

The network interface unit 303 can transmit or receive data to another user or another electronic device via the network to which it is connected or another network linked to the connected network.

The storage unit 305 may store a program for each signal processing and control in the control unit 306 or may store a signal-processed video, audio, or data signal. The storage unit 305 may also function to temporarily store video, audio, or data signals input from the external device interface unit 304 or the network interface unit 303. The storage unit 305 stores various OSs, middleware, and platforms, for example. In addition, the storage unit 305 stores a program or an application for recognizing the gesture by analyzing the image received from the remote controller 100.

The user interface 311 transmits a signal input by the user to the control unit 306 or transmits a signal from the control unit 306 to an external device (for example, a remote control device). For example, the user interface 311 may control signals such as power on / off, channel selection, and screen setting from a remote control device in accordance with various communication methods such as an RF (Radio Frequency) communication method and an infrared And transmits the control signal from the control unit 306 to the remote control device. The remote control device may be a remote controller 100 or a separate remote controller oriented in the multimedia device 300. The user interface 311 can receive a control signal transmitted from the remote controller 100 according to the IR communication method.

The control unit 306 demultiplexes the input stream or processes the demultiplexed signals through the broadcast receiving module 301, the demodulating unit 302, the network interface unit 303 or the external device interface unit 304 And generate and output signals for video or audio output.

The video output unit 307 converts a video signal, a data signal, an OSD signal processed by the control unit 306, a video signal and a data signal received from the external device interface unit 304 into R, G, and B signals, respectively Thereby generating a driving signal. The video output unit 307 may be a PDP, an LCD, an OLED, a flexible display, a 3D display, or the like.

The audio output unit 308 receives a voice processed signal, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal from the control unit 306, and outputs the voice. The audio output unit 308 may be implemented by various types of speakers.

The power supply unit 309 supplies corresponding power to the multimedia device 300. In particular, a control unit 306 that can be implemented in the form of a system on chip (SOC), a video output unit 307 for video display, and an audio output unit 308 for audio output .

4 is a block diagram illustrating the controller of the multimedia device shown in FIG. 3 in greater detail.

4, the control unit 306 of the multimedia device 300 includes a demultiplexing unit 310, an image processing unit 320, a GUI generating unit 330, a mixer 340, a frame rate converting unit 350, and a formatter 360. It is also within the scope of the present invention to further design a voice processing unit (not shown) and a data processing unit (not shown).

The demultiplexer 310 demultiplexes the input stream. For example, when an MPEG-2 TS is input, it can be demultiplexed into video, audio, and data signals, respectively.

The image processing unit 320 may perform image processing of the demultiplexed image signal. To this end, the image processing unit 320 may include a video decoder 322 and a scaler 324. The video decoder 322 decodes the demultiplexed video signal and the scaler 324 performs scaling so that the resolution of the decoded video signal can be output from the video output unit. The video signal decoded by the video processor 320 is input to the mixer 340.

The GUI generating unit 330 generates a GUI signal according to a user input or itself. Accordingly, the mixer 340 can mix the GUI signal generated by the GUI generating unit 330 and the decoded video signal processed by the image processing unit 320. The GUI generation unit 300 includes an OSD generation unit for generating an OSD (On Screen Display) signal including environment setting information such as volume control or screen control.

The mixed signal is provided to formatter 360. Since the decoded broadcast video signal or the external input signal and the OSD signal are mixed, the OSD can be overlaid on the broadcast image or the external input image.

A frame rate converter (FRC) 350 can convert a frame rate of an input image. For example, a frame rate of 60 Hz is converted to 120 Hz or 240 Hz. When converting the frame rate of 60 Hz to 120 Hz, it is possible to insert the same first frame between the first frame and the second frame, or insert the third frame predicted from the first frame and the second frame. When converting a frame rate of 60 Hz to 240 Hz, it is possible to insert three more identical frames or insert three predicted frames. On the other hand, it is also possible to maintain the input frame rate without performing another conversion.

The formatter 360 receives the output signal of the frame rate conversion unit 350, changes the format of the signal to be suitable for the video output unit, and outputs the signal. For example, the R, G, and B data signals may be output as low voltage differential signaling (LVDS) signals or mini-LVDS signals.

Particularly, the following description will be given in connection with the embodiment of the present invention.

The control unit 306 is designed to recognize the gesture by analyzing the image received from the remote controller 100. A process of recognizing the gesture by the multimedia device 300 will now be described. It is assumed that the control mode of the specific device is set in advance before the image is transmitted from the remote controller 100. [ That is, when a user enters an external input through a remote controller 100, a separate remote controller oriented in the multimedia device 300, or a local key provided in the multimedia device 300, The process of selecting the control mode may be preceded.

The control unit 306 extracts image information from the image received through the external device interface unit 204. [ The image information includes at least one of outline information of a subject, motion vector information, or color information of an image. That is, the control unit 306 can extract the outline information of the subject, which is distinguished from the background, from the received image. The outline information may include information for distinguishing the outline of each part of the body (for example, a finger, a hand bottom, an elbow, etc.) of the subject. Also, the control unit 306 can extract motion vector information from the received image, which can identify the locus of movement of the subject. The control unit 306 may extract information on each color R (Red), G (Green), and B (Blue) of the image from the received image.

The control unit 306 recognizes the gesture based on the extracted image information. The recognized gesture may include a first type gesture recognized by the locus of the subject motion and a second type gesture recognized by the shape of a part of the subject. A first type of gesture is a gesture that is recognized by a user using a part of his / her body to move in a specific direction (e.g., a movement using a hand to draw a circle in a clockwise direction) and tracking the motion of the subject. A second type of gesture is a gesture that is recognized by a user in a particular form (for example, a form of lifting a thumb with a hand held fist) using a portion of the body.

The storage unit 305 stores a database constructed by dividing the IR code values matched to the gestures into a plurality of external devices.

The control unit 306 accesses the storage unit 305 to detect the IR code value matched to the recognized gesture and transmits the detected IR code value to the remote controller 100 through the external device interface unit 204 do.

5 is a diagram illustrating an example of a database of code values stored in the multimedia device according to an embodiment of the present invention.

5, the storage unit 305 of the multimedia device 300 divides the code values matched to the gesture into a plurality of devices (for example, TV, STB, etc.) to construct a database, Lt; RTI ID = 0.0 > memory address. That is, the IR code value for the functions of the TV is stored in the first address, and the IR code value for the functions of the STB is stored in the second address.

For example, for user's convenience, a database can be constructed so that the same gesture is mapped to the channel up / down functions commonly present in TV and STB. When the TV control mode is set by the user and a gesture in which the trajectory of the subject motion draws a circle in the clockwise direction is recognized, the control unit 306 can access the address 1 of the storage unit 305 and detect the value 0X01. On the other hand, if the same gesture is recognized but the STB control mode is set by the user, the control unit 306 can access Address 2 of the storage unit 305 and detect the value 0X51.

Also, the gesture stored in the database may include at least one of the first type gesture G1 or the second type gesture G2 described above.

6 is a flowchart illustrating an example of a method of controlling a device by gesture recognition using a remote controller according to an embodiment of the present invention. Hereinafter, an example of a method of controlling a device by gesture recognition using a remote controller according to an embodiment of the present invention will be described with reference to FIG.

First, the image sensor 150 of the remote controller 100 is turned on (S401). The image sensor 150 may be turned on by the user pressing a local key provided on the remote controller 100 or may be turned on by the motion sensor 140 or the controller 130 if motion is detected by the motion sensor 140. [ It may be turned on by control.

The image sensor 150 photographs a subject to generate an image (S402). The remote controller 100 may compress the image according to a predetermined communication standard to transmit the generated image to the multimedia device 300. [

The remote controller 100 transmits the generated image to the multimedia device 300 (S403). If the generated image is compressed, the remote controller 100 may transmit the compressed image file to the multimedia device 300.

The multimedia device 300 receives the image transmitted from the remote controller 100 and extracts image information from the received image (S404). The image information includes at least one of outline information of a subject, motion vector information, or color information of an image.

Then, the multimedia device 300 recognizes the gesture based on the extracted image information (S405). The recognized gesture may include a first type gesture recognized by the locus of the subject motion and a second type gesture recognized by the shape of a part of the subject.

The multimedia device 300 accesses the storage unit to detect an IR code value matched to the recognized gesture (S406), and transmits the detected IR code value to the remote controller 100 (S407).

The remote controller 100 receives the IR code value transmitted from the multimedia device 300 and transmits an IR signal corresponding to the received IR code value (S407).

Since the IR signal is mapped to a specific function of the specific device, the external device 200 specified by the IR signal receives the IR signal and executes a specific function accordingly (S408).

According to the embodiment, after the specific function is executed in step S408, the external device 200 transmits its status information to the multimedia device 300, and the multimedia device 300 is designed to receive the status information . In this case, the multimedia device 300 generates and displays a GUI based on the received status information, and the user can know the current status of the external device 200 through the GUI. For example, when the specific function executed in step S408 is a function of lowering the set temperature of the air conditioner, the user can control the operation of the air conditioner 300 through the GUI displayed on the multimedia device 300, It can be confirmed that the set temperature of the temperature sensor is lowered.

In the case of the mode for controlling the multimedia device 300 according to the embodiment, the multimedia device 300 recognizes the gesture in step S405, generates a control signal corresponding to the recognized gesture, Function. In this case, steps S406 to S408 are omitted. That is, the control unit of the multimedia device 300 can recognize the gesture from the extracted image information, and generate control signals corresponding to the recognized gesture to control the configuration modules in the multimedia device 300.

7 is a schematic view illustrating a system including a remote controller and a plurality of devices controlled by the remote controller according to an embodiment of the present invention.

As shown in FIG. 7, the remote controller 100 according to an embodiment of the present invention can control the multimedia device 300 and other external devices 200. The DVD player 210, the BD (Blu-ray Disc) player 220, the sound bar 230, and the air conditioner 240 are illustrated as the external device 200 controlled by the remote controller 100, It does not.

The remote controller 100 includes a plurality of IR signal transmission units 101 and transmits the IR signals in a plurality of directions through the plurality of IR signal transmission units 101. [

For example, when the user desires to control the air conditioner 240, the user takes a gesture related to the function of the air conditioner 240, and the remote controller 100 photographs the user and transmits the generated image to the multimedia device 300 do. Then, the remote controller 100 receives the IR code value detected by the gesture recognition in the multimedia device 300, and transmits the IR signal corresponding to the received IR code value. In this case, the IR signal is transmitted in a plurality of directions via the plurality of IR signal transmitters 101. The IR signal transmitted through the specific IR signal transmitting unit 101 among the plurality of IR signal transmitting units 101 is received by the air conditioner 240 and the air conditioner 240 is controlled.

In the case of the conventional remote controller, there is an inconvenience that the control signal must be transmitted to the device to be controlled while the user holds the remote controller in the hand due to the linearity of the IR signal.

According to the embodiment, it is not necessary for the user to find where the remote controller 100 is, and if the user wishes to control an arbitrary device, the remote controller 100 receives the IR signal corresponding to the gesture of the user It is possible to easily control the device. The user merely needs to arrange the remote controller 100 in an arbitrary place in consideration of the positions of the plurality of devices 200 and 300 to be controlled.

The IR signal transmitter 101 is embodied in the IR module 164 of FIG.

8 is a diagram for explaining an embodiment in which a plurality of remote controllers are present.

As shown in FIG. 8, a case where a total of three remote controllers 100A, 100B, and 100C exist in the home will be described as an example. For convenience, the remote controller 100A is referred to as a first remote controller, the remote controller 100B as a second remote controller, and the remote controller 100C as a third remote controller. When three remote controllers 100A, 100B, and 100C are referred to collectively, they will be referred to as a remote controller 100. [

When there is a barrier between the remote controller 100 and the devices 200 and 300, the IR signal is hardly transmitted to the devices 200 and 300 due to the linearity of the IR signal transmitted from the remote controller 100 . Accordingly, the user can install and use at least one remote controller 100 in the living room and the room, respectively, as needed.

8, a multimedia device 300 and a DVD player 210 connected to the multimedia device 300 are installed in the living room. An air conditioner 240 is installed in a first room, and a sound bar 230 is installed in a second room. .

When the user desires to control the multimedia device 300 and the DVD player in the living room, the remote controller 100A can be used to control the multimedia device 300 and the DVD player. Similarly, when the user wishes to control the air conditioner 240 in the first room, it can be controlled using the second remote controller 100B. When the user wants to control the sound bar 230 in the second room, And can be controlled using the third remote controller 100C.

On the other hand, for example, when the user wants to control the air conditioner 240 installed in the first room in the living room, since the living room and the first room are blocked by the wall, the IR signal transmitted from the first remote controller 100A The air conditioner 240 can not be controlled.

In this case, the No. 1 remote controller 100A transmits the IR code value matching the specific function of the air conditioner 240 to the No. 2 and No. 3 remote controllers 100B and 100C through the RF module 162, And the third remote controllers 100B and 100C receive the IR code value and transmit the corresponding IR signal. The air conditioner 240 can receive the IR signal transmitted from the No. 2 remote controller 100B and accordingly the specific function of the air conditioner 240 is executed. If the first remote controller 100A knows in advance the information that the air conditioner 240 and the second remote controller 100B are installed in the same place, the first remote controller 100A determines that the IR The code value may be designed to be transmitted only to the second remote controller 100B.

9 is a flowchart showing an example of a method of utilizing a remote controller according to an embodiment of the present invention as a security device. Hereinafter, an example of a method of utilizing a remote controller according to an embodiment of the present invention as a security device will be described with reference to FIG.

Referring to FIG. 9, the remote controller 100 is set to the security mode by the user (S601).

The remote controller 100 turns on the image sensor 150 when any motion is detected. The remote controller 100 may be designed such that the image sensor 150 is turned on under the control of the motion sensor 140 or the controller 130 when any motion is detected by the motion sensor 140. [

The remote controller 100 takes an image of a subject and generates an image (S603). The generated image may be a still image or a moving image according to predetermined conditions. When the generated image is a still image, it may be implemented such that a plurality of still images are generated by photographing the subject at predetermined intervals so as to be faithful to the crime prevention function.

The remote controller 100 transmits a control signal for controlling the multimedia device 300 (S604). The control signal may be an IR signal transmitted from the IR module 164 in the remote controller 100. The control signal includes a first control signal for turning on the multimedia device 300 and a second control signal for storing an image transmitted from the remote controller 100. The control signal may include a third control signal for displaying an image transmitted from the remote controller 100.

The multimedia device 300 turns on the power according to the first control signal (S605).

The remote controller 100 transmits the generated image to the multimedia device 300 (S606). The image may be transmitted via the RF module 162 in the remote controller 100.

The multimedia device 300 receives the image and stores the received image according to the second control signal (S607). If the control signal includes the third control signal, the multimedia device 300 displays the received image (S607).

According to the embodiment, the remote controller 100 simultaneously transmits the first control signal to the third control signal in step S604, and the multimedia device 300 receives the first control signal through the third control signal at once can do. In this case, the multimedia device 300 turns on the power according to the first control signal in the step S605, and temporarily stores the second and third control signals in the storage unit. The multimedia device 300 may be designed to store an image according to the second control signal when the image is received from the remote controller 100 and display the image according to the third control signal.

Alternatively, according to the embodiment, the remote controller 100 transmits only the first control signal in step S604, and the multimedia device 300 is designed to turn on the power in accordance with the first control signal in step S605 It is possible. Then, the remote controller 100 may transmit the second control signal and the third control signal together while transmitting the image in step S606, and the multimedia device 300 receives the image and, in accordance with the second control signal, Stores the image, and displays the image according to the third control signal.

The remote controller 100 transmits the control signal (S604) after completing the shooting of the subject for a preset time by the user (S602) until the external motion is no longer detected, (S606). ≪ / RTI >

Alternatively, the remote controller 100 may be designed to transmit the control signal and transmit the image generation completed portion to the multimedia device 300 while continuing to photograph the subject. In this case, the remote controller 100 may transmit the image to the multimedia device 300 a plurality of times until the object photographing is completed. That is, the steps S603, S606, and S607 may be repeated a plurality of times, or some of the steps may be performed in an overlapping manner.

Although the drawings have been described for the sake of convenience of explanation, it is also possible to combine the embodiments described in the drawings to design a new embodiment. It is also within the scope of the present invention to design a computer-readable recording medium in which a program for executing the previously described embodiments is recorded according to the needs of those skilled in the art.

The multimedia device and its operation method according to the present invention are not limited to the configuration and method of the embodiments described above but can be applied to all or some of the embodiments so that various modifications can be made. Or may be selectively combined.

Meanwhile, the method of operating the multimedia device of the present invention can be implemented as a code that can be read by a processor on a recording medium readable by a processor included in the display device. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the present invention.

In this specification, both the invention and the invention of the method are explained, and the description of the two inventions can be supplementarily applied as necessary.

100: remote controller
200: External device
300: Multimedia device

Claims (14)

A control method of a remote controller for controlling a multimedia device and at least one external device,
Sensing an external motion;
When the external motion is detected, turning on the image sensor;
Capturing a subject from the image sensor to generate an image;
Transmitting the generated image to a multimedia device;
Receiving a code value corresponding to the transmitted image from the multimedia device; And
And transmitting a control signal corresponding to the received code value. The method according to claim 1,
Wherein the image is transmitted to a multimedia device according to a first communication standard and the control signal is transmitted according to a second communication standard. The method according to claim 1,
Wherein the control signal transmitting step transmits the control signal toward a plurality of directions. A remote controller for controlling a multimedia device and at least one external device,
A motion detection sensor that detects an external motion and turns on an image sensor when an external motion is detected;
An image sensor for photographing a subject to generate an image;
A first communication module for transmitting the generated image to the multimedia device and receiving a code value corresponding to the transmitted image from the multimedia device; And
And a second communication module for transmitting a control signal corresponding to the received code value. 5. The method of claim 4,
Wherein the image is transmitted to a multimedia device according to a first communication standard and the control signal is transmitted according to a second communication standard. 5. The method of claim 4,
And the second communication module transmits the control signal toward a plurality of directions. A control method of a remote controller for controlling a multimedia device and at least one external device,
Setting a specific mode;
Sensing an external motion;
When the external motion is detected, turning on the image sensor;
Capturing a subject from the image sensor to generate an image;
Transmitting a control signal for controlling the multimedia device according to a first communication standard; And
And transmitting the generated image to a multimedia device according to a second communication standard. 8. The method of claim 7,
Wherein the control signal includes a first control signal for turning on the multimedia device and a second control signal for storing an image transmitted from the remote controller. 9. The method of claim 8,
Wherein the control signal further comprises a third control signal for displaying an image transmitted from the remote controller. A method of controlling a multimedia device connected to at least one external device,
Receiving an image from a remote controller;
Extracting image information from the received image;
Recognizing a gesture from the extracted image information;
Detecting a code value matching the recognized gesture; And
And transmitting the detected code value to a remote controller. 11. The method of claim 10,
And constructing a database by classifying code values matched to the gesture by the at least one external device. 11. The method of claim 10,
Wherein the image information includes at least one of outline information of a subject, motion vector information, or color information of an image. 11. The method of claim 10,
Wherein the gesture recognition step recognizes a trajectory of a subject motion. 11. The method of claim 10,
Wherein the gesture recognition step recognizes a shape of a part of a subject.

Images