JP2011239279A - Remote control device and remote control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize detection of a starting point of a user's gesture operation accurately even with a cheap controlled device while suppressing battery consumption of a remote control.SOLUTION: While a key code signal from an infrared remote control is received by 101, the remote control itself emitting light is photographed with a camera 103. A position detection part 104, using timing information received by a decode part 102, detects a position of the remote control easily and with high accuracy by extracting a portion with a brightness change from the photographed picture. A gesture recognition part recognizes the user's operation by tracking a motion of the image at the position. A control part 110 receives both the control by the received remote control key code and the control by the recognized user's operation.

Description

  The present invention relates to a remote control device and a remote control method for remotely controlling a controlled device by an operation of a user at a remote position.

Conventionally, an infrared remote controller has been used to remotely control controlled devices at remote locations such as a television (hereinafter referred to as a television), a video recorder, and an air conditioner. In such a remote control method using an infrared remote controller, an infrared remote controller having a plurality of buttons (hereinafter referred to as a remote controller) has been used.
However, in recent years, the operation method has become more complicated as the functions of the controlled device become higher. For example, in an operation method by sequential pointer movement and selection by a cursor button, which is a general remote control operation, usability has deteriorated. On the other hand, an operation method in which a user's (operator) 's gesture or gesture (gesture) is photographed with a camera to recognize the image, and based on the result, the user's control intention is judged and the controlled device is controlled. Has been proposed.

JP-A-6-75695 JP 2007-189613 A JP 2010-17405 A JP 2007-193403 A Japanese Patent No. 4318056

  When discriminating user gestures and gestures as described above by image recognition, prevent erroneous recognition due to the influence of bright objects such as fluorescent lights in the background of the user, humans other than the user, movable objects, etc. is important.

For example, Patent Literature 1 and Patent Literature 2 disclose a control method in which a remote controller equipped with a light emitting element is photographed with a camera provided in a body of a controlled device, and a cursor is moved by detecting the position and movement of the camera. Yes. In this case, according to the control methods described in Patent Document 1 and Patent Document 2, the position of the remote control can be recognized by making the remote control always emit light.
However, in order to prevent the influence of illumination other than the remote control or external light, the light from the remote control must be output with high luminance. Therefore, there is a problem that the power consumption of the remote control is severe if the remote control is always lit.
Further, in the device of Patent Document 1, the light from the remote control is infrared of a specific wavelength, and the camera of the controlled device is provided with an infrared filter that cuts visible light to reduce the influence of external light. Recognition accuracy has been improved. However, a camera provided with such a filter cannot capture general visible light. For this reason, there is also a problem that this camera cannot be used as an imaging means for, for example, a videophone.

In contrast to Patent Document 1 and Patent Document 2, in Patent Document 3 and Patent Document 4, infrared light emitted from the controlled device body or displayed image is displayed using a camera provided on the remote control side. A pointing device that inputs an instruction direction, a position, and the like of a remote controller by photographing is disclosed.
However, in the apparatuses described in Patent Documents 3 and 4, it is necessary to mount an imaging unit such as a CCD sensor or a CMOS sensor and an analysis processing unit for an image captured by the imaging unit on the remote controller side, and the remote controller becomes expensive. There is a problem that power consumption is large.

Furthermore, as described above, many control methods using gesture operations have been proposed in which the user does not hold the remote control in his / her hand and recognizes the movement of the hand itself. For example, in Patent Document 5, a user's image is taken with a depth by a three-dimensional camera (3D camera) provided in the controlled device, and an operation area plane is virtually set on a predetermined distance in the depth direction. In addition, a remote control device is disclosed that performs control by recognizing the shape and motion of an object that projects forward from a set plane.
In Patent Document 5, a remote controller is not required on the user side. Therefore, from among images in which random movements of the user when the remote control operation is not performed and movements of objects such as the user's background are mixed, By recognition, it is necessary to extract only the gesture when the user intends to operate the controlled device. In order to facilitate this, a 3D camera capable of measuring the depth information of the photographic pixel, such as a TOF (Time Of Flight) sensor, is required, or a large amount of images by a high-performance CPU (Central Processing Unit). There is a problem that it is necessary to perform shape recognition in the processing, and the controlled device is expensive. In addition, since such a 3D camera responds only to infrared rays having a specific wavelength, there is also a problem that a normal image cannot be taken. That is, one image pickup means cannot be used for both remote control operation recognition and videophone.

  The present invention has been made in order to solve the above-described problems. The object of the present invention is to make it possible to detect a user's operation accurately even with an inexpensive controlled device while suppressing battery consumption of the remote controller. A control device and a remote control method are provided.

  In order to solve the above-mentioned problems, the present invention provides a receiving means for receiving an infrared signal emitted from a remote controller by a user operation of an operator, and a reception timing detection by decoding the infrared signal received by the receiving means as a remote control key code. Decoding means for capturing, imaging means for capturing the field of view range, position detecting means for extracting an area emitted from the remote control in synchronization with the reception timing from the captured image captured by the imaging means, and the position detection A movement tracking unit that tracks the movement of the region extracted by the unit and recognizes a user operation of the operator, and a control unit that controls the controlled device according to the user operation of the operator, The control means includes the infrared signal received by the receiving means and the user operation of the operator recognized by the motion tracking means. The following one instruction at least one is a remote control device for controlling the controlled device.

Further, the motion tracking means of the remote control device of the present invention uses at least one information of position coordinates, movement amount, movement speed, movement direction, movement locus of the area extracted by the position detection means. It recognizes user operations.
Further, the control means of the remote control device of the present invention described above, the motion tracking of the motion tracking means is triggered by the reception timing only when the infrared signal decoded by the decoding means is a specific remote control key code. The operation is started.
Further, the control means of the remote control device according to the present invention further comprises warning means based on at least one of video and audio, and the control means is configured to receive the reception detected by the position detection means by the decoding means. If the area emitted from the remote control corresponding to the timing cannot be specified, the warning means is controlled to warn the operator.

  In order to solve the above-described problems, the present invention further includes a light projecting unit that projects light onto a subject, and the photographing unit includes a brightness image that represents the brightness of each pixel of the subject using only ambient light, and the light projecting unit. Two types of distance images representing the distance to each pixel of the subject obtained by measuring the reflected light reflected from the subject by the light from the means are photographed, and the control means is the remote controller using the position detecting means. After the luminance image is taken by the photographing means without projecting light from the light projecting means until the image of the remote control is detected and the image of the remote control is detected by the position detecting means, the light from the light projecting means is detected. Is a remote control device that projects a distance image by the photographing means.

  In order to solve the above-mentioned problems, the present invention captures an infrared signal emitted from a remote control, a timing detection step of detecting a reception timing of the infrared signal received by the reception step, and a visual field range. An imaging step, and a position detection that extracts a region emitted from the remote control in synchronization with the reception timing from the captured image captured in the imaging step, identifies a region emitted from the remote control, and detects its position A movement tracking step for recognizing a user operation by tracking a movement of the region detected in the position detection step, an instruction for the infrared signal received in the reception step, and a user operation recognized in the movement tracking step And a user operation reflecting step according to any of the instructions

According to the remote control device of the present invention, since there is a decoding means for detecting the timing at which the remote control emits light separately from the imaging means, it is not necessary to keep the remote control always on, and the battery consumption of the remote control can be suppressed. .
In addition, since the captured image is extracted using the timing information of the remote control decoding based on the infrared signal, even if the captured image itself is a normal image obtained by capturing visible light, it is the same as providing an infrared filter in the photographing means The influence of external light and illumination can be eliminated, and the operation of the user (operator) can be recognized with low cost and high accuracy.

It is a block diagram which shows the structure except the remote control of the digital television which is one Embodiment of this invention. 1 is an overall view for explaining a remote control device and a remote control method according to an embodiment of the present invention. It is a flowchart which shows the process sequence of the control part of the remote control apparatus of one Embodiment of this invention. It is a conceptual diagram which shows the image analysis process of the position detection part in the control part of the remote control apparatus of one Embodiment of this invention. It is a block diagram which shows the structure except the remote control of the digital television which is one Embodiment of this invention. It is a flowchart which shows the process sequence of the control part of the remote control apparatus of one Embodiment of this invention.

The main feature of the present invention is that it includes an image pickup means for taking an infrared light emission of the remote control as an image together with a decoding means for receiving a remote control key code signal from the infrared remote control.
In addition, the remote control image is identified by extracting the portion of the captured image that changes in brightness in synchronization with the timing of receiving the key code signal, and the operation is performed by tracking the movement of the image at that position. Recognizes the user's user operation. Both the control by the received remote control key code and the control by the recognized user operation are accepted.
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In addition, the following description is for describing one embodiment of the present invention, and does not limit the scope of the present invention. Accordingly, those skilled in the art can employ embodiments in which these elements or all of the elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

A first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing a configuration of a digital television set excluding a remote controller according to an embodiment of the present invention. FIG. 2 is an overall view for explaining a remote control device and a remote control method according to an embodiment of the present invention.
1 and 2, a digital television 100 is installed indoors, and an operator who is a user who views the digital television 100 operates the remote controller 210 to control the digital television 100 as a controlled device. 1 includes an infrared light receiving unit 101, a decoding unit 102, a camera 103, a position detection unit 104, a gesture recognition unit 105, a control unit 110, a display unit 151, and a panel 152. Furthermore, the structure which contains the audio | voice output part 153 and the speaker 154 may be sufficient.
The camera 103 can shoot a range in which the infrared light receiving unit 101 can receive the infrared signal of the remote controller 210 operated by the operator or a range including the light receiving range as a shooting visual field range (a range surrounded by two broken lines). It is.

  In FIG. 1, a television receiver 150 of a digital television 100 receives a television signal from an antenna or an input terminal (not shown), and the video is sent to an LCD (Liquid Crystal device) panel 152 via a display 151 and the sound is transmitted. This is output to the speaker 154 via the audio output unit 153. That is, the panel unit 152 displays an image, and the speaker 154 outputs sound. Of course, the video display is not limited to the LCD panel, but may be a CRT in addition to an FPD panel such as a PDP.

  As shown in FIG. 2, an operator 200 as a user operates the digital television 100 with the remote controller 210 in his / her hand. The state in which the operator 200 operates is photographed by the camera 103. The camera 103 is, for example, a camera that captures visible light as a moving image using a two-dimensional image sensor such as a CCD sensor or a CMOS sensor. The camera 103 is illuminated with an image of the operator 200 as a background, such as a fluorescent lamp 220 installed indoors. In addition, the subject other than the operator 200 is also photographed, and the photographed image (captured image) is output to the gesture recognition unit 105.

In addition to being used for gesture recognition, the camera 103 outputs a captured image to the encoder 160 and the position detection unit 104.
The encoder 160 can also be used as a videophone by compressing an input image and outputting the compressed image to the Internet (not shown) via the communication unit 161. The position detection unit 104 will be described later.
Note that a conventional digital television that also serves as a video phone includes an infrared light receiving unit 101 that receives an infrared signal from an infrared remote controller (not shown), a decoding unit 102, a camera 103, an encoder 160, a communication unit 161, a control unit 110, a display unit 151, It includes a panel 152, a television receiver 150, an audio output unit 153, and a speaker 154.

There are two types of operation methods that allow the operator 200 to control the television. One is to operate the buttons of the remote controller 210, and the other is to perform a gesture operation while holding the remote controller 210.
Regarding the button operation, the infrared light receiving unit 101 of the television 100 receives the infrared signal from the remote controller 210 and outputs it to the decoding unit 102. The decoding unit 102 analyzes the received signal and outputs it to the control unit 110 as a remote control key code. The control unit 110 controls the television 100 based on the input remote control key code.
As for the gesture operation, an image captured and input by the camera 103 is analyzed by the position detection unit 104 and the gesture recognition unit 105 and recognized as a user (operator) operation, and is processed by the control unit 110. . The operation information recognized by the gesture recognition unit 105 is output to the control unit 110, and the control unit 110 performs a control process on the television 100 based on the input operation information.

Next, the process control procedure executed by the control unit 110 will be described with reference to the flowchart of FIG. Here, a case where the volume of sound output from the speaker 154 of the television 100 is adjusted will be described as an example. FIG. 3 is a flowchart showing a processing procedure executed by the control unit of the remote control device according to the embodiment of the present invention.
First, when the operator presses a button on the remote controller 210, the key code is output from the remote controller 210 as an infrared signal. The digital television 100 side performs the following steps S301 to S308, S309, and S310.

In step S301, the infrared light receiving unit 101 receives the infrared signal of the key code, and the decoding unit 102 decodes the key code. The period in which the infrared light receiving unit 101 in step S301 confirms reception of the infrared signal from the remote controller 210 is at least shorter than the light emission time of the remote control signal.
Next, in step S302, the decoded key code is determined, and if this is a key other than the “gesture button” specially provided for gesture operation, the process of step S310 is performed. If the key is a “gesture button”, the process proceeds to the gesture recognition process in step S <b> 303 and step S <b> 308. That is, the “gesture button” key detects the timing (reception timing) at which the processing of the gesture (steps S303 to S308) is started.
In step S310, individual operation processing corresponding to the key code is performed, and the processing of the flowchart of FIG. For example, if the volume up button, the audio output unit 153 is controlled to increase the output volume of the speaker 154 by one level, or if the volume down button, the audio output unit 153 is controlled to decrease the output volume of the speaker 154 by one level. . The operator performs adjustment so as to achieve a desired volume level by pressing the volume button a plurality of times. Up to this point, the control is the same as the normal remote control operation.

The processing from step S303 to S308 starts with the reception timing as a trigger.
In step S303, the image photographed by the camera 103 is analyzed, and a portion where the remote controller 210 emits light is extracted from the image. The extraction process in step S303 will be described with reference to the conceptual diagram of FIG.
FIG. 4 shows a state where time t elapses in the right direction along the time axis 400. At time t = 0 [ms], the operator 200 presses the gesture button of the remote controller 210, and thereby an infrared signal 450 from the remote controller 210 is output for a time of, for example, 108 [ms]. At this time, the decoding unit 102 detects that the remote control signal is received at time t = 108 [ms], and outputs a timing signal to the position detection unit 104.
On the other hand, the captured images 401, 402, and 403 are obtained by arranging images captured by the camera 103 in time series. The photographed image 401 is an image photographed at time t = −115 [ms], the photographed image 402 is photographed at time t = 10 [ms], and the photographed image 403 is photographed at time t = 135 [ms]. It is an image. For example, if the shooting frame period is 8 [fps], that is, every 125 [ms], the captured images 401, 402, and 403 can be acquired.
Here, the remote controller 210 emits light for the time of 108 [ms] as shown by the shaded portion in the remote control signal of FIG. Therefore, only the image 402 is captured by the light from the remote controller 210, and this is imaged as a region 405 of the captured image 402. In the photographed image 401 and the photographed image 403 before and after that, the light emitting area is not photographed.
The position detection unit 104 analyzes these continuous images as follows, and extracts a region 435 where the light emission of the remote controller 210 is photographed.

First, in order to simplify the subsequent processing, the luminance signals of the captured images 401 to 402 are determined to be binary with a certain threshold value, and binarized images 411, 412 and 413 are obtained.
Here, an area 415 is an area obtained by binarizing 405 obtained by photographing the light emitted from the remote controller 210 and exists only in the image 412. On the other hand, a region 419 is a binarized portion of the image of the fluorescent lamp 220 in the background, and exists in all the images 411, 412, and 413. Images other than these have luminance lower than the threshold and do not appear in the binarized image.
Next, the brightness of each pixel of the binary images 411 and 413 before and after the light emission timing is inverted in black and white to generate inverted images 421 and 423. Then, the three logical products of these two inverted images and the binary image 412 of the light emission timing are calculated to obtain an extracted image 432.
As a result, in the extracted image 432, the part of the area 419 that is always lit is deleted, and only the area 435 where the remote controller 210 emits light remains in synchronization with the light emission timing. Finally, in order to eliminate the influence of minute noise and blurring of the image of a large object that always emits light, only a portion where the area of the light emitting region is within a certain range is extracted from the extracted image 432. This process may also be executed when binarizing.
By performing simple image processing as described above, it is possible to extract only the region 435 that is lit in synchronization with the reception timing of the remote controller 210 (light emission of the remote controller 210). In the following description, this region 435 is referred to as a “synchronous light emission region”.

In the embodiment of FIG. 4, the light emission timing and the three images before and after the light emission timing are used. Instead, only the light emission timing and the previous two images may be used. In this case, if there is an object that moves while brightly emitting light, there is a possibility that a part in the traveling direction will be mistakenly detected as a synchronous light emitting region, but on the other hand, the image after light emission is processed. Therefore, it is possible to detect at a higher speed, which is effective when a quick response is required.
Alternatively, in the embodiment of FIG. 4, the shooting frame rate of the camera 103 is 8 [fps]. However, by performing shooting at a higher speed, the detection accuracy can be improved by increasing the number of images used for analysis. good. Furthermore, the frame rate and the output rate of the photographing by the camera 103 may be performed at a high speed, and an image used for analysis may be sampled at a predetermined cycle.
In the embodiment of FIG. 4 described above, in order to simplify the processing, the region is extracted by binarizing the captured image and performing a simple logical operation. However, the image filter processing may be performed by performing a numerical calculation with the continuous amount before binarization. In addition, for example, the difference process between the captured images 401 and 402 and the difference process between the captured images 402 and 403 are performed, and the difference image between the two difference processed images is binarized to emit light from the remote controller 210. May be extracted.

Returning to the flowchart of FIG. 3, the description of the processing after step S303 is continued. In step S304, it is determined in a predetermined cycle whether or not the presence of the synchronized light emission region has been extracted. If the presence of the synchronized light emission region is extracted, the process proceeds to step S305, and if not, the process proceeds to step S309.
In step S309, a display to warn the operator 200 is displayed on the panel 152 via the display unit 151, and the processing of the flowchart of FIG. This is because, for example, when the operator 200 operated the remote controller 210 from outside the imaging range of the camera 103, the remote control signal could be received by reflection from a wall or the like but could not be captured from the camera 103, or noise or external light Occurs when a false detection occurs due to the effect of and the recognition fails. In such a case, it becomes impossible to track the feature point after that, so that the operator 200 is warned and prompted to press the gesture button again from another position. Note that the warning display in step S309 may be replaced with the reproduction of the warning sound or a combination of both.

In step S305, the barycentric coordinates of the synchronous light emitting area are calculated and set as the initial tracking position.
Next, in step S306, feature points of the image are extracted by searching for edges of luminance components and vertices of regions in the captured image. At this time, the feature point closest to the barycentric coordinates of the synchronous light emission area obtained in step S305 is selected and set as a subsequent tracking target.
In step S307, the feature point is tracked to recognize the gesture operation of the operator 200, and control according to the operation is performed.
For example, the amount of movement to the left and right is detected using the initial tracking position as the origin, and the volume level output from the speaker 154 is increased or decreased by the amount of movement. Increases the volume level when moving to the right and decreases when moving to the left. By making the distance moved and the amount of change in volume level proportional, if you move a lot, it will change that much. As a result, the volume can be set directly, rather than the time-consuming operation of pressing and adjusting the volume adjustment button of the remote controller 210 many times.
Then, the operator 200 presses the gesture button of the remote controller 210 again when the adjustment value can be adjusted to a desired setting value.

In step S308, it is determined at a predetermined cycle whether or not the gesture button has been pressed. When the remote control code based on the press of the gesture button is received via the infrared light receiving unit 101 and the decoding unit 102, the process of the flowchart of FIG. If the remote control code based on the press of the gesture button is not received, the process returns to step S306, returns to the process from step S306, and continues to track the feature points.
Here, the reflection of the volume setting value by the gesture operation is performed at any time according to the gesture operation of the operator until the gesture button is recognized in S308, but it is performed only once at the end. May be.

As described above, according to the first embodiment of the present invention, it is possible to realize a remote operation device and a remote operation method that can accurately recognize a gesture operation using an inexpensive device and that can maintain a battery of a remote controller. That is, the remote controller does not always emit light, but only emits light when the operator performs a button operation, so there is no wasteful consumption of the battery. On the operated device side, a normal inexpensive CCD sensor camera or CMOS sensor camera can be used. In addition, since such a normal inexpensive camera can be used, it can be used also as the camera for the videophone function described above.
In addition, since the tracking target point is extracted using the reception timing of the infrared remote controller, even when external light or multiple people are moving at the same time, the possibility of false detection is reduced and detection is accurate. It can be performed. Furthermore, since the detection process can be realized by simple image processing, an inexpensive and low-speed microcomputer can be used.

In the above description, it is assumed that the position detection unit 104 of the synchronized light emission area is executed by a software program on the digital television 100, but may be realized by a dedicated hardware circuit.
Further, in the gesture operation process in step S307, the detected amount of movement and the change rate of the volume level may be associated with each other. Also, instead of detecting left and right movements, a rotational movement based on the initial coordinates is detected and the set value is changed according to the rotational direction, or a movement locus is detected and compared with a previously registered locus pattern. Thus, a predetermined operation may be performed.
Further, in the embodiment of FIG. 3, the case where the volume of the sound output from the television speaker is adjusted has been described. However, other control of the television may be used, and the present invention is not limited to the television. Needless to say, the present invention is applicable to any controlled device capable of mounting the remote control device.

  A second embodiment of the present invention will be described with reference to FIGS. FIG. 5 is a block diagram showing a configuration excluding the remote control of the digital television which is an embodiment of the present invention. The same parts as those in FIG. 1 are denoted by the same reference numerals, and redundant description is omitted. FIG. 6 is a flowchart showing a processing procedure executed by the control unit of the remote control device according to the embodiment of the present invention. 6 has almost the same procedure as that in FIG. 2, and the same reference numerals are given to portions corresponding to those in FIG.

  In the digital television 500 of FIG. 5 of the present embodiment, a three-dimensional camera using a TOF sensor 503 is used instead of the camera 103 using a CCD sensor in the digital television 100 of FIG. This improves the gesture recognition accuracy and enables various and accurate operations. 5 includes a TOF sensor 503, a light source 530, a control unit 510, an infrared light receiving unit 101, a decoding unit 102, a position detection unit 104, a gesture recognition unit 105, a display unit 151, and a panel. 152. Furthermore, the structure which contains the audio | voice output part 153 and the speaker 154 may be sufficient.

In FIG. 5, an infrared illumination unit 530 is a light source that illuminates a subject to be photographed by projecting infrared light that is modulated in synchronization with the TOF sensor 503. The TOF sensor 503 detects the phase of the reflected infrared light and measures the depth of each pixel to obtain a distance image. The control unit 510 controls the infrared illumination unit 530 according to a procedure to be described later, and selects presence / absence of infrared irradiation. When infrared rays are not irradiated, the TOF sensor 503 functions as a normal infrared imaging camera and obtains a passively incident infrared luminance image. Similar to the camera 103 of the television 100 in the first embodiment, the TOF sensor 503 can capture a range in which the infrared light receiving unit 101 can receive an infrared signal of a remote controller operated by an operator. Similarly, the infrared illumination unit 530 can irradiate the entire region of the image detection range of the TOF sensor 503 with infrared rays.
FIG. 6 is a flowchart showing a processing procedure executed by the control unit of the remote control device according to the embodiment of the present invention. FIG. 6 also illustrates these processing control procedures executed by the control unit 110, as in FIG.

First, in FIG. 5 and FIG. 6, the infrared illumination 530 is turned off in the initial stage. In this state, the TOF sensor 503 receives infrared rays not caused by the infrared illumination 530 from the image detection range and detects them as a luminance image.
As in FIG. 3, first, when the operator presses a button on the remote controller 210, the key code is output from the remote controller 210 as an infrared signal. The digital television 100 side performs the following steps S301 to S304, S605, S305 to S308, S651, S309, and S310.

In the process of step S304, it is determined at a predetermined cycle whether or not the presence of the synchronous light emission region has been extracted. If the presence of the synchronized light emission region is extracted, the process proceeds to step S650, and if not, the process proceeds to step S309.
In step S650, the infrared light source 530 is turned on, and the process proceeds to step S305.
Hereinafter, steps S305 to S308 are the same as those in FIG.
However, in step S308, if the gesture button is pressed, the process proceeds to step S651.
In the infrared light source extinguishing step of step S651, the lighting of the infrared illumination unit 530 is stopped (extinguishes), and the processing of the flowchart of FIG.

  By configuring as described above, it is not necessary to always turn on the infrared illumination 530 required when the TOF sensor is used, and power consumption can be reduced. This is particularly effective when the operated device is a battery-driven mobile device.

  100: Digital television, 101: Infrared light receiving unit, 10: Remote control signal decoding unit, 103: Camera, 104: Position detection unit, 105: Gesture recognition unit, 110: Control unit, 150: Television reception unit, 151: Display unit, 152: Panel, 153: Audio output unit, 154: Speaker, 160: Encoder, 161: Communication unit, 200: Operator, 210: Remote control, 220: Fluorescent lamp illumination, 500: Digital television, 503: TOF sensor, 510: Control unit, 530: infrared illumination unit.

Claims (6)

Receiving means for receiving an infrared signal emitted from the remote control by a user operation of the operator;
Decoding means for detecting the reception timing by decoding the infrared signal received by the receiving means as a remote control key code;
Imaging means for photographing the field of view;
Position detecting means for extracting an area emitted from the remote controller in synchronization with the reception timing from a captured image photographed by the imaging means;
A movement tracking unit that tracks the movement of the region extracted by the position detection unit and recognizes a user operation of the operator;
Control means for controlling the controlled device according to the user operation of the operator,
The control means controls the controlled device in accordance with at least one instruction of the infrared signal received by the receiving means and a user operation of the operator recognized by the movement tracking means. apparatus. The movement tracking unit recognizes a user operation using at least one piece of information of a position coordinate, a movement amount, a movement speed, a movement direction, and a movement locus of the area extracted by the position detection unit. The remote control device according to claim 1. The control means starts the motion tracking operation of the motion tracking means only with the reception timing as a trigger only when the infrared signal decoded by the decoding means is a specific remote control key code. The remote control device according to claim 1 or 2. It further comprises warning means by at least one of video or audio,
The control means controls the warning means to control the operator when the position detection means cannot identify the area emitted from the remote controller corresponding to the reception timing detected by the decoding means. The remote control device according to claim 1, wherein a warning is issued to the remote control device. A light projecting means for projecting light onto the subject;
The photographing unit calculates a distance from the luminance image representing the luminance of each pixel of the subject by only ambient light to each pixel of the subject obtained by measuring the reflected light reflected by the light from the light projecting unit. Take two types of images of the distance image shown,
The control means takes a luminance image with the photographing means without projecting light from the light projecting means until an image of the remote control is detected with the position detecting means, and the image of the remote control is taken with the position detecting means. 5. The remote control device according to claim 1, wherein after being detected, light is projected from the light projecting unit, and a distance image is photographed by the photographing unit. A receiving step for receiving an infrared signal emitted from the remote control;
A timing detection step of detecting the reception timing of the infrared signal received by the reception step;
An imaging step for photographing the field of view;
A position detection step of extracting a region emitted from the remote control in synchronization with the reception timing from the captured image captured in the imaging step, identifying a region emitted from the remote control and detecting its position;
A motion tracking step of recognizing a user operation by tracking the movement of the region detected in the position detection step;
A remote control method comprising: an instruction of the infrared signal received in the receiving step; and a user operation reflecting step according to any instruction of the user operation recognized in the motion tracking step.

Images