JP4589828B2 - Remote control method and remote control apparatus thereof - Google Patents

Description

  The present invention does not have a remote control terminal (remote control terminal, hereinafter referred to as “remote control terminal”), and does not include a controlled device (for example, a television receiver, a video recording device, a DVD (Digital Versatile Disc; CD (Compact Disc)). ), HD (Hard Disc)) recording device, air conditioner, lighting device, security device, etc.), and a remote control method (remote control device, hereinafter referred to as “remote control device”). ).

  Conventionally, in remote control of controlled devices, switch on / off and function control are performed for each controlled device using a remote control terminal corresponding to each controlled device. For example, when recording a television (hereinafter referred to as “TV”) program, a remote control terminal for a television receiver is used to select a channel after the television receiver is turned on, and then for a video recording device. The video recording device is turned on and the recording function is selected. At this time, there is also an integrated television and video remote control terminal so that the remote control terminal can be controlled without changing.

  In addition, in general, a plurality of controlled devices such as those used in digital information home appliances (multi-media home information devices that can be handled as easily as conventional home appliances using digital technology), etc. Is a network (communication circuit network) wired or wireless, and is called, for example, a set-top box (set-top box, what is necessary when using so-called interactive television in multimedia communication services) There is also a method of collectively managing controlled devices on a network by providing one.

  Furthermore, in a network system using a set-top box, the set-top box is connected to the Internet or the like, and a function for displaying information suitable for each controlled device on a display unit mounted on the controlled device is provided. There may be. In addition, controlled device control by a self-propelled robot represented by a biped robot has also been proposed.

  Examples of documents relating to this type of remote control device include the following.

JP 2004-363930 A (FIGS. 1 and 2)

  1 and 2 of Patent Document 1 describe a technology of a television broadcast receiving system including a television receiver and a remote control terminal for remotely controlling the television receiver. In this technique, a program information list screen generated based on electronic program guide information is projected and displayed on a projection surface such as a wall from a remote control terminal, and details about program information selected by a cursor on the program information list screen are displayed. Information is displayed on a detailed information display screen provided in the remote control terminal. Accordingly, the television image, the program information list screen, and the detailed information are simultaneously displayed without hindering the display of the television image by the television receiver, thereby improving the usability of the electronic program guide screen.

  However, the conventional remote control method and remote control device have the following problems (a) to (f), for example.

  (A) For example, when controlling both a television receiver and a video recording device, two remote control terminals for the television receiver and the video recording device are always placed on the side, and each is controlled. Therefore, it is necessary to change the remote control terminal. An integrated TV and video remote control terminal is intended to reduce the burden, but in this case, it becomes larger and heavier than a remote control terminal that is not integrated. Further, in the integrated remote control terminal, there are many push buttons for identifying a plurality of functions of the television and the video, which is complicated when used. Furthermore, even with an integrated remote control terminal, the number of terminals is reduced, but the problem of having to be always placed on the side cannot be solved.

  (B) The example of (a) is only for a combination of a television receiver and a video recording device, but the problem becomes more serious when other controlled devices and information home appliances are also controlled. .

  (C) For example, in a method of collectively managing controlled devices on the network by providing one processing control unit called a set top box, each controlled device on the network is a standard corresponding to the set top box. Need to be unified. Therefore, until this standard is standardized and spread, there are cases where controlled devices of different manufacturers cannot be installed in the same network, and the options for purchasing controlled devices are significantly narrowed for the user. Furthermore, many of the conventional controlled devices do not have network-compatible specifications. If you have such conventional controlled devices, even if the manufacturer is the same, a new one will be provided. It becomes difficult to connect to a designated network.

  (D) Even in a network system using a set-top box, the user has the problem described above that a terminal for controlling the set-top box must always be placed on the side.

  (E) In the function of connecting a set-top box to the Internet or the like and displaying information suitable for each controlled device on the display unit provided in the controlled device, the information is temporarily stored in the set-top box. Later, a procedure is taken in which the set-top box is transmitted again to each controlled device and then displayed on the display unit of each controlled device. In this way, the communication is in two stages, and not only the information reception and transmission functions are required for the set top box, but also the information reception function is required for each controlled device, and the mechanism is complicated. . On the other hand, regarding the number of addresses required for communication between controlled devices, it is necessary to allocate addresses not only to the set-top box but also to each controlled device that receives calls from the set-top box. The problem is that the number increases. This is more serious when taking a method of directly connecting each controlled device to the Internet without going through a set-top box.

  (F) The display unit provided in each controlled device is generally small, and it is difficult to display a large amount of information, still images, and moving images from the Internet etc. on the display unit at a time. However, it is difficult for the user to see with a small screen, and it causes a big trouble in working while viewing the information. In order to solve this, using the technology of Patent Document 1, a display unit is provided on the remote control terminal to display information, still images, and moving images on the remote control terminal, or a projector is provided on the remote control terminal to provide the information. It is also conceivable to enlarge and display such as on a projection surface such as a wall. However, it is inconvenient because the remote control terminal must always be on the side, and when the projector is provided on the remote control terminal, the remote control terminal becomes large and heavy, and the usability is poor.

  As described above, it has been difficult to provide a remote control method and a remote control device that are sufficiently satisfactory technically with the conventional technology.

  In order to solve the above problems, in the remote control method of the present invention, the first imaging process, the first determination process, the first projection process, the second imaging process, the second determination process, and the second projection are performed. Processing, third imaging processing, third determination processing, and transmission processing are performed.

  In the first photographing process, in response to the tracking result of the tracking means for tracking the user's movement, the user's start instruction operation is photographed by the photographing means whose photographing direction is directed to the user. 1 received image is output. In the first determination process, the content of the start instruction operation is determined by a determination unit based on the first received image, and a first determination result is output. In the first projection processing, a projection unit that performs a predetermined operation by receiving an operation signal based on the first determination result by a projection unit that directs a projection direction toward the user in response to the tracking result. A menu image for selecting a control device is projected onto a projection plane in the vicinity of the user. In the second photographing process, the photographing means photographs the user's selection instruction operation in the menu image and outputs a second received image. In the second determination process, based on the second received image, the determination unit determines the user's selection instruction location in the menu image and outputs a second determination result.

  In the second projection processing, based on the second determination result, the projection unit selects a function selection image for selecting the function of the controlled device corresponding to the selection instruction location in the menu image. Project onto the projection plane. In the third photographing process, the photographing means photographs the user's selection instruction operation in the function selection image and outputs a third received image. In the third determination process, based on the third received image, the determination unit determines the selection instruction location of the user in the function selection screen and outputs a third determination result. Further, in the transmission process, based on the third determination result, the operation signal corresponding to the selection instruction location in the function selection screen is transmitted from the transmission unit to the selected controlled device to be a predetermined value. Let the action take place.

  In another remote control method of the present invention, a storage process, a first imaging process, a first determination process, a first projection process, a second imaging process, a second determination process, a second projection process, 3 imaging processing, 3rd determination processing, and transmission processing are performed.

  In the storage process, data of a menu image for selecting a controlled device that performs a predetermined operation by receiving an operation signal and data of a function selection image for selecting a function of the controlled device are transmitted by a control unit. Pre-stored in the storage means. In the first photographing process, in response to the tracking result of the tracking means for tracking the user's movement, the user's start instruction operation is photographed by the photographing means whose photographing direction is directed to the user. 1 received image is output. In the first determination process, the content of the start instruction operation is determined by a determination unit based on the first received image, and a first determination result is output. In the first projection process, the data of the stored menu screen is read by the control unit, and the first determination is performed by a projection unit that directs the projection direction toward the user in response to the tracking result. Based on the result, the read menu image is projected onto a projection plane in the vicinity of the user. In the second photographing process, the photographing means photographs the user's selection instruction operation in the menu image and outputs a second received image.

  In the second determination process, based on the second received image, the determination unit determines the user's selection instruction location in the menu image and outputs a second determination result. In the second projection processing, the stored function selection screen data is read by the control unit, and based on the second determination result, the projection unit corresponds to the selection instruction location in the menu image. The read function selection image is projected onto the projection plane. In the third photographing process, the photographing means photographs the user's selection instruction operation in the function selection image and outputs a third received image. In the third determination process, based on the third received image, the determination unit determines the selection instruction location of the user in the function selection screen and outputs a third determination result. Further, in the transmission process, based on the third determination result, the operation signal corresponding to the selection instruction location in the function selection screen is transmitted from the transmission unit to the selected controlled device to be a predetermined value. Let the action take place.

  The remote control device of the present invention includes tracking means for tracking the user's movement and outputting the tracking result, photographing means, determination means, projection means, and transmission means.

  In response to the tracking result, the photographing unit directs the photographing direction to the user, photographs the user's start instruction operation, outputs a first received image, and performs a predetermined operation by receiving an operation signal. The user's selection instruction operation in a menu image for selecting a controlled device to be performed is photographed and a second received image is output, corresponding to the user's selection instruction location in the menu image The user's selection instruction operation in a function selection image for selecting the function of the controlled device is photographed and a third received image is output. The determination means determines the content of the start instruction operation based on the first received image, outputs a first determination result, and selects the selection in the menu image based on the second received image. An instruction location is determined and a second determination result is output, and the user's selection instruction location in the function selection screen is determined based on the third received image, and a third determination result is output. Is.

  The projecting means directs a projection direction toward the user in response to the tracking result, projects the menu image on a projection surface in the vicinity of the user based on the first determination result, and The function selection image is projected onto the projection plane based on the determination result of 2. Furthermore, the transmission means transmits the operation signal corresponding to the selection instruction location in the function selection screen to the selected controlled device based on the third determination result.

  In another remote control device of the present invention, menu image data for selecting a controlled device that performs a predetermined operation by receiving an operation signal, and function selection image data for selecting a function of the controlled device are stored. Storage means for storing, tracking means for tracking a user's movement and outputting a tracking result, photographing means, determination means, projection means, and transmission means are provided.

  The photographing means directs the photographing direction to the user in response to the tracking result, photographs the user's start instruction operation, outputs a first received image, and is read from the storage means The user's selection instruction operation in the menu image in the menu screen data is photographed to output a second received image, and in the function selection image data read from the storage means, in the menu image The user's selection instruction operation in the function selection image corresponding to the user's selection instruction location is taken and a third received image is output. The determination means determines the content of the start instruction operation based on the first received image, outputs a first determination result, and selects the selection in the menu image based on the second received image. An instruction location is determined and a second determination result is output, and the user's selection instruction location in the function selection screen is determined based on the third received image, and a third determination result is output. Is.

  The projecting means directs a projection direction toward the user in response to the tracking result, projects the menu image on a projection surface in the vicinity of the user based on the first determination result, and The function selection image is projected onto the projection plane based on the determination result of 2. Furthermore, the transmission means transmits the operation signal corresponding to the selection instruction location in the function selection screen to the selected controlled device based on the third determination result.

  According to the present invention, there is no need to place a remote control terminal corresponding to each controlled device on the side, there is no need to change a plurality of remote control terminals, and there is no need to press a complicated push button. This produces the effect. Furthermore, these effects can be produced in the same manner regardless of the number of controlled devices to be controlled.

  In addition, according to the present invention, all controlled devices, even controlled devices from different manufacturers whose standards are not unified, or conventional controlled devices that are not network compatible specifications, are all The user can easily register with the remote control device.

  Furthermore, according to the present invention, by connecting the controlled device remote control device to the Internet or the like, it is possible to directly project and display information suitable for each controlled device on any location on the controlled device side. . In other words, the remote control device to be controlled only needs to have an information reception function, and this is a problem in a network system using a conventional set top box. The problem of adding an information receiving function to the controlled device can be solved. Therefore, the mechanism is simplified, and there is no need for laying wired wiring or the like. If, for example, infrared rays are used as operation signals, microwaves that cannot completely deny adverse effects on the human body will not fly. In addition, the size of the information projection can be enlarged or reduced, and the size most easily seen by the user can be selected.

  In the remote control method of the best mode of the present invention, the first imaging process, the first determination process, the first projection process, the second imaging process, the second determination process, the second projection process, the third Shooting processing, third determination processing, and transmission processing.

  First, in the first shooting process, in response to the tracking result of the tracking unit that tracks the movement of the user, the user uses the shooting unit (for example, a video camera) whose shooting direction is directed to the user. The start instruction operation is photographed and a first received image is output. In the first determination process, based on the first received image, the determination means determines the content of the start instruction operation and outputs a first determination result. In the first projection process, an operation signal (for example, an image signal) is projected on the basis of the first determination result by a projection unit (for example, an image projector or the like) whose projection direction is directed toward the user in response to the tracking result. , Radio waves, ultrasonic waves, infrared rays, etc.), a menu image for selecting a controlled device that performs a predetermined operation is projected onto a projection plane in the vicinity of the user.

  In the second photographing process, the photographing unit photographs the user's selection instruction operation in the menu image and outputs a second received image. In the second determination process, based on the second received image, the determination means determines the user's selection instruction location in the menu image and outputs a second determination result. In the second projection processing, based on the second determination result, a function selection image for selecting a function of the controlled device corresponding to the selection instruction location in the menu image is selected by the projection unit. Project onto the projection plane.

  In the third photographing process, the photographing means photographs the user's selection instruction operation in the function selection image and outputs a third received image. In the third determination process, based on the third received image, the determination unit determines the user's selection instruction location in the function selection screen and outputs a third determination result. Thereafter, in the sending process, based on the third determination result, the sending means sends the operation signal corresponding to the selection instruction location in the function selection screen to the selected controlled device. Let the action take place.

(External appearance of Example 1)
FIG. 2 is a schematic external view of a remote control device showing Embodiment 1 of the present invention.

  The remote control device 1 has a pedestal 2 attached to a ceiling of a room, for example. A rotating shaft 4 that is rotated by a servo motor 3 is attached to the pedestal 2, and a first rotating body 5 for storing electronic components is mounted on the rotating shaft 4, and further below this is a second rotating body for storing electronic components. 6 is installed. The first rotator 5 and the second rotator 6 are arranged at a position overlooking the room, and are configured to reciprocate independently by, for example, a right and left angle of ± 180 ° by the rotation shaft 4. A transmitting means (for example, an infrared light emitting element such as an infrared light emitting diode or an infrared laser diode) 7 is attached to the side surface of the first rotating body 5. On the side surface of the second rotator 6, there are a zoom lens 8 of a projection unit (for example, an image projector) and an imaging unit (for example, a charge coupled device (hereinafter referred to as “CCD”) camera which is one of video cameras). A zoom lens 9 and a receiving means (for example, an infrared light receiving element such as an infrared photodiode) 10 are attached. The first rotator 5 and the second rotator 6 are connected to the device control panel 12 via an electrical wiring 11. The device control panel 12 stores electronic component circuits and the like, and is installed indoors or outdoors.

FIG. 3 is a schematic perspective view showing an installation example of the remote control device 1 of FIG.
2 is attached to the ceiling 20a of the room 20, for example. A controlled device (for example, a television receiver 21) is installed on the floor 20c of the room 20, and another controlled device (for example, an air conditioner 22) is attached to the wall 20b of the room 20. The television receiver 21 is provided with a receiving means (for example, an infrared light receiving unit) 21a. The infrared light receiving unit 21a receives an operation signal (for example, infrared for TV control) transmitted from a television remote control terminal. Power on / off, channel switching, etc. are remotely controlled. Similarly, the air conditioner 22 is also provided with a receiving means (for example, an infrared light receiving unit) 22a, and an operation signal (for example, an air conditioner control infrared ray) transmitted from the air conditioner remote control terminal is transmitted by the infrared light receiving unit 22a. Light is received and power on / off, operation mode switching, and the like are remotely controlled.

  The feature of the first embodiment is that the television receiver 21 and the air conditioner 22 that are remotely controlled by each remote control terminal are remotely controlled by the common remote control device 1 without using the remote control terminal. For this reason, the operation surface of each remote control terminal and each operation button function on the operation surface are stored in the remote control device 1 in advance by storage processing, and the positions of the television receiver 21 and the air conditioner 22 to be controlled are remote controlled. Register in the device 1 in advance.

  At the time of use, the user 23 is photographed by the CCD camera through the zoom lens 9 of the remote control device 1, the start instruction operation and the selection instruction operation of the user 23 are automatically determined, and the menu screen is zoomed in by the image projector. Projection is performed through the lens 8 onto a projection surface (for example, a wall) 20 b in the vicinity of the user 23. When the user 23 instructs to select a desired controlled device (for example, the television receiver 21) for the projection image 24 composed of the menu screen, this instruction is photographed by the CCD camera through the zoom lens 9, and the instruction content is displayed. The determination is automatically made, and the operation surface of the television control remote control terminal is projected onto the wall 20b through the zoom lens 8 of the projector. When the user 23 selects and designates an operation button (for example, an operation start button) on the projection image 24 composed of the operation surface, the instruction is photographed by the CCD camera through the zoom lens 9, and the content of the instruction is automatically determined. Based on this determination result, an operation start signal (for example, infrared light for television control) c1 is transmitted from the infrared light emitting element 7, and is received by the infrared light receiving unit 21a of the television receiver 21, and the television receiver is received. 21 starts operation.

  When the operation surface of the remote control terminal for air conditioner control is projected on the projection image 24, when the user 23 instructs to select an operation button (for example, an operation start button), the infrared light emitting element 7 is based on this instruction. The operation signal for starting operation (for example, infrared light for controlling the air conditioner) c2 is transmitted and received by the infrared light receiving unit 22a of the air conditioner 22, and the air conditioner 22 starts to operate.

(Circuit configuration of Example 1)
FIG. 1 is a schematic circuit configuration diagram of the remote control device 1 of FIG. 2 showing Embodiment 1 of the present invention.

  The remote control device 1 includes a control unit 30 that controls the entire device, a memory 36 that is accessed (read / written) by the control unit 30, an infrared light emitting unit 40, an image projection unit 50, and an image receiving / receiving unit. The control unit 30, the infrared light receiving unit 70, the control unit 30, the infrared light emitting unit 40, the image projecting unit 50, the image receiving / processing unit 60, and the infrared light receiving unit 70 are configured by the processing unit 60 and the infrared light receiving unit 70. The multibus lines 37 are connected to each other.

  The control unit 30 is housed in the device control panel 12, and includes a storage unit (for example, a high-speed storage device) 31, a control unit (for example, a central processing unit (hereinafter referred to as "CPU")) 32, a rotation drive control circuit 33, And communication means (for example, a communication control circuit) 34, which are connected to a multibus line 37. A memory 36 accommodated in the apparatus control panel 12 is connected to the CPU 32, and a communication line 35 such as the Internet is connected to the communication control circuit 34.

  The high-speed storage device 31 includes, for example, a menu image, a function selection image of each controlled device captured from the image receiving unit in the image receiving / processing unit 60, for example, a remote control terminal surface image, and each of the function selection images It is configured by a non-volatile storage device that stores information on the infrared light irradiation pulse signal corresponding to the selection button image and the position of the controlled device in the room 20 as a database. The CPU 32 accesses the high-speed storage device 31 and program-controls the processing order of the entire device through the multi-bus line 37. The memory 36 connected to the CPU 32 is for temporarily waiting for a signal waiting for processing when the CPU 32 performs various controls, and is faster than using the high-speed storage device 31 through the multi-bus line 37. This is used when there is no need to store the data in the high-speed storage device 31 as in the case of temporary standby.

  The rotation drive control circuit 33 is a circuit that controls the rotation angles of the first rotating body 5 and the second rotating body 6 through the rotating shaft 4 that is driven to rotate by the servo motor 3 under the control of the CPU 32. For example, when the user 23 is tracked by the rotation drive control circuit 33, the CPU 32 calculates a rotation angle based on a signal relating to the movement of the user 23 obtained from the image receiving / processing unit 60, and this rotation The servo motor 3 is controlled to rotate the second rotating body 6 by an angle. Further, when irradiating the controlled device with the pulse signal by the infrared light from the infrared light emitting unit 40, it corresponds to the position of the controlled device that is stored in advance as a database in the high-speed storage device 31. A difference between the angle and the current angle is instructed by the CPU 32, and the servo motor 3 is controlled so that the first rotating body 5 rotates by the angle difference.

  The communication control circuit 34 is a circuit that controls communication with a communication line 35 such as the Internet or an indoor local area network (hereinafter referred to as “LAN”). For example, the communication control circuit 34 is periodically used in a remote place or other room. By viewing the log file, it is possible to know the control of the controlled device performed by the remote control device 1 of the first embodiment. Thus, for example, when it is found that the controlled device is forgotten to be turned off, the CPU 32 is operated from a remote place or another room to switch off the controlled device from the infrared light emitting unit 40. By transmitting an infrared light pulse signal for that purpose, the controlled device can be switched off.

  The infrared light emitting unit 40 includes an infrared light emitting element 7 attached to the side surface of the first rotating body 5 and a pulse generation circuit that is accommodated in the device control panel 12 and generates a pulse to be given to the infrared light emitting element 7. The pulse generation circuit 41 is connected to the multi-bus line 37. In the infrared light receiving unit 40, the function selection image projected from the image projecting unit 50 sends information on the pulse signal corresponding to the function instructed by the user 23 to the pulse generating circuit 41 according to the instruction of the CPU 32. Then, an infrared light pulse signal is transmitted from the infrared light emitting element 7 toward the corresponding controlled device.

  The image projection unit 50 includes a zoom lens 8 attached to a side surface of the second rotator 6, a projector 51 that is accommodated in the second rotator 6 and that projects a projected image on the zoom lens 8, and the apparatus control panel 12. And a projector interface 52 for connecting the projector 51 to the multi-bus line 37. The image projection unit 50 projects a menu image or function selection image stored as a database in the high-speed storage device 31 from the projector 51 through the zoom lens 8 in accordance with an instruction from the CPU 32 received via the projector interface 52. The projector 51 is accommodated in the second rotating body 6, and is integrated with the CCD camera 61 provided in the second rotating body 6 and the infrared light receiving element 10 attached to the side surface of the second rotating body 6. To rotate.

  The image receiving / processing unit 60 includes a zoom lens 9 attached to a side surface of the second rotating body 6, a CCD camera 61 that is housed in the second rotating body 6 and photographs the user 23 through the zoom lens 9, The image processing circuit 62 is housed in the apparatus control panel 12 and performs processing of an image taken by the CCD camera 61, and the image processing circuit 62 is connected to the multibus line 37.

  The infrared light receiving unit 70 includes an infrared light receiving element 10 attached to a side surface of the second rotating body 6 and an amplifier (amplifier) that is accommodated in the second rotating body 6 and amplifies an output pulse of the infrared light receiving element 10. (Hereinafter referred to as “amplifier”) 71 and a pulse analysis circuit 72 which is accommodated in the apparatus control panel 12 and analyzes the pulse amplified by the amplifier 71, and this pulse analysis circuit 72 is connected to the multi-bus line 37. It is connected. There are independent remote control terminals corresponding to each controlled device, and each remote control terminal has selection buttons corresponding to many functions. The transmitted infrared light pulse signal is different for each selection button. In the infrared light receiving unit 70, the infrared light receiving element 10 receives the infrared light pulse signal from each selection button of each remote control terminal, and the amplifier 71 amplifies the photocurrent output from the infrared light receiving element 10. After that, the pulse analysis circuit 72 performs pulse analysis, and information related to the pulse signal is stored in the high-speed storage device 31 in accordance with an instruction from the CPU 32.

  In particular, the image receiving / processing unit 60 according to the first embodiment has the following functions as a tracking unit and a determination unit.

  The image receiving / processing unit 60 tracks the movement of the user 23 (function as a tracking unit), or determines what the user 23 is instructing with respect to the image projected from the image projecting unit 50. (Function as determination means).

  When tracking the movement of the user 23, for example, by taking the difference between pixels in temporally adjacent images, the horizontal midpoint of the changing pixel portion is always at the center of the image. The CCD camera 61 is rotated. When there are no more changing pixels (still state), the CCD camera 61 stops rotating and waits at this position.

  In the remote control device 1 according to the first embodiment, the CCD camera 61 always tracks the movement of the user 23. As a method for tracking the movement of the user 23, for example, there are the following two methods (i) and (ii).

(I) Tracking method 1
4A to 4C are received image diagrams showing in detail the tracking method 1 of the remote control device 1 of FIG.

  The CCD camera 61 captures the image of the user 23 at certain time intervals. Then, by the processing of the image processing circuit 62 and the CPU 32, two changed pixels are arbitrarily extracted for each pixel, and the angle is adjusted so that the CCD camera 61 faces in the middle between the two pixels. For example, an image 23a-1 of the user 23 is captured at a certain time T (FIG. 4A), and then at a time (T + t) after a certain time t ((FIG. 4B)), the user 23 A case where the image 23a-2 is captured will be described.

  When a portion changing for each pixel is extracted (FIG. 4C), the pixel in the region 23a-3 surrounded by A, B, C, D, and E is between time T and time (T + t). It has changed. Therefore, when the difference between the pixels is extracted in order from the upper row of the received image 23a-1 and 23a-2, A can be extracted first as a changed pixel, and B can be subsequently extracted. Finally, C can be extracted. In this case, the angle at which the CCD camera 61 faces may be between the pixels A and B or between the pixels A and C. Alternatively, the difference between the pixels may be sequentially extracted from the lower row of the received image 23a-1 and 23a-2, and may be, for example, between the pixels D and E. Once the changed part can be arbitrarily extracted by two pixels, all subsequent extractions are stopped, the CCD camera 61 is rotated to the midpoint of the two pixels, and the next image is received at this position. As a result, it is possible to greatly reduce the time for comparing the pixels and extracting the changing pixels. Repeat the above. By rapidly rotating the CCD camera 61 to the midpoint of the two pixels based on the two pixels that can be extracted first at an arbitrary position as a reference, extremely rapid tracking can be performed. Finally, if the tracking user 23 leaves the room 20 from the door, there will be no pixels that change, so the CCD camera 61 will stand by facing the door, and thereafter If the user 23 enters from the door, the pixel starts to change again and tracking is resumed.

(Ii) Tracking method 2
The user 23 irradiates the remote control device 1 when necessary using a portable terminal that irradiates infrared light. Then, the CCD camera 61 of the remote control device 1 rotates in the direction irradiated with infrared light. In this case, the portable terminal that irradiates infrared light needs only one button that irradiates infrared light, and does not need buttons such as a channel and volume like a dedicated terminal attached to a normal controlled device. The size can be significantly reduced. For example, it can be a pen type or a pendant type.

  Next, in the image receiving / processing unit 60 according to the first embodiment, which menu or function is selected by the user 23 for the menu image or function selection image projected from the image projection unit 50. The determination method of will be described.

  When determining which menu or function is selected by the user 23, for example, image comparison is performed. That is, the menu image or the function selection image that is stored as a database in the high-speed storage device 31 before the user 23 selects and the menu image that is received by the CCD camera 61 and that the user 23 selects and instructs. Or, by comparing and collating the function selection image with each other, the difference between the pixels is taken, the location where the user's 23 finger is instructed to select in the menu image or the function selection image is specified, and this specific location means Information on menus and functions to be sent is sent to the CPU 32.

  The zoom lens 9 of the CCD camera 61 can be interlocked with the zoom lens 8 of the projector 51 in the image projection unit 50. When the projector 51 projects a menu image or a function selection image, the CCD camera 61 Can always be brought together with the projector 51 or at a predetermined relative position, so that information relating to the size of the projection image 24 of the projector 51 can be received. Thereby, in image processing, the step of image cutout processing can be omitted, and the image processing can be speeded up.

  FIG. 5 is a schematic circuit diagram showing the CCD camera 61 and the image processing circuit 62 in FIG.

  The image processing circuit 62 includes, for example, a camera interface 62a connected to the CCD camera 61, a processing circuit main body 62b connected to the multibus line 37, and an image memory 62c connected to the multibus line 37. The camera interface 62a, the processing circuit main body 62b, and the image memory 62c are connected to each other by a high-speed image bus line 62d. The processing circuit main body 62b includes, for example, a binarization processing circuit 62b-1 and a comparison / collation processing circuit 62b-2.

  The image processing circuit 62 has a function of performing image processing as shown in the following (a) to (j).

  (A) The first start instruction operation (such as raising a hand) of the user 23 is received by the CCD camera 61.

  (B) When the received image signal is input to the camera interface 62a, and the first start instruction operation of the user 23 is confirmed in the processing circuit main body 62b, it is already stored as a database in accordance with an instruction from the CPU 32. The menu image data and the image data after the binarization processing are transferred from the high speed storage device 31 to the image memory 62c. At the same time, a menu image is projected from the projector 51 via the projector interface 52.

  (C) The user 23 selects and designates the target controlled device in the menu image (points with a finger).

  (D) Preprocessing of binarization processing by the binarization processing circuit 62b-1 is performed on the image signal received from the CCD camera 61.

  (E) The obtained preprocessed image is compared and verified by the comparison / collation processing circuit 62b-2 with the image data after the binarization processing of the original menu image already transferred to the image memory 62c. This process is performed through the high-speed image bus line 62d.

  (F) By comparing and collating in the comparison / collation processing circuit 62b-2, it is determined which part in the image the tip of the finger of the user 23 indicates by taking a difference between pixels.

  (G) It is determined from the database that has already been transferred to the image memory 62c which controlled device corresponds to the portion indicated by the tip of the finger of the user 23.

  (H) The infrared light emitting unit 40 rotates in the installation direction of the controlled device (this is stored as a database in the high-speed storage device 31), and the infrared light pulse signal corresponding to the controlled device is It is transmitted from the infrared light emitting element 7.

(I) Next, a function selection image is projected.
(J) Hereinafter, the same as (c) to (h). However, since the infrared light emitting element 7 is already facing the direction of the corresponding controlled device, the rotation is no longer necessary.

(Remote control method of the first embodiment)
In the first embodiment, (1) remote control terminal registration processing (storage processing), (2) controlled device storage processing (registration processing), and (3) remote control processing of remote control device 1 are as follows. Is done.

(1) Remote terminal registration process (memory process)
FIG. 6 is a flowchart showing the registration processing procedure of the remote control terminal.

  When the user 23 performs an initial instruction operation for registering a remote control terminal (step S1), this is photographed (image reception) by the CCD camera 61 through the zoom lens 9 of the remote control device 1 (step S2). The image processing circuit 62 processes the photographed image and determines “whether there is a menu display request” from the user 23 (step S3). When the determination result is “no request”, the registration process is terminated. (Step S12). When the determination result is “Requested”, the projection image 24 including the menu image 24-1 is projected from the projector 51 through the zoom lens 8 onto, for example, the wall 20b in the vicinity of the user 23 (step S4).

  When the user 23 instructs “register controlled device” in the menu image 24-1, this is photographed (image received) by the CCD camera 61 through the zoom lens 9 (step S5). When the user 23 selects and designates a device name in the “Registered controlled device” screen, this is photographed (image received) by the CCD camera 61 through the zoom lens 9, and a list of controlled device names from the projector 51 through the zoom lens 8. An image is projected. When the user 23 selects and designates the target controlled device name in the list image, this is received by the CCD camera 61, and an image relating to the image of the remote control terminal operation “plane” is projected from the projector 51 through the zoom lens 8. The At the same time, a small area to be shown with the remote control terminal facing the CCD camera 61 is also projected in a rectangular pattern, for example.

  The user 23 selects and instructs “registration” while holding the actual remote control terminal operation “plane” in the rectangular pattern within the projected image 24. As a result, the remote control terminal operation “surface” is received by the CCD camera 61 (step S6), subjected to image processing by the image processing circuit 62, and stored in the high-speed storage device (step S7). The stored remote control terminal operation “image” (that is, function selection image 24-2) is projected from the projector 51 through the zoom lens 8 (step S8).

  On the projected function selection image 24-2, the user 23 designates necessary functions one by one with a finger and presses the function button from the actual remote control terminal each time, thereby corresponding to the function. An infrared light pulse signal is transmitted toward the infrared light receiving element 10 of the remote control device 1. The infrared light pulse signal (step S9) received by the infrared light receiving element 10 is amplified by the amplifier 71, and then the pulse signal is analyzed by the pulse analysis circuit 72 and stored in the high speed storage device 31 as a database. . Which button on the function selection image 24-2 projected from the projector 51 through the zoom lens 8 is designated by a finger is received by the CCD camera 61 (step S9), and after image processing by the image processing circuit 62, It is stored in the high speed storage device 31.

  As a result, the correspondence between the button on the function selection image 24-2 and the infrared light pulse signal, together with the image of the remote control terminal operation “surface” stored in the high-speed storage device 31 in step S7, is a high-speed database. It is stored in the storage device 31 (step S10). When the user 23 gives an instruction to end the projected “Registered controlled device” image, the image is received by the CCD camera 61 (step S11), and the registration process ends (step S12).

  7 to 9 are diagrams showing the image processing of FIG. 6 (part of the menu image 24-1), of which FIGS. 7A, 7B, 8A, 8B. ) Is a diagram showing the projected image 24, and FIGS. 9A and 9B are diagrams showing the received image 61a.

  With reference to these FIGS. 7 to 9, the process “Step S6 in FIG. 6” projects an image relating to the image of the remote control terminal operation “surface” through the zoom lens 8 from the projector 51. At the same time, a small area to be shown with the remote control terminal facing the CCD camera 61 is also projected in a rectangular pattern, for example. ] Will be described in detail.

  For example, the case where the projected image 24 of the menu image 24-1 is displayed as shown in FIGS. 7A and 7B will be described. Since the number of pixels of the projector 51 is determined, when the size of the projection image 24 is enlarged as shown in FIG. 7A or reduced (changed) as shown in FIG. Change.

  Therefore, regardless of the size of the projection image 24, the pixels included in the same menu button image are exactly the same. For example, in FIGS. 8A and 8B, the pixels included in the button image 24a of the television receiver are 5th or 6th line from the top and 2 from the left regardless of the size of the projection image 24. It is composed of pixels in the ninth to ninth columns.

  Therefore, when the user 23 points to the button image 24a of the television receiver with a finger, the pixel within the same range is pointed regardless of the size of the projection image 24.

  The same can be said for the CCD camera 61. The number of pixels of the CCD camera 61 does not change regardless of the size of the received image 61a, and only the size of each pixel changes. Even if the CCD camera 61 rotates integrally with the projector 51 or is separated, the relative positional relationship is constant at the time of image reception / processing, so that the projected image of the projector 51 is always present. It is possible to receive an image in a form incorporating information on 24 sizes. That is, the size of the received image 61 a of the CCD camera 61 can always be the same as the size of the projected image 24 of the projector 51. As a result, with respect to the image 61a received by the CCD camera 61, as shown in FIGS. 9A and 9B, the pixels indicated by the finger have the same range regardless of the size of the image 61a. It points to the pixel inside. Strictly speaking, regardless of the size of the projected image 24, the size of the received image 61a captured in the CCD camera 61 is no longer meaningful, and only the number of pixels is related to the subsequent processing.

  From the above, regardless of the size of the image, if you have a correspondence relationship between the position and function of each pixel as a database, if you know the position of the pixel pointed by your finger, it corresponds to the pixel at that position. What is necessary is just to perform function control. For example, if a pixel within the range of the button image 24a-1 of the television receiver in FIGS. 9A and 9B is indicated, then the operation screen (function selection image) of the television receiver is displayed on the projector 51. Will be projected from.

(2) Controlled device registration process (memory process)
FIG. 10 is a flowchart showing the registration processing procedure of the controlled device. FIG. 11 is a diagram showing a rotation function selection image 24-3 in the infrared light emitting element 7 of FIG.

  When registering the installation position of the controlled device in the remote control device 1, when the user 23 performs an initial start instruction operation for registration (step S 20), this image is received by the CCD camera 61 through the zoom lens 9. (Step S21). From the image received by the CCD camera 61, it is determined by the image processing circuit 62 whether or not “menu display is requested” (step S22). When there is no request, the registration process is terminated (step S29) and there is a request. At this time, the projection image 24 including the menu image 24-1 is projected from the projector 51 through the zoom lens 8 onto the wall 20b in the vicinity of the user 23 (step S23).

  In the menu image 24-1, when the user 23 designates the target controlled device with a finger, this image is received by the CCD camera 61 (step S24), and the function selection regarding the instructed controlled device is performed from the projector 51. A projection image 24 composed of the image 24-2 is projected. In the function selection image 24-2, when the user 23 instructs "rotation of the infrared light emitting element" with a finger, this image is received by the CCD camera 61 (step S25), and as shown in FIG. A function selection image 24-3 relating to the rotation of the infrared light emitting element 7 is projected.

  In the function selection image 24-3, when the user 23 gives a rotation operation instruction (for example, “clockwise” or “counterclockwise”), this image is received by the CCD camera 61 and is determined by the image processing circuit 62. Under the control of the rotation drive control circuit 33 based on the processing, the servo motor 3 and the rotating shaft 4 operate to rotate only the first rotating body 5. If the user 23 continues to instruct until the infrared light emitting element 7 is directed to the target controlled device, the direction of the infrared light emitting element 7 is adapted to the controlled device (step S26).

  In the function selection image 24-3, when the user 23 selects “Register”, this image is received by the CCD camera 61, and the infrared light emitting element 7 is controlled by the determination processing of the image processing circuit 62 and the control of the CPU 32. The rotation angle suitable for the target controlled device is stored in the high speed storage device 31 (step S27). Thereafter, when the user 23 selects “end” in the function selection image 24-3, the image is received by the CCD camera 61 (step S28), and the registration process is ended (step S29).

(3) Remote Control Processing of Remote Control Device 1 FIG. 12 is a flowchart showing a remote control processing procedure of the remote control device 1 of FIG. 13 to 15 are diagrams showing the processing of FIG.

  When the user 23 performs an initial start instruction operation such as raising his hand to use the remote control device 1 (step S30), this image is received by the CCD camera 61 through the zoom lens 9 (first photographing process). Step S31). From the image received by the CCD camera 61, it is determined by the image processing circuit 62 whether or not “menu display is requested” (step S32, which is the first determination process). In step S41), when requested, the projection image 24 including the menu image 24-1 is projected from the projector 51 through the zoom lens 8 onto, for example, the wall 20b near the user 23 (step of the first projection process). S33, FIG. 12).

  In the menu image 24-1, when the user 23 designates a target controlled device (for example, the television receiver 24a-1) with a finger, this image is received by the CCD camera 61 (second photographing process). In step S34), the image processing circuit 62 determines whether or not “the controlled device is selected” (step S35 as the second determination processing). When there is no selection, the process is terminated (step S41). When there is a selection, the projector 51 projects a projection image 24 composed of a function selection image 24-2 relating to the television receiver 21 instructed to be selected (second). Step S36, which is the projection processing of FIG. 14, FIG. In the function selection image 24-2, when the user 23 instructs to select “channel 2”, for example, this image is received by the CCD camera 61 (step S 37 which is the third photographing process), and the image processing circuit 62. Determination is made (third determination processing), and a pulse corresponding to channel 2 is generated by the pulse generation circuit 41. By this pulse, a TV control infrared ray c1 for selecting channel 2 is transmitted from the infrared light emitting element 7 (transmission processing), and this is received by the infrared light receiving portion 21a of the television receiver 21, and a two-channel TV image is obtained. The image is displayed (step S38).

  In the function selection image 24-2, for example, when the user 23 instructs “end” or “continue” (FIG. 15), this image is received by the CCD camera 61 (step S39), and the image processing circuit 62 “ends”. Whether or not there is an instruction "is determined (step S40). When there is no end instruction, the process returns to step S33, and when there is an end instruction, the remote control process is ended (step S41).

(Effect of the first embodiment)
The first embodiment has the following effects (a) to (c).

  (A) There is no need to place a remote control terminal corresponding to each controlled device on the side, there is an effect that there is no need to change a plurality of remote control terminals, and there is no need to press a complicated push button. Arise. Furthermore, these effects can be produced in the same manner regardless of the number of controlled devices.

  (B) Even if it is a controlled device by a different manufacturer whose standards are not unified, or a conventional controlled device that is not a network compatible specification, the user 23 Can be easily registered.

  (C) By connecting a communication line 35 such as the Internet to the communication control circuit 34, information suitable for each controlled device can be directly projected and displayed at an arbitrary location on the controlled device side. In other words, the remote control device 1 only needs to have an information reception function, which is a problem in a network system using a conventional set-top box, and adds an information reception and transmission function to the set-top box and each controlled device. The problem of adding an information receiving function to a device can be solved. Therefore, the mechanism is simplified, the installation of wired wiring or the like is unnecessary, and since infrared rays are used as the operation signal, microwaves that cannot completely deny adverse effects on the human body do not fly. In addition, the size of the information projection can be enlarged or reduced, and the size most easily seen by the user 23 can be selected.

(External appearance of Example 2)
FIG. 16 is a schematic external view of a remote control device 1A showing Embodiment 2 of the present invention. Elements common to those in FIG. 2 showing Embodiment 1 are denoted by common reference numerals.

  In the remote control device 1A of the second embodiment, a television broadcast receiving function is added to the remote control device 1 of the first embodiment. A third rotating body 13 is added to the rotating shaft 4, and the third rotating body 13 is configured to rotate independently of the first rotating body 5 and the second rotating body 6. In the first embodiment, the zoom lens 8 and the projector 51 are provided on the second rotating body 6. However, in the second embodiment, the zoom lens 8 and the projector 51 are moved to the third rotating body 13, and the zoom lens 8 is moved to the third rotating body 13. And the projector 51 is accommodated in the third rotating body 13.

(Circuit structure of Example 2)
FIG. 17 is a circuit configuration diagram showing the remote control device 1A of FIG. 16. Elements common to those in FIG. 1 showing the first embodiment are denoted by common reference numerals.

  In the remote control device 1A of the second embodiment, a television broadcast receiving circuit 80 and a television antenna 81 are added to the remote control device 1 of the first embodiment, and the television broadcast receiving circuit 80 is connected to the multibus line 37 and controlled by the CPU 32. It has become so. The television broadcast receiving circuit 80 is accommodated in the apparatus control panel 12 of FIG. 16 and constitutes a video receiving means together with the image projection unit 50. Although not shown, the television broadcast receiving circuit 80 is connected to an audio output means (for example, a speaker) that receives an audio signal and outputs the audio.

  The image projection unit 50 includes the zoom lens 8, the projector 51, and the projector interface 52, and has a function of projecting a menu image or a function selection image stored as a database in the high-speed storage device 31 in accordance with an instruction from the CPU 32. ing. In the first embodiment, the projector 8 is used to display a simple menu image or a function selection image. In the second embodiment, however, the projector 8 is used as a projector television. Are controlled by the rotation drive control circuit 33 so as to rotate independently.

(Operation / Effect of Example 2)
The remote control device 1A according to the second embodiment performs substantially the same operation as the remote control device 1 according to the first embodiment. However, since the television broadcast receiving circuit 80 is provided, the television 51 can be placed at an arbitrary position from the projector 51 through the zoom lens 8. Can project images.

  In addition, this invention is not limited to the said Example 1, 2, The various deformation | transformation is possible. Examples of this modification include the following (i) to (iii).

(I) Infrared rays are used as operation signals for remote control, but other signals such as radio waves and ultrasonic waves may be used.
(Ii) The television broadcast receiving circuit 80 may be configured by a television receiver, for example.
(Iii) The external structure, circuit configuration, remote control processing, and the like of the remote control devices 1 and 1A are not limited to those shown in the drawings, and other elements can be added or changed to other elements.

FIG. 3 is a schematic circuit configuration diagram of the remote control device of FIG. 2 illustrating Embodiment 1 of the present invention. 1 is a schematic external view of a remote control device showing Embodiment 1 of the present invention. It is a schematic perspective view which shows the example of installation of the remote control apparatus of FIG. FIG. 3 is an image reception image detailing a tracking method 1 of the remote control device of FIG. 1. FIG. 2 is a schematic circuit configuration diagram showing a CCD camera and an image processing circuit in FIG. 1. It is a flowchart which shows the registration processing procedure of a remote control terminal. It is a figure which shows the image processing of FIG. It is a figure which shows the image processing of FIG. It is a figure which shows the image processing of FIG. It is a flowchart which shows the registration process procedure of a controlled apparatus. It is a figure which shows the function selection image of the rotation in the infrared light emitting element in FIG. It is a flowchart which shows the remote control processing procedure of the remote control apparatus of FIG. It is a figure which shows the process of FIG. It is a figure which shows the process of FIG. It is a figure which shows the process of FIG. It is a general | schematic external view of the remote control apparatus which shows Example 2 of invention. FIG. 17 is a schematic circuit configuration diagram of the remote control device of FIG. 16 illustrating Embodiment 2 of the present invention.

Explanation of symbols

1, 1A Remote control device 7 Infrared light emitting element 8, 9 Zoom lens 10 Infrared light receiving element 30 Control unit 31 High-speed storage device 32 CPU
33 Rotation Drive Control Circuit 34 Communication Control Circuit 36 Memory 40 Infrared Light Emitting Unit 50 Image Projecting Unit 51 Projector 60 Image Receiving / Processing Unit 61 CCD Camera 70 Infrared Light Receiving Unit

Claims (9)

In response to the tracking result of the tracking means for tracking the movement of the user, the first instruction image of the user is captured and the first received image is output by the imaging means whose imaging direction is directed to the user. 1 shooting process,
A first determination process for determining a content of the start instruction operation based on the first received image and outputting a first determination result;
A menu for selecting a controlled device that performs a predetermined operation by receiving an operation signal based on the first determination result by a projection unit that directs the projection direction toward the user in response to the tracking result. A first projection process for projecting an image onto a projection plane in the vicinity of the user;
A second photographing process for photographing a selection instruction operation of the user in the menu image by the photographing means and outputting a second received image;
A second determination process for determining a selection instruction location of the user in the menu image based on the second received image and outputting a second determination result;
Based on the second determination result, the projection unit projects a function selection image for selecting a function of the controlled device corresponding to the selection instruction location in the menu image onto the projection plane. Projection processing;
A third photographing process for photographing a selection instruction operation of the user in the function selection image by the photographing means and outputting a third received image;
A third determination process for determining a user's selection instruction location in the function selection screen based on the third received image and outputting a third determination result;
Based on the third determination result, transmission processing for transmitting the operation signal corresponding to the selection instruction location in the function selection screen from the transmission means to the selected controlled device and performing a predetermined operation; ,
A remote control method comprising: Menu image data for selecting a controlled device that performs a predetermined operation upon reception of an operation signal and function selection image data for selecting a function of the controlled device are stored in advance in a storage unit by the control unit. Memory processing to keep
In response to the tracking result of the tracking means for tracking the movement of the user, the first instruction image of the user is captured and the first received image is output by the imaging means whose imaging direction is directed to the user. 1 shooting process,
A first determination process for determining a content of the start instruction operation based on the first received image and outputting a first determination result;
The stored menu screen data is read by the control means, and is read out based on the first determination result by the projection means in which the projection direction is directed toward the user in response to the tracking result. A first projection process for projecting the menu image onto a projection plane in the vicinity of the user;
A second photographing process for photographing a selection instruction operation of the user in the menu image by the photographing means and outputting a second received image;
A second determination process for determining a selection instruction location of the user in the menu image based on the second received image and outputting a second determination result;
The stored function selection screen data is read by the control means, and based on the second determination result, the read function selection image corresponding to the selection instruction location in the menu image by the projection means. A second projection process for projecting onto the projection plane;
A third photographing process for photographing a selection instruction operation of the user in the function selection image by the photographing means and outputting a third received image;
A third determination process for determining a user's selection instruction location in the function selection screen based on the third received image and outputting a third determination result;
Based on the third determination result, transmission processing for transmitting the operation signal corresponding to the selection instruction location in the function selection screen from the transmission means to the selected controlled device and performing a predetermined operation; ,
A remote control method comprising: Tracking means for tracking the user's movement and outputting the tracking result;
A controlled device that directs a shooting direction toward the user in response to the tracking result, shoots a start instruction operation of the user, outputs a first received image, and performs a predetermined operation by receiving an operation signal The user's selection instruction operation in the menu image for selection is photographed and a second received image is output, and the controlled device corresponding to the user's selection instruction position in the menu image is output. Photographing means for photographing a selection instruction operation of the user in a function selection image for selecting a function and outputting a third received image;
The content of the start instruction operation is determined based on the first received image and a first determination result is output, and the selection instruction location in the menu image is determined based on the second received image. Determining means for outputting a second determination result, determining a user's selection instruction location in the function selection screen based on the third received image, and outputting a third determination result;
In response to the tracking result, the projection direction is directed toward the user, and the menu image is projected on a projection plane in the vicinity of the user based on the first determination result. Projecting means for projecting the function selection image on the projection plane based on:
A transmission means for transmitting the operation signal corresponding to the selection instruction location in the function selection screen to the selected controlled device based on the third determination result;
A remote control device comprising: Storage means for storing data of a menu image for selecting a controlled device that performs a predetermined operation by receiving an operation signal, and data of a function selection image for selecting a function of the controlled device;
Tracking means for tracking the user's movement and outputting the tracking result;
In response to the tracking result, the shooting direction is directed to the user, the start instruction operation of the user is shot, a first received image is output, and the menu screen data read from the storage means The user's selection instruction operation in the menu image is photographed, a second received image is output, and the user in the menu image in the function selection image data read from the storage means Photographing means for photographing the user's selection instruction operation in the function selection image corresponding to the selection instruction portion and outputting a third received image;
The content of the start instruction operation is determined based on the first received image and a first determination result is output, and the selection instruction location in the menu image is determined based on the second received image. Determining means for outputting a second determination result, determining a user's selection instruction location in the function selection screen based on the third received image, and outputting a third determination result;
In response to the tracking result, the projection direction is directed toward the user, and the menu image is projected on a projection plane in the vicinity of the user based on the first determination result. Projecting means for projecting the function selection image on the projection plane based on:
A transmission means for transmitting the operation signal corresponding to the selection instruction location in the function selection screen to the selected controlled device based on the third determination result;
A remote control device comprising: Video receiving means for receiving a video signal and projecting and displaying the video at an arbitrary position by the projection means;
Audio output means for receiving an audio signal and outputting audio;
The remote control device according to claim 3 or 4, characterized by comprising:   The remote control device according to any one of claims 3 to 5, wherein the operation signal transmitted from the transmitting means is radio waves, ultrasonic waves, or infrared rays.   The remote control device according to claim 3, wherein the photographing unit is a video camera.   The remote control device according to claim 3, wherein the projection unit is an image projector.   The remote control device according to claim 5, wherein the video receiving unit is a television receiver.

Images