JP2013172211A - Remote control device and remote control system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a remote control device which enables a user to control a device by using an object near him or her.SOLUTION: The remote control device takes a preliminarily set specific object as an input device and controls a control object device dependently on user's operation on the input device and includes: an operating action detection unit which detects an operating action on the input device; a control information readout unit which obtains contents of operation for the control object device on the basis of contents of the operating action detected by the operating action detection unit and a correspondence relation between preliminarily set contents of the operating action and contents of operation for the control object device; and a control information transmission unit which transmits control information read out by the control information readout unit, to the control object device.

Description

  The present invention relates to a remote operation technique for operating a predetermined device by an operation performed on an arbitrary object.

  Conventionally, a remote controller (hereinafter referred to as “remote controller”) has been mainly used to satisfy the need to operate an electronic device or the like from a remote location. The remote control has the great merit that it can be operated with very little effort by just pressing a button, but there is a problem that it often falls into a state where it cannot be operated because the location of the remote control is lost.

  In order to solve this problem, there has been proposed a method of performing an operation by a gesture using an operator's hand which does not have to be lost (see Patent Document 1 below).

  In addition, a technique has been proposed in which a temporal positional change or angular change of an arbitrary object that is obtained using a photographing apparatus is used for an operation (see Patent Document 2 below). According to this method, the problem that tactile feedback cannot be obtained can be solved.

JP 2001-5975 A JP 2006-222622 A

  However, in the gesture input used in the technique described in Patent Document 1, the operator generally needs to move his hand greatly, and in order to prevent erroneous input due to hand movement as a daily operation, a special operation is performed. The operation required a large operation compared to the remote control, and it was difficult to operate in a narrow place, and the operation itself was easy to feel troublesome. Also, since tactile feedback cannot be obtained for the operation, it is difficult to check whether the operation is possible.

  Further, in the technique described in Patent Document 2, it is necessary to move an object itself used as an input device, and it has not been solved with respect to the point that a large operation is required as compared with the operation of a remote controller. Furthermore, even those close to us, large ones and heavy ones could not be practically used as input devices.

  The present invention provides a remote operation technique that enables an apparatus to be operated using an object that is close to the user and can be freely used by a user, not a shared dedicated apparatus such as a remote controller. For the purpose.

  According to one aspect of the present invention, a remote operation device that operates a control target device depending on an operation of an operator with respect to the input device using a preset specific object as an input device, the remote control device with respect to the input device. An operation operation detection unit for detecting an operation operation, the content of the operation operation detected by the operation operation detection unit, a correspondence relationship between the content of the operation operation set in advance and the content of operating the device to be controlled; A control information reading unit that obtains the content of the operation on the corresponding control target device based on the control information, and a control information transmission unit that transmits the control information read by the control information reading unit to the control target device. A remote control device is provided.

  A distance information acquisition unit configured to acquire distance information to the specific object, wherein the operation motion detection unit recognizes the shape of the specific object using the distance information, and the shape of the specific object; It is preferable to detect the operation of the input device based on the above.

  Furthermore, it has a video acquisition unit for acquiring video information of the specific object, and the operation motion detection unit detects the operation motion using both the shape of the object and the video information. You may do it.

  The operation motion detection unit detects the content of contact with the specific object, and responds based on the content of the operation motion set in advance and the content of contact detected by the operation motion detection unit. It is preferable to obtain the contents of the operation action.

  It is preferable that the content of the operation is a temporal change in the relative position between the specific object and the user's hand / finger or a locus thereof.

  Furthermore, it has an input device stationary detector for detecting the stationary state of the input device, and the input device stationary detector determines whether or not the specific object is in a stationary state. It is preferable to determine that the operation is not an operation regardless of the detection result of the operation operation detector.

  In addition, the operation operation detection unit may detect an operation operation using both the distance information and the video information.

  It is preferable that the distance information acquisition unit includes at least two or more video acquisition units and estimates distance information from a plurality of video information obtained from the video acquisition unit.

  Furthermore, the present invention further includes the input device according to any one of the above, wherein the input device uses the specific object, or is drawn or displayed on a medium instead of the specific object. A remote device system characterized by using graphics.

  According to the present invention, it is possible to operate a device by an operation of touching a nearby object. In addition, it is possible to prevent malfunctions when the object is used for an original purpose that is not an operation of the device.

It is a functional block diagram showing an example of 1 composition of a remote control system concerning a 1st embodiment of the present invention. It is a functional block diagram which shows the example of 1 structure of an input device registration part among remote control systems. It is a functional block diagram which shows one structural example of the operation content registration part among remote control systems. It is a functional block diagram which shows one structural example of an operation | movement detection part among remote control systems. 1 is an external view showing a configuration example of a remote control system according to a first embodiment of the present invention. It is a flowchart figure showing the flow of a process at the time of registering the input device which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of the operation action identification table which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of the control information identification table which concerns on the 1st Embodiment of this invention. It is a flowchart figure showing the flow of a process at the time of operating the apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the figure used as an input device which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the operation action identification table which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the operation action identification table which concerns on the 3rd Embodiment of this invention.

  Hereinafter, preferred embodiments of a remote control system according to the present invention will be described in detail with reference to the accompanying drawings.

  In this specification, the term “motion” refers to general user movements captured by the remote control device according to the present invention, and includes, for example, finger movements for operations such as taps and swipes, but is not necessarily intended for device control. .

  Control refers to processing inside the device that is performed to change the state of the target device, for example, processing such as turning on or off the television.

  The operation refers to an action given from the outside of the device in order to cause control, for example, an operation such as pressing a button with the remote controller pointing at the device or pressing a button provided on the device.

  The operation operation is generally only an operation irrelevant to the device control. However, by using the remote operation device according to the present invention, it can be handled as an operation with respect to the input device for the purpose of device control. 8, FIG. 9, and the like.

(First embodiment)
A first embodiment of the present invention will be described with reference to the drawings. FIG. 1A is a block diagram illustrating a configuration example of a remote control system according to the first embodiment of the present invention. FIG. 1B is a functional block diagram illustrating a configuration example of the input device registration unit in the remote operation system. FIG. 1C is a functional block diagram illustrating a configuration example of an operation content registration unit in the remote operation system. FIG. 1D is a functional block diagram illustrating a configuration example of the motion detection unit in the remote operation system. FIG. 2 is a drawing showing a part of the block diagram of FIG. 1A as a specific appearance image example, and the numbers in the drawing are attached in correspondence with FIG. 1A.

  As shown in FIG. 1A, the remote operation system A according to the present embodiment includes an input device 100 and a remote operation device B, and shows an example of controlling a control target device 103. The remote operation device B includes an operation detection device 101, a control signal transmission device 102, a control unit 101c, and a memory 101d. The motion detection device 101 is an operator 100a based on the input device 100 or a photographing unit 101a composed of the input device 100 and a camera for acquiring the surrounding video and distance information, and the video and distance information obtained by the photographing unit 101a. A motion detection unit 101b that detects the motion of The control signal transmission device 102 reads out a control signal for an operation corresponding to the motion detected by the motion detection device 101, and transmits the control signal to the control target device 103. The control unit 101c has an input device registration unit 1 and an operation content registration unit 3, and the memory 101d has a table storage unit 5 for storing a table to be described later. Generally, a program for executing processing is stored. The control target device 103 exemplifies a television receiver (hereinafter referred to as “television”) here, and the manner in which the same control as when the television is operated with a remote controller is described below.

  FIG. 1B is a functional block diagram illustrating a configuration example of the input device registration unit 1. The input device detection unit 1-1 detects the input device 100, and the input device recognition unit 1-2 recognizes the input device 100. , An operation type setting unit 1-3 for setting an operation available for device operation, and an input device registration processing unit 1-4 for registering an input device.

  The operation content registration unit 3 includes an input device designating unit 3-1 for designating the input device 100, an operation operation designating unit 3-2 for designating an operation operation, and a corresponding operation selection unit 3- for selecting a corresponding operation. 3, a setting content registration unit 3-4 for registering setting content, and a display unit 3-5.

  FIG. 1D is a functional block diagram illustrating a configuration example of the operation detection unit 101b. The motion detection unit 101b includes a video / distance information acquisition unit 11, an input device and operation preparation detection unit 12, an input device motion detection unit 13, an operation motion detection unit 14, a registered motion comparison unit 15, and an input device. The stationary detection unit 16, the operation information reading unit 17, and the operation information transmission unit (instruction unit) 18 are provided.

  As shown in FIG. 2, in the remote control system including the remote control device B according to the present embodiment, an input device (subject) 100 is an arbitrary object to be subjected to an input operation. In the present embodiment, chewing gum An example of this bottle will be described below. For example, the input device 100 is placed on a table 110 that is not a subject. The control target device 103 is, for example, a home electric appliance to be operated, and is a television in the present embodiment. In FIG. 1A, for the sake of convenience, the remote operation device B and the control target device 103 are drawn separately, but in this embodiment, the operation detection device 101 and the control signal transmission device 102 are incorporated in the control target device 103 in advance. An example will be described. The existing remote controller 107 can also be used as another input device. Here, a photographing unit (camera) 101 a attached to the television 103 is provided at a position where the input device 100 can be photographed.

  The input device registration process will be described with reference to the flowchart of FIG. First, the remote control device B is shifted to a registration mode. Since the remote control device B is incorporated in the control target device 103, the remote controller 107 can be used for the operation for shifting to the registration mode. Further, the remote controller 107 is operated to start processing, and input device registration is selected in step S0, and the flow proceeds to the input device registration flow.

  In subsequent step S1, the input device detection unit 1-1 detects the input device. First, the table 110 around the state where the input device 100 is removed is photographed by the photographing unit 101 a of the motion detection device 101. At the time of input device registration, it is preferable that the captured video is displayed on the screen of the control target device 103 as needed. If there is no problem in checking the screen, the remote controller 107 is operated, and then the imaging unit 101a captures a state where the input device 100 is placed on the table 110. The input device 100 is specified from the difference between the two captured images of the image of the table 110, the table, the input device 100, and the image of the table 110, and displayed on the screen. After confirming that the input device 100 has been correctly identified, the remote controller 107 is operated to proceed to the next step. At this time, the input device 100 is given unique identification information (identifier, ID) for distinguishing it from other registered input devices.

  In step S2, the input device recognition unit 1-2 recognizes the size and shape of the input device 100 by, for example, rotating and photographing the input device 100 detected in step S1. Since the photographing unit 101a has a distance measuring function that acquires not only an image but also distance information, it is possible to know the distance in the optical axis direction of the camera at an arbitrary point in the image. Also, when camera parameters such as the angle of view and focal length of the lens of the photographing unit 101a are used, when a plane perpendicular to the optical axis of the camera is photographed, the length of a certain number of pixels in the photographed image is on the actual plane. It is possible to determine how many distances it corresponds to.

  Therefore, the distance (x-axis, y-axis) on the plane perpendicular to the optical axis of the camera and the distance (z-axis) in the optical axis direction of the camera can be known for any point in the image. The three-dimensional coordinates in the xyz space can be determined. Therefore, the absolute size and shape of each part of the input device 100 can be obtained.

  Furthermore, by shooting while rotating the input device one by one in the horizontal direction, the blind spot in shooting is reduced, and the possibility that the motion detection device 101 loses sight of the input device 100 during actual operation is reduced and prepared in advance. By comparing with the shape template, it is possible to perform shape recognition processing such as whether the input device is a cylinder or a rectangular parallelepiped.

  Here, a configuration of a camera capable of acquiring both video information and distance information will be described. Such a camera configuration method can be roughly divided into two. One is a method for acquiring video information and distance information using different sensors, and the other is a method for acquiring a plurality of video information and estimating the distance information from the obtained plurality of video information. In either case, a general CMOS image sensor or CCD image sensor can be used to acquire video information. There are several methods for acquiring the distance information separately. As a typical method, TOF is used to obtain the distance by irradiating the subject with infrared pulsed light and measuring the phase delay of the pulse in the reflected light. There are a (Time of Flight) method and a method of obtaining a distance from a pattern distortion on an object by irradiating a geometric pattern by an infrared laser. For distance information estimation from multiple video information, stereo matching is performed on images taken by two cameras arranged in parallel, and how far the corresponding points between the cameras are on the captured image, There is a method of converting it into a distance using a camera parameter. The effect of the present invention can be obtained regardless of which method is used to acquire the distance. However, the higher the distance resolution and accuracy, the better the detection accuracy of the operation according to the present invention. If the effect of lens distortion, etc. remains in the captured image, the image will be compressed in the surroundings compared to the center of the screen, causing an error in determining the length of the object, so sufficient distortion correction is performed. It is desirable to keep it.

  When a video sensor and a distance information sensor are used to use the TOF method or the like, the distance accuracy itself depends on the ability of the distance information sensor, but the distance information is that of the subject at any coordinate of the video. It is difficult to take a correct response. In addition to the fact that the optical characteristics of the two sensors are different, it is difficult to accurately match the angle of view, etc. In addition, because of the parallax between the two sensors, the distance information of a certain coordinate indicates which coordinate of the image Although it is possible to determine whether the subject is a subject, the relationship is not one-on-one, so that multiple distance information corresponding to the coordinates of the video appears or there is no corresponding distance information. It is.

  As much as possible, adjust the optical characteristics of each sensor and use a half mirror to prevent parallax in the input, so that there is not much coordinate shift between the image and distance information. It is desirable to keep it.

  On the other hand, when estimating distance information using stereo matching, the distance to the subject reflected in the coordinates is obtained for any coordinates in the video, based on the principle of stereo matching. The difference in coordinates between the image and the distance information does not need to be taken into account so much. However, since it affects the accuracy of the distance information, the cameras used for stereo matching should have similar characteristics. It is desirable to keep the optical axes parallel.

  Here, the reason why the absolute size and shape of the input device 100 are obtained is that recognition of an object that does not limit the target is inherently difficult, but this recognition is easily performed. Many of the object recognition techniques are based on the premise that characteristic elements that can distinguish recognized objects are known in advance. For example, in the case of human face recognition, search for objects that are likely to be human faces from video based on the shape of the entire face, skin color, and the relative positional relationship between eyes, nose, and mouth. . However, in this embodiment, since the input device is not limited, characteristic elements as a video are not known in advance, and therefore it is very difficult to recognize a specific object only with information as a video. . However, information on the size and shape of an object is a characteristic element that distinguishes the object from other objects. For example, two objects with similar shapes and similar colors but different sizes are difficult to distinguish from each other only with information as an image, but can easily be distinguished if there is size information. can do. Alternatively, when there are two objects of the same shape and different colors, the two cannot be distinguished only by information on the size and shape, but can be easily distinguished by using information as an image. In this way, by using a combination of information as an image and size and shape information, it is possible to greatly reduce the difficulty of recognition.

  Subsequently, in step S3, the action type setting unit 1-3 defines what action is performed on the input device 100 registered by the operator 100a as a special action for device operation, that is, an operation action. Do. In the present embodiment, taps, swipes, and traces will be described below as examples of operation operations using contact with the input device 100.

  Since contact with the input device 100 is determined, it is necessary to detect the hand of the operator 100a. However, hands have relatively few individual differences such as color and shape, so it is relatively easy to detect a representative pattern as a template and compare the moving object in the captured video with the template. It is. The operation of touching the input device 100 with the operator's hand and releasing it immediately is called a tap. In order to detect the tap, it is only necessary to detect that the operator's hand and the input device 100 overlap each other in the captured image and that the two pieces of distance information substantially match. By setting these two as detection conditions, it is possible to reduce the number of erroneous operations by excluding the case where they appear to overlap on the image even though the distance is long.

  Here, it supplements about the coincidence of distance information. If the hand of the operator 100a and the input device 100 are in contact with each other and the contact portion can be photographed from the photographing unit 101a, the distance information is almost the same, but the contact portion is blocked by the hand. It may be. In this case, there is a difference in distance information between the hand and the input device 100 by the thickness of the hand as viewed from the photographing unit 101a. Therefore, considering that the thickness of the hand changes according to the thickness of the hand and the angle viewed from the photographing unit 101a, contact is made if the difference in distance information between the hand and the input device 100 is a certain value or less. It is desirable to assume that In this case, the threshold of the difference in distance information is suitably about 2 to 3 cm, for example, when touching with a hand. If the resolution and accuracy of distance information are insufficient, a larger value may be used. In addition, the touch determination is not performed at the moment when the input device is touched, but is also performed by determining whether the input device 100 is not moving when the touch is released. Therefore, it is possible to prevent misrecognition when it is not an operation purpose. Providing a lower limit or an upper limit for the contact time determined to be touched for operation is also effective in preventing erroneous recognition. It can also be set as a different operation depending on the tapping position.

  For example, different operation operations may be performed on the upper and side surfaces of the chewing gum bottle that is the input device 100, or whether the side surface of the bottle is a tap from the right side or the left side as viewed from the photographing unit 100a. Each can be set as a different operation. Further, it is possible to distinguish a case where the tap is continuously performed a plurality of times and to set different operation operations such as a single tap, a double tap, and a triple tap depending on the number of times the tap is performed.

  It is also possible to detect a swipe in the same way as a tap. Swipe refers to moving only the hand without moving the input device 100 from a state where the hand touches the surface of the input device 100. Based on the positional relationship between the operator 100a and the hand of the operator 100a, it is possible to determine the vertical and horizontal directions as viewed from the operator 100a, and to set different operation operations depending on the swipe direction based on the determined direction.

  Similar to swipe, which is an operation of moving the hand while touching the surface with respect to the surface, an operation of moving the hand along the surface with respect to the line can also be used. This is called a trace. Tracing along a specific edge of the input device 100 or a specific portion of a pattern on the surface of the input device 100 can also be set as one operation operation. Further, it is possible to set different operation operations depending on the direction of the trace, such as whether the trace is performed clockwise or counterclockwise.

  Note that the content of the operation operation can be grasped as a change in time or a locus of the relative position between the specific object and the user's hand / finger.

  In addition, considering the operation performed when the bottle of chewing gum, which is the input device 100 in this embodiment, is used in a normal application, the bottle does not move after being placed on the table or lifted. Grab and open the lid. This operation differs greatly from the normal operation by using operations that are not used in normal applications such as parallel movement without moving away from the table, such as moving the bottle sideways and rolling it as operation operations. It is also possible to prevent malfunction.

  With respect to such an operation, what can be used in the registered input device is set in this step. The operation operation identification information as shown in the operation operation identification table of FIG. 4 is assigned to each operation operation set here. The operation operation identification information is set for each input device 100. In subsequent step S4, the input device registration processing unit 1-4 obtains information for identifying the input device 100 obtained in step S2 and operation operation identification information usable in the input device set in step S3. , It is registered in the motion detection apparatus 101.

  Next, the operation registration by the operation content registration unit 3 that associates the operation operation on the input device 100 with a specific operation of the device will be described with reference to FIG. First, the remote controller is operated to move to the operation registration flow. In step T1, since a list of input devices registered in the motion detection device 101 is displayed on the display unit 3-5, the input device 100 used for the operation of the device is designated therefrom. Here, the screen of the television as the control target device 103 also serves as the display unit 3-5. While viewing the list of registered input devices displayed on the screen of the television 103, the input device designation unit 3- 1 is instructed. In step T2, a list of the operation actions of the input device selected in step T1 is displayed on the display unit 3-5. Which of the operation actions is selected, the operation action contents are designated using the remote controller. An instruction is given to the unit 3-2. In step T3, an instruction is given to the corresponding operation selection unit 3-3 using the remote controller as to which operation content is to be set for which device to be controlled in the selected operation. Regarding the control target device 103 and its operation content, since the motion detection apparatus 101 is incorporated in the control target device 103 in this embodiment, it has already been registered in the control target device 103 in the process of manufacturing as a product. Here, by referring to a database provided on the Internet based on identifiable information such as the manufacturer name and model number of the device to be controlled as well as information registered in advance, a list of operation contents is acquired and used. May be. When all necessary operation settings have been set, the setting content registration unit 3-4 uses the operation relationship identification information and the control content set so far as the control information identification table as an operation detection device in step T4. 101.

  FIG. 5 shows an example of the operation action identification information (ID) and control (action) contents assigned here. For example, from the operation identification table of FIG. 4, the operation operation identification information indicating the operation of single-tapping the upper surface of the input device 100 is indicated by “AA0A1”. Since the operation operation identification information “AA0A1” is associated with the help display of the television from the control information identification table of FIG. 5, the operation of single-tapping the upper surface of the input device 100 is a control device called the help display of the television. Is assigned to the operation. Since the operation action identification information is internal data of the action detection apparatus 101, it is not displayed on the screen. Although no specific example is shown in the figure, the control information is also stored in the form of identification information indicating specific control contents, and is used in the control signal transmitting apparatus 102 described later.

  FIG. 6 is a flowchart showing a flow of processing when the control target device 103 is actually operated using the input device 100. Again, the remote control 107 is operated to shift to the operation detection mode. When performing an operation, the input device 100 is placed on the table 110. The place where the input device 100 is placed does not necessarily have to be on the table 110 used for registration, and may be on the floor or the like as long as it is within a range that can be photographed from the photographing unit 100a. In the motion detection mode, the video / distance information acquisition unit 11 in the motion detection unit 101b of the motion detection device 101 always acquires video and distance information (step U1).

  In step U2, the input device and operation preparation detection unit 12 detects whether or not the registered input device is in the video, and detects whether or not there is a hand in the captured image. If both the registered input device and the operator's hand are detected, the process proceeds to step U3. If not detected, the process returns to step U1, and photographing is continued until it is detected.

  In step U3, the input device motion detector 13 determines whether or not the detected input device is stationary depending on whether or not the stationary state has been continued for a predetermined time or more. If it can be confirmed that it is stationary, the process proceeds to the next step U4.

  In step U4, the operation motion detection unit 14 detects the movement of the operator's hand, and determines whether the registered motion comparison unit 15 matches the registered operation motion. When it is determined that they match, it is determined whether or not the input device 100 is stationary even after the operation is completed. Here, the reason for determining whether or not it is in a stationary state is that the input device 100 is a bottle of chewing gum, and is used for taking out the chewing gum that is originally used in addition to operating the control target device 102. This is because there is a possibility that an action that matches the registered operation action may be performed during the act.

  In this way, when an operation corresponding to the operation operation is performed for an original use of the input device other than the operation purpose, the position of the input device is changed by being lifted, or the shape is changed by opening the bottle lid. In many cases, the state of the input device changes. Therefore, by determining whether or not the input device 100 is in the stationary state by the input device stationary detection unit 16, it is possible to distinguish whether the operation has been performed to operate the control target device 103 or not. It is possible to prevent an erroneous operation caused by determining an operation that is not the operation purpose as an operation operation. If it is determined that the input device is still after the operation operation and the operation is for the operation, the process proceeds to step U5, where the operation information reading unit 17 refers to the previous operation operation identification table to obtain the operation operation identification information. The operation information transmission instruction unit 18 acquires the operation operation identification information to the control signal transmission device 102, and returns to step U4 again to monitor the movement of the operator's hand in preparation for the next operation. Corresponding to the detection of steps U2, U3, U4, respective determination steps of steps U7, U8, U9, U10 are provided.

  The control signal transmission device 102 determines the control content to the control target device 103 corresponding to the operation motion identification information transmitted from the motion information detection device 101 using the control information identification table of FIG. The controlled signal is issued to the control target device, and the control target device 103 is operated. As described above, it is possible to operate a device using an object that is close to the operator without being caught by a limited number of shared dedicated devices such as a remote controller.

  In the present embodiment, the example in which the motion detection device 101 and the control signal transmission device 102 are incorporated in the input device 100 has been described. However, the operation detection device 101 and the control signal transmission device 102 may function independently without being incorporated. Similarly, the photographing unit 101a and the motion detection unit 101b of the motion detection device 101 may be separately provided.

  It should be noted that the installation location of the imaging unit 101a is preferably installed above the plane on which the input device 100 is placed. By doing so, since it looks down from the top, even when the input device 100 does not have a very high height, the area of the input device 100 in the captured video increases, and the input device itself is recognized and the operation to the input device. This is because it becomes easier to recognize this. When the photographing apparatus 101a and the control apparatus 107 are attached to the ceiling of the room, the entire room can be monitored, which is particularly useful when there are a plurality of devices to be controlled.

  In the present embodiment, only the type of operation and the target location of the operation are distinguished. However, the same operation is assigned to different operation operations depending on the orientation and front and back of the input device 100 as viewed from the operator, and operations that can be assigned. The operation may be increased. Regarding the detection of the input device 100, even if the size does not match that of the registered input device 100, if only the shape such as a columnar shape or a rectangular parallelepiped shape matches, the input device 100 can be substituted. It may be. In this way, for example, when a coffee cup is registered as an input device, even if the coffee cup used for registration is being washed in the dishwasher, the registration work is performed using another coffee cup. It can be used for operations without having to redo.

  A plurality of registered input devices may be used simultaneously by a single operator, or may be used individually by a plurality of operators. In the latter case, when another operator different from the operator who has operated the device until then appears, it is possible to perform processing different from the case where there is only one operator. For example, if the first operator has operated the TV so far, and another operator who has started operation later tries to change the channel, confirm with the first operator whether the channel can be changed really. Or a confirmation screen asking whether to switch the TV to a two-screen display so that the user can continue to watch the original channel.

  As described above, in the present embodiment, the device can be operated by an operation of touching a nearby object. In addition, it is possible to prevent malfunctions when used for an object intended for an object that is not an operation of the device.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment, the input device 100 is an object, but may be a graphic or the like on which printing or the like has been performed.

  The input device 100 in the present embodiment is a paper on which a geometric figure is printed by a printer or the like, in other words, a figure displayed on a medium. When paper is used for the input device, the distance information hardly changes when the image of the input device and the distance information are acquired in the registration work, so that it can be determined that the input device has little thickness. In such a case, the operation of the control target device 103 is performed by performing an operation on the graphic on the surface of the input device instead of the paper itself as the input device.

  That is, when a hexagonal figure as shown in FIG. 7 is used, a tap is performed on any one of the hexagon 200 at the center and the six surrounding triangles 201 to 206, or a plurality of taps are applied to any part. It can be registered as a different operation depending on whether it is performed. An example of the registered data is shown in the operation action identification table of FIG. In order to correctly discriminate even when the pattern is rotated, only the triangle 201 is filled so that it can be distinguished from the others. Further, in order to correctly detect a tap by hand, it is desirable that each part of a figure with a different operation has a size that is at least four times as large as the fingertip. In addition to assigning operations by dividing one figure into several areas, it is also possible to print a plurality of figures and assign operations in units of figures.

  Alternatively, since the absolute size can be determined from the distance information, it is possible to print a print with only the same graphic size changed, and assign a different operation for each size. Alternatively, a plurality of figures whose colors are changed may be printed and distinguished by colors. The registered figure is always pasted to a nearby chair or table when using the device to be controlled, so that even if the operator does not have any object that can be used as an input device, it can be operated Can do.

  According to the present embodiment, since the apparatus can be controlled based on a figure or the like instead of an actual object, there is an advantage that it is more convenient to use.

(Third embodiment)
Next, a third embodiment of the present invention will be described.
The input device in this embodiment is a table, for example. Since the table is basically for placing objects, it is often difficult to touch a specific part on the surface of the table, but the edge of the table, that is, the boundary with the surrounding space, is on the table. In many cases, it remains as an independent space that does not interfere with what is placed. Therefore, it is possible to trace the edge of the table.

  Because the table is somewhat large, there is only one or two edges of the table that a person can touch without bothering to move around. For this reason, it is better to treat all of them in the same way without considering which edge is touched. The distinction is whether the edge touched with one hand or both hands, and the trace direction was reversed several times, starting in either the clockwise or counterclockwise direction before touching and releasing the hand. When touching with both hands, it is possible to detect whether the directions of the traces of the left and right hands are the same or opposite. FIG. 9 shows an example of the operation identification table created only by tracing as described above. In this way, if you set the operation using only the edge of the table, even if there is nothing on the table and it is tidy, there are various things scattered around, that is, without depending on the state of the table The device can be operated in the same way.

  In the above-described embodiment, the configuration and the like illustrated in the accompanying drawings are not limited to these, and can be changed as appropriate within the scope of the effects of the present invention. In addition, various modifications can be made without departing from the scope of the object of the present invention.

  In addition, a program for realizing the functions described in the present embodiment is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to execute processing of each unit. May be performed. The “computer system” here includes an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.

  The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the above-described functions, or may be a program that can realize the above-described functions in combination with a program already recorded in a computer system.

  The present invention can be used as a remote operation system.

DESCRIPTION OF SYMBOLS 100 ... Input device, 101 ... Motion detection apparatus, 101a ... Imaging | photography part, 101b ... Motion detection part, 102 ... Control signal transmission apparatus, 103 ... Control object apparatus, 110 ... Table, 200-206 ... The figure handled as an input device.

Claims (9)

A remote operation device that operates a control target device depending on an action of an operator with respect to the input device, using a predetermined specific object as an input device,
An operation operation detector for detecting an operation operation on the input device;
Based on the content of the operation motion detected by the operation motion detection unit and the correspondence between the content of the operation motion set in advance and the content of operating the device to be controlled, to the corresponding device to be controlled A control information reading unit for obtaining the content of the operation of
A remote operation device comprising: a control information transmitting unit that transmits the control information read by the control information reading unit to the control target device. The operation motion detector is
It has a distance information acquisition unit for acquiring distance information to the object,
The remote control device according to claim 1, wherein the distance information is used to recognize a shape of the object, and the preset input device is detected based on the shape of the object. The operation motion detector is
A distance information acquisition unit for acquiring distance information to the object;
A video acquisition unit for acquiring video information of the object,
The shape of the object is recognized using the distance information, and the preset input device is detected based on both the shape of the object and the video information. Remote control device. The operation motion detector is
Using the distance information acquisition unit to acquire distance information to the operator's hand / finger,
4. The operation operation is detected by using both distance information to the specific object and distance information to the operator's hand / finger. The remote control device according to item 1. The operation motion detector is
Detecting the content of contact with the specific object;
The content of the operation operation corresponding to the content of the operation operation set in advance and the content of the contact detected by the operation operation detection unit is obtained. The remote control device according to item.   The remote operation device according to claim 5, wherein the content of the contact is a temporal change or a locus of a relative position between the specific object and the hand / finger of the operator. further,
An input device stationary detector that detects a stationary state of the input device;
The input device stationary detector is
Determine whether the specific object is stationary, and if not,
7. The remote operation device according to claim 1, wherein the remote operation device determines that the operation is not an operation operation regardless of a detection result of the operation operation detection unit. 8.   The distance information acquisition unit includes at least two or more image acquisition units, and estimates distance information from a plurality of image information obtained from the image acquisition unit. The remote control device according to any one of the above. The remote control device according to any one of claims 1 to 8,
The remote operation system uses the specific object, or uses a figure drawn or displayed on a medium instead of the specific object.

Images