JP2007072548A - Object indication specification system, information terminal, operation object equipment and object indication specification method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object indication specification system, in which a user can specify an object nearby and display information of the object. <P>SOLUTION: This object indication specification system comprises an indication device 20 including a light emitting part 201 emitting light by the user's operation; operation object equipment 300 including an operation object information storage part 303 storing operation object information illustrating the own operation object equipment or showing the state thereof, a second communication part 302, and a response part 304 responding to an acquisition request of operation object information; and the information terminal 101 including an image photographing part 101, an image analysis part 106 specifying an object when superposition of the light emitting part 201 with an identification part 301 of the operation object equipment 300 is detected, a first communication part 103, and an operation object information acquisition part 105 requesting operation object information to the specified operation object equipment and acquiring it therefrom, and a display part 102 displaying the acquired operation object information with the photographic image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an object instruction specifying system in which a user specifies and specifies a nearby object. Further, an instruction device is positioned between the information terminal and the operation target device to be specified, and an object instruction specifying system for specifying the operation target device, an information terminal constituting the device, an operation target device, and an object instruction It relates to a specific method.

  In recent years, with the advancement of network technology, it has become easy to construct a network, and an increasing number of users construct a home network by connecting a digital device equipped with a network connection function or a personal computer by wire or wireless. Mobile terminals such as mobile phones equipped with a network connection function and PDAs (Personal Digital Assistants) are also widespread, and there is an increasing need to operate devices connected to the network using these mobile terminals.

  Conventionally, the following techniques are known as techniques for specifying and specifying objects such as these devices. Research has been conducted on a user interface using a head-mounted display that a user always wears on the head and obtains information from the outside world. For example, according to the body-mounted information display device disclosed in Patent Document 1, it is possible to specify a target that the user is looking through the information display device by direction, position information, distance, and the like, and display information related to the target. Furthermore, a technique is disclosed that enables to display more detailed information about an object by selecting a menu displayed on the information display device using a light emitting device attached to a part of the user's hand. ing.

In addition, according to the pointing device disclosed in Patent Document 2, information emitted from the light emitting unit is obtained by the CCD camera using the pointing device having the light emitting unit carried by the user and the CCD camera installed in the operation target device. Thus, a technique for executing a user instruction is disclosed.
WO 97/31477 pamphlet (page 53, Fig. 17) Japanese Patent Laying-Open No. 2004-227230 (page 30, FIG. 14)

  In the above conventional method, the operation target device is specified by the direction, position information, distance, and the like, and therefore, compared with the case where the target is specified as a building or the like, for example, a small device such as a digital device or a housing facility is used. There is a problem that the target cannot be easily identified. Further, the method of installing a CCD camera on the target and specifying the light emitting unit of the user's pointing device has a problem that the configuration is complicated because the CCD camera must be installed on all the operation target devices.

  An object of the present invention is to solve such a problem, and an object thereof is to provide an object instruction specifying system that allows a user to specify an object in the vicinity and to perform an instruction operation with a simple configuration.

In order to solve the above problems, an object instruction specifying system of the present invention is an object instruction specifying system that specifies an operation target device by positioning an instruction device between an information terminal and an operation target device to be specified. The pointing device includes an input unit for inputting a user operation and a light emitting unit that emits light based on the user operation, and the operation target device is a visible identification unit for identifying the other operation target device. An operation target information storage unit that stores operation target information indicating the description or status of the device to be operated by itself, a second communication unit that transmits and receives an operation target information acquisition request and an acquisition response, and an external acquisition request A response unit that generates an acquisition response including the operation target information stored in the operation target information storage unit, and the information terminal captures a video in the vicinity of the operation target device to be identified. And A video analysis unit that identifies, as an operation target device, an object that identifies the identification unit when detecting an overlap between the light emitting unit of the pointing device and the identification unit of the operation target device based on the video captured by the image capturing unit; The first communication unit that transmits and receives the operation target information acquisition request and the acquisition response, and the operation target information acquisition request for the operation target device specified by the video analysis unit are generated, and the acquisition response as a response to the generated acquisition request An operation target information acquisition unit that acquires the operation target information included in the acquisition response acquired by the operation target information acquisition unit on the video captured by the video imaging unit, An object having an identification unit identified by the video analysis unit is identified as an operation target device. Therefore, it becomes possible to determine the operation target device that the instruction device superimposes on the video captured from the user's viewpoint, and even if it is a small operation target device, the instruction device is placed between the information terminal and the operation target device to be specified. It is possible to easily identify the operation target device.

  Further, in the object instruction specifying system according to the present invention, the operation target information includes commentary information indicating contents describing the device to be operated by itself, and the video analysis unit of the information terminal is based on the video taken by the video shooting unit. Further, the display position of the identification unit of the operation target device is analyzed, and the display unit of the information terminal determines the display position of the explanation information based on the display position of the identification unit of the operation target device analyzed by the video analysis unit. ,indicate. Therefore, the explanation information of the operation target device can be displayed near the position of the operation target device, and the description of the specified operation target device can be easily confirmed even when a plurality of devices are displayed. .

  In the object instruction specifying system of the present invention, the display unit of the information terminal determines the display position of the explanation information so that it does not overlap the display position of the identification unit of the operation target device, and displays the explanation information in a balloon shape. To do. As a result, information can be displayed on the display unit so as not to overlap the identification unit of the operation target device, and the user can be presented with a particularly easy-to-recognize relationship between the information to be displayed and the operation target device. .

  In the object instruction specifying system according to the present invention, the operation target information includes menu information indicating the content stored in the operation target device or the operation function, and the video analysis unit of the information terminal displays the video captured by the video shooting unit. The display position of the identification unit of the operation target device is further analyzed based on the display position of the menu information display unit based on the display position of the identification unit of the operation target device analyzed by the video analysis unit. Determine and display. Therefore, the content or operation function stored in the operation target device can be performed near the image of the operation target device, and the content or operation function of the specified operation target device can be displayed even when a plurality of devices are displayed. It can be easily confirmed.

  Further, in the object instruction specifying system of the present invention, the pointing device further includes a light emission switching unit that switches a light emission state of the light emitting unit based on a user operation, and the video analyzing unit of the information terminal is taken by the video photographing unit. Based on the video, the light emitting state of the light emitting unit of the pointing device is further analyzed, and the information terminal displays the operation target information displayed on the display unit according to the light emitting state of the light emitting unit of the pointing device analyzed by the video analyzing unit. Further provided is a display switching unit for instructing to switch display contents. Therefore, the display contents classified into a plurality of categories can be switched according to the light emission state of the pointing device, and the desired content display can be switched and displayed in an easy-to-understand manner by operating the pointing device.

In addition, the object indication specifying system of the present invention is configured such that the light emission state of the light emitting unit, which can be switched by the light emission switching unit of the light emitting unit, is a light emission color, brightness, number of blinks, or blinking pattern. Therefore, it is possible to instruct the user to change the lighting color, brightness, number of blinks, or blinking pattern by the user's operation, and the user confirms the user's operation by, for example, meaning the lighting state for the operation of the pointing device. The light emission state can be adjusted in accordance with the light / dark state of the environment in which the system is operated.

  According to the object instruction specifying system of the present invention, even if it is a small operation target device, it is possible to easily specify the operation target device by positioning the pointing device between the information terminal and the operation target device to be specified. . Further, it becomes possible to acquire and display information related to the operation target device specified by the information terminal, and to easily check the description and state of the specified operation target device.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
In the object instruction specifying system according to the present embodiment, a user specifies an operation target device using an instruction device, and further displays information related to the operation target device. FIG. 1 is a diagram illustrating a schematic configuration of an object instruction specifying system according to the first embodiment of the present invention.

  In FIG. 1, the object instruction specifying system includes an information terminal 100, an instruction device 200, and an operation target device 300. The object instruction specifying system analyzes an image of a viewpoint similar to that of a user captured by the image capturing unit 101 of the information terminal 100, and instructs When it is determined that the light emitting unit 201 of the apparatus 200 and the identification unit 301 of the operation target device 300 overlap, the operation target device 300 is specified as the operation target device that the user wants to specify. After identifying the operation target device, the information terminal 100 collects information related to the operation target device 300 from the identified operation target device 300 and displays the information on the display unit 102. Information regarding the operation target device 300 will be described later.

  At this time, the instruction device 200 is used when a user designates an operation target device, and the light emitting unit 201 emits light by a user operation. The light emitted from the pointing device 200 is preferably visible light that can be confirmed by the user. In addition, the operation target device 300 includes an identification unit 301 used by the information terminal 100 to identify the operation target device. In the present embodiment, it is assumed that the identification unit 301 can emit light. The light emitted from the identification unit 301 is preferably visible light that can be confirmed by the user. The identification unit 301 does not necessarily need to be able to emit light, and may be in any form that can be viewed by the user. In response to a request from the information terminal 100, the operation target device 300 transmits display information about itself as a response.

  FIG. 2 is a diagram for explaining the operation of the object instruction specifying system according to the first embodiment of the present invention. FIG. 2 shows a state in which the light emitting unit 201 of the instruction device 200 is superimposed on the identification unit 301 of the operation target device 300. The user designates the operation target device 300 by causing the light emitting unit 201 to emit light while holding the light emitting unit 201 of the pointing device 200 so as to overlap the identification unit 301 of the operation target device 300.

FIG. 3 is a diagram for explaining an example of use of the object instruction specifying system according to the first embodiment of the present invention. In FIG. 2, a state in which the user 990 wears the pointing device 200 and the information terminal 100 is illustrated. The display unit 102 of the information terminal 100 preferably has a shape that can be worn by the user 990 without holding it. In this embodiment, an example using a head-mounted display that is worn on the ear is shown. The shape of the display unit 102 is not limited to these, and can be changed to any shape that is easy for the user to use. In addition, the video shooting unit 101 of the information terminal 100 is preferably present in the vicinity of the user's eyes in order to be able to take a video of the same viewpoint as that of the user 990. In this embodiment, a head-mounted display type is used. The shape is attached to the display unit 102 of the information terminal 100. Video shooting part 1
The shape of 01 is not limited to these, and can be changed to any position as long as an image of the same viewpoint as the user 990 can be taken.

  The pointing device 200 can be held or worn by the user 990 in his / her hand, and in this embodiment, a pen-shaped shape is used. However, the indication device 200 is not limited to a shape, and may be a ring type that can be attached to a finger, for example. Moreover, when not using the instruction | indication apparatus 200, it is desirable to be able to attach to a part of information terminal 100. FIG.

  Next, the internal configuration of the information terminal, the pointing device, and the operation target device constituting the target object specifying system according to the present embodiment will be described. 4A to 4C are diagrams illustrating configurations of the information terminal, the instruction device, and the operation target device according to the first embodiment of the present invention.

  4A, the information terminal 100 includes a first communication unit 103, a terminal identifier generation unit 104, a video shooting unit 101, a video analysis unit 106, an operation target information acquisition unit 105, and a display unit 102.

  The first communication unit 103 performs data communication with the operation target device 300. The terminal identifier generation unit 104 generates a terminal identifier that identifies the information terminal 100. Note that the terminal identifier may be unique within the information terminal that exists in a range where communication with the information terminal 100 is possible. In the present embodiment, “1” is set as the value of the terminal identifier. As a method of generating a terminal identifier, for example, a method of acquiring a terminal identifier of a neighboring information terminal via the first communication unit 103 and assigning an unused value can be considered. Note that the terminal identifier generation method is not limited to the above.

  The video imaging unit 101 captures video from the same viewpoint as the user. The video analysis unit 106 analyzes the video captured by the video imaging unit 101, acquires the operation target identifier of the operation target device 300 notified by light from the identification unit 301 of the operation target device 300, and Superimposition between the identification unit 301 and the light emitting unit 201 of the pointing device 200 is detected, and the operation target device is specified.

  The operation target information acquisition unit 105 communicates with the operation target device 300 via the first communication unit 103 based on the operation target identifier of the operation target device specified by the video analysis unit 106, and information regarding the operation target device 300. To get.

  The display unit 102 displays information related to the operation target device 300 based on the video captured by the video imaging unit 101 and the operation target information acquired from the operation target device 300 by the operation target information acquisition unit 105.

  In FIG. 4B, the instruction device 200 includes an input unit 202 and a light emitting unit 201. The input unit 202 detects a user input operation. In the present embodiment, the input unit 202 can switch between an on state and an off state by a user input operation. In addition, the shape of the input unit 202 is preferably a switch type, a button type, or the like. The light emitting unit 201 emits light based on the output of the input unit 202.

  4C, the operation target device 300 includes an identification unit 301, a second communication unit 302, an operation target information storage unit 303, a response unit 304, and an operation target identifier generation unit 305.

The operation target identifier generation unit 305 generates an operation target identifier that identifies the operation target device 300. The operation target identifier may be unique in the operation target device that exists in a range where communication with the information terminal 100 is possible. In the present embodiment, “1” is set as the value of the operation target device identifier. As a generation method of the operation target device identifier, for example, a method of acquiring the value of the operation target device identifier of the operation target in the vicinity via the second communication unit 302 and assigning an unused value can be considered. . Note that the method for generating the operation target device identifier is not limited to the above.

  The identification unit 301 emits light and notifies the information terminal 100 of the operation target identifier generated by the operation target identifier generation unit 305. As a notification method, for example, a method of notifying the operation target identifier when the identification unit 301 blinks can be considered. As a general method, when the identification unit 301 is a visible light LED and the image capturing unit 101 is a CCD camera with a frame rate of 30 Hz, the identification unit 301 is an operation target for light having a preset RGB value. An optical signal modulated by Manchester encoding the identifier value at 15 Hz is output. The video shooting unit 101 acquires a video at a frame rate of 30 Hz, extracts pixel values corresponding to the RGB values of the identification unit 301 from the video frame, and a luminance change pattern of light spots at the same position on the frame By decoding, the operation target identifier can be obtained. Note that the method for notifying information is not limited to the above-described method, and any method may be used as long as the information is captured by the video imaging unit 101 of the information terminal 100 and can be analyzed by the video analysis unit 106. . Further, the identification unit 301 may be configured to emit light only when the operation target device 300 is usable, and may be configured to easily identify the target that can be used by the user. Or it is good also as a form which can determine the state which can be utilized in the image | video analysis part 106 of the information terminal 100. FIG.

  The operation target information storage unit 303 stores operation target information 400 indicating the attributes of the operation target device 300. The operation target information 400 will be described later.

  The second communication unit 302 communicates with the information terminal 100.

  The response unit 304 generates operation target information to be transmitted to the information terminal 100 based on the detailed information regarding the operation target device 300 accumulated in the operation target information storage unit 303, and transmits information regarding the operation target device 300. To do.

  Note that the first communication unit 103 of the information terminal 100 desirably communicates with a nearby operation target device using, for example, short-range radio. By adopting such a configuration, communication is performed only with the operation target device that is within the reach of the wireless radio wave, so that it is possible to suppress information from the operation target device in a remote place that does not enter the user's field of view. Can do.

  Here, the operation target information stored in the operation target information storage unit 303 of the operation target device 300 will be described. FIG. 5 is a diagram illustrating a data structure of the operation target information stored in the operation target information storage unit of the operation target device according to the first embodiment of the present invention. In FIG. 5, the operation target information 400 is composed of commentary information 410 indicating the contents describing the operation target device, and contents held by the operation target device or menu information 420 indicating the operation function of the operation target device. Yes. The explanation information 410 includes a display content operation target name 401, an operation target identifier 402, and an operation target attribute 403. The display content information 420 includes a display format 404, a list including a display identifier 405, an attribute 406, and a display content 407. In the present embodiment, an example of data of the operation target information 400 when the operation target device 300 is a video device is shown. In the present embodiment, it is assumed that the operation target information is stored in advance in the operation target information storage unit before the information terminal collects data.

The operation target name 401 indicates a value used by the user to determine the type of the operation target device 300. In the present embodiment, the value “Living room video” is input. As another example, it may be possible to input a value “an older brother's video” or the like that specifies a person who owns the operation target device 300. The operation target name 401 preferably has a form in which a user can easily input a name that can easily identify the type of the operation target device 300.

  The operation target identifier 402 indicates a value for the information terminal 100 to specify the operation target device 300. In the present embodiment, the value “1” is input. As another example, a mode in which a network address assigned to the operation target device 300 is input is also conceivable. The operation target identifier 402 is desirably set automatically by determining a value that does not overlap with the operation target identifiers of neighboring operation target devices between the operation target devices.

  The operation target attribute 403 indicates the type of the operation target device 300. In this embodiment, “video” is input. Since the operation target attribute 403 is uniquely determined by the type of the operation target device 300, it is desirable that the operation target attribute 403 is input as a fixed value in advance.

  A display format 404 indicates arrangement information of the display content 407 that is referred to when the display unit 102 of the information terminal 100 displays information related to the operation target device 300 in a list format. In the present embodiment, a value “n × 1 list” indicating that the list is arranged and displayed in the form of n rows and 1 column is input. As another example, a value “n × n list”, “1 × n list”, or the like indicating that the list is arranged and displayed in an n-row n-column format or a 1-row n-column format may be used. Conceivable. The display format 404 preferably has a form in which the user can freely input a value suitable for the display content 407 when the display content 407 is input.

  The display identifier 405 indicates a value that specifies the display content 407. The display identifier 405 is a value assigned to each item of the display content 407, and in this embodiment, numbers are assigned in ascending order. The method of assigning the display identifier 405 is not limited to these methods, and any value that can specify the display content 407 may be used. It is desirable that the display identifier 407 is automatically input by the operation target device 300.

  An attribute 406 indicates an attribute of the display content 405. In the present embodiment, a value “text” indicating that the display content 407 is a character string is input. As another example, a value “moving image” or “still image” indicating a moving image or a still image can be considered. Further, if the display content is a character string, a character code such as “ASCII”, if it is a moving image, a compression format such as “MPEG2” or “MPEG4”, if it is a still image, a data compression format such as “JPEG”. You may input as a value which shows. Since the attribute 406 is uniquely determined by the type of the display content 407, it is desirable that the attribute 406 is input as a fixed value in advance.

  The display content 407 indicates the content displayed on the display unit 102 of the information terminal 100. In the present embodiment, character strings “Taiga Drama”, “Horror Movie”, and “Travel Photos” indicating the content names accumulated in the video are input. As another example, menu display character strings “play” and “record” for operating the operation target device 300, image data related to the operation target device 300, and the like can be considered. It is desirable that the display contents are freely input by the user. Note that the information included in the operation target information 400 is not limited to the above.

  Next, the operation of the object instruction specifying system according to the present embodiment will be described. FIG. 6 is a sequence diagram showing an operation of the object instruction specifying system according to the first exemplary embodiment of the present invention. In FIG. 6, the operation target device 300 is identified by light emission or data communication among the information terminal 100, the instruction device 200, and the operation target device 300.

  In sequence S101, the video photographing unit 101 of the information terminal 100 starts photographing near the operation target device.

In sequence S102, the operation target identifier generation unit 305 of the operation target device 300
An operation target identifier of the operation target device is generated.

  In sequence S <b> 103, the identification unit 301 of the operation target device 300 emits light that notifies the operation target identifier 700. At this time, light reaching the operation target device 300 is captured by the video capturing unit 101 of the information terminal 100. Here, although the identification part 301 of the operation target apparatus 300 shall emit light regularly, it is not limited to this.

  In sequence S <b> 104, the video analysis unit 106 analyzes the information included in the light from the captured video and acquires the operation target identifier 700 of the operation target device 300.

  In sequence S <b> 105, the input unit 202 of the instruction device 200 is set to the user in a position state where the identification unit 301 of the operation target device 300 and the light emitting unit 201 of the instruction device 200 are overlapped when viewed from the video photographing unit 101 of the information terminal 100. It is operated by operation.

  In sequence S106, the light emitting unit 201 of the pointing device 200 emits light. At this time, the video photographing unit 101 of the information terminal 100 captures the light emitted from the pointing device 200.

  In sequence S107, the video analysis unit 106 of the information terminal 100 analyzes the captured video, and determines that the identification unit 301 of the operation target device 300 and the light emitting unit 201 of the pointing device 200 are superimposed. Then, the operation target device 300 having the operation target identifier 700 is specified.

  In sequence S <b> 108, the operation target information acquisition unit 105 of the information terminal 100 generates an operation target information acquisition request 500 for the operation target device 300 using the specified operation target identifier 700 of the operation target device 300.

  In sequence S <b> 109, the operation target information acquisition unit 105 of the information terminal 100 transmits the generated operation target information acquisition request 500 to the operation target device 300 via the first communication unit 103. At this time, the request may be transmitted by specifying individual operation target devices in the vicinity, but it is desirable in terms of efficiency to use multicast or broadcast transmission that simultaneously transmits data to a plurality of operation target devices. The operation target information acquisition request will be described later.

  In sequence S <b> 110, the response unit 304 of the operation target device 300 confirms whether the request is for itself from the operation target identifier 700 included in the operation target information acquisition request 500 acquired via the second communication unit 302. If it is addressed to itself, an operation target information acquisition response 600 is generated. The operation target information acquisition response 600 will be described later.

  In sequence S <b> 111, the response unit 304 of the operation target device 300 transmits an operation target information acquisition response 600 including the generated operation target information to the information terminal 100 via the second communication unit 302.

  In sequence S <b> 112, the operation target information acquisition unit 105 of the information terminal 100 displays the operation target information included in the operation target information acquisition response 600 acquired via the first communication unit 103 on the display unit 102. The contents to be displayed will be described later.

Here, the operation target information acquisition request 500 transmitted from the information terminal 100 to the operation target device 300 in the sequence S109 will be described. FIG. 7 is a diagram illustrating a data structure of an operation target information acquisition request generated by the information terminal according to the first embodiment of the present invention. In FIG. 7, the operation target information acquisition request 500 includes an operation target identifier 501 that specifies the operation target device 300 and an information terminal identifier 502 that specifies the information terminal 100. The operation target identifier 501 is set with the operation target identifier of the operation target device 300 acquired by the video analysis unit 106 from the identification unit of the operation target device 300. In the present embodiment, the value “1” is set. In the information terminal identifier 502, the terminal identifier generated by the terminal identifier generation unit 104 is set. In the present embodiment, the value “2” is set. Note that the information included in the operation target information acquisition request is not limited to the above.

  The operation target information acquisition response 600 generated by the operation target device 300 in sequence S111 will be described. FIG. 8 is a diagram illustrating a data structure of an operation target information acquisition response generated by the operation target device according to the first embodiment of the present invention. In FIG. 8, an operation target information acquisition response 600 includes an information terminal identifier 601, an operation target name 602, an operation target identifier 603, an operation target attribute 604, a display format 605, a display content number 606, a display identifier 607, an attribute 608, and a display content. 609.

  A value is set in the information terminal identifier 601 based on the information terminal identifier 501 set in the operation target information acquisition request 500 received by the operation target device. In the present embodiment, the value “2” is set.

  In the operation target name 602, the operation target identifier 603, and the operation target attribute 604, values based on the operation target information 400 are set. In the present embodiment, values “video in the living room”, “1”, and “video” are set from the example of the operation target information shown in FIG.

  A value is set in the display format 605 based on the operation target information 400. In the present embodiment, the value “n × 1 list” is set in the display format 605.

  Values based on the operation target information 400 are set in the display identifier 607, the attribute 608, and the display content 609. In the present embodiment, “Taiga Drama”, “Horror Movie”, and “Travel Photos” are set in the display content 609. The display identifier 607 is set to “1” for the taiga drama, “2” for the horror movie, and “3” for the travel photo. In the attribute 608, “text” is set for each of the taiga drama, the horror movie, and the travel photo.

  In the display content number 606, a value based on the number of display contents 407 of the operation target information 400 is set. In the present embodiment, since there are three display contents of “Taiga Drama”, “Horror Movie”, and “Travel Photos”, “3” is set. Note that the information included in the operation target information acquisition response is not limited to the above.

  Next, operations of the information terminal, the instruction device, and the operation target device according to the present embodiment will be described in order. First, the operation of the information terminal 100 will be described. FIG. 9 is a flowchart showing the operation of the information terminal 100 according to the first embodiment of the present invention.

  In step S <b> 201, the video shooting unit 101 starts video shooting and outputs the shot video to the video analysis unit 106.

  In step S202, the video analysis unit 106 analyzes the video shot by the video shooting unit 101, and determines whether the identification unit 301 of the operation target device 300 is shot. When the video of the identification unit 301 is not included, the video analysis is continued again. If the video of the identification unit 301 is included, the process proceeds to step S203.

In step S <b> 203, the video analysis unit 106 analyzes the video of the identification unit 301 captured by the video shooting unit 101 and acquires the operation target identifier of the operation target device 300. In the present embodiment, the operation target identifier of the operation target device 300 is analyzed, and the coordinate information on the image of the identification unit 301 of the operation target device 300 in the video captured by the video imaging unit 101 is also analyzed. In a later-described step, information is displayed in a balloon shape using the coordinates as a base point. With the above, it is possible to easily present the user with the relevance between the information displayed on the display unit 102 and the operation target device 300.

  In step S <b> 204, the video analysis unit 106 determines whether a video of the light emitting unit 201 of the pointing device 200 is captured. When the video of the light emitting unit 201 is not included, the video analysis is continued again. If an image of the light emitting unit 201 is included, the process proceeds to step S205.

  In step S <b> 205, it is determined whether the video of the light emitting unit 201 is superimposed on the video of the identification unit 301 of the operation target device 300. If the light emitting unit 201 and the identifying unit 301 are superimposed, the process proceeds to step S206; otherwise, the process returns to step S204.

  In step S206, the terminal identifier generation unit 104 generates a terminal identifier of the information terminal 100.

  In step S <b> 207, the operation target information acquisition unit 105 generates the operation target information acquisition request 500 from the terminal identifier of the information terminal 100 generated by the terminal identifier generation unit 104 and the operation target identifier analyzed by the video analysis unit 106.

  In step S <b> 208, the operation target information acquisition unit 105 transmits an operation target information acquisition request 500 to the operation target device 300 via the first communication unit 103.

  In step S <b> 209, the operation target information acquisition unit 105 receives the operation target information acquisition response 600 from the operation target device 300 via the first communication unit 103.

  In step S <b> 210, the operation target information acquisition unit 105 analyzes the operation target information acquisition response 600 and determines whether the terminal identifier included in the operation target information acquisition response 600 matches its own terminal identifier. If they do not match, the process returns to S209, and if they match, the process proceeds to S211.

  In step S211, the operation target information acquisition unit 105 acquires information related to the operation target device 300 from the operation target information acquisition response 600, that is, the operation target information 400, and stores the acquired information.

  In step S <b> 212, the operation target information acquisition unit 105 outputs display information related to the operation target device 300 to the display unit 102 according to the operation target information 400 and displays the display information. A display example will be described later.

  Next, the operation of the operation target device 300 will be described. FIG. 10 is a flowchart showing the operation of the operation target device according to the first exemplary embodiment of the present invention.

  In step S301, the operation target identifier generation unit 305 generates an operation target identifier that identifies itself.

  In step S302, the identification unit 301 emits light to notify the operation target identifier together with its presence. Here, it is assumed that the identification unit 301 emits light periodically.

  In step S <b> 303, the response unit 304 determines whether the operation target information acquisition request 500 is received from the information terminal via the second communication unit 302. If the operation target information acquisition request 500 has not been received, the process returns to step S302.

  In step S304, after receiving the operation target information acquisition request 500, the response unit 304 analyzes the received operation target information acquisition request 500, and the operation target identifier included therein matches the own operation target identifier. It is determined whether or not. If they do not match, the process returns to step S302.

  In step S305, the response unit 304 acquires the information terminal identifier set in the operation target information acquisition request 500.

  In step S306, the response unit 304 generates an operation target information acquisition response 600 from the operation target information 400 stored in the operation target information storage unit 303 together with the information terminal identifier acquired in step S305.

  In step S <b> 307, the response unit 304 transmits an operation target information acquisition response 600 to the information terminal 100 via the second communication unit 302.

  Next, the operation of the pointing device 200 will be described. FIG. 11 is a flowchart showing the operation of the pointing device according to the first embodiment of the present invention.

  In step S <b> 401, the input unit 202 detects a user input operation and determines whether it is in an on state or an off state. If it is off, step S401 is repeated.

  In step S <b> 402, the light emitting unit 201 emits light based on the notification from the input unit 202.

  Next, a display screen example of the information terminal 100 will be described as an operation example of the object instruction specifying system according to the present embodiment. FIG. 12 is a diagram illustrating a display screen example of the information terminal 100 according to the first embodiment. In FIG. 12, the display unit 102 of the information terminal 100 stores an operation target device image 701, an operation target name 703 included in the operation target information 400 acquired from the operation target device 300, and the operation target device 300. A list 707 of content names 704, 705, and 706 is displayed. Here, the operation target name 703 is displayed in a balloon shape with the video 702 of the identification unit 301 of the operation target device as a base point. The content name list 707 is displayed in a 3 × 1 list based on the display format “n × 1 list” and the number of display contents “3” of the acquired operation target information 400. The display form is not limited to the above, and any form may be used as long as it is based on the content of the display information.

Further, the information terminal 100 uses the information terminal 100 and the instruction device 200 by configuring the information terminal 100 to detect the superimposition of the light emitting unit of the instruction device 200 and the list item and select the operation content list item. Then, a remote operation instruction may be given to the operation target device 300. For example, in the example of FIG. 12, it is possible to perform an operation such as selecting desired content from the content list displayed on the screen and controlling playback of the operation target device. FIG. 13 is a diagram illustrating another display screen example of the information terminal according to the first embodiment. In FIG. 13, the display unit 102 of the information terminal 100 includes n rows of an automatic door image 711 that is an operation target device, an automatic lock image 712, and a key number image 713 of a personal identification number that is an operation content menu of the automatic lock. Displayed in n columns. In the operation target information at this time, the operation target device 300 is “automatic door with automatic lock”, the display format is “n × n list”, and the number of display contents is “9”. The user can unlock the automatic door by selecting his / her room number and password using the pointing device.

(Embodiment 2)
Next, the object instruction specifying system according to the second exemplary embodiment of the present invention will be described. The object instruction specifying system according to the present embodiment switches the contents displayed on the display unit of the information terminal according to the emission color of the light emitting unit of the indicating device. The user can appropriately change and display information on the operation target device by changing the emission color of the pointing device.

  First, the configuration and operation of an information terminal and an instruction device constituting the object instruction specifying system according to the present embodiment will be described. Note that the configuration and operation of the operation target device according to the present embodiment are the same as those in the first embodiment, and thus the description thereof is omitted. FIGS. 14A to 14B are diagrams illustrating configurations of an information terminal and an instruction device according to the second embodiment of the present invention. In FIG. 14A, the information terminal 150 further includes a display switching unit 157 as compared with the first embodiment. In addition to the operation of the video analysis unit 106 described in Embodiment 1, the video analysis unit 156 of the information terminal 150 determines the emission color of the light emission unit 251 of the pointing device 200. The display switching unit 157 selects information to be displayed on the display unit 102 from the operation target information based on the emission color analyzed by the video analysis unit 156, and changes the display of the display unit 102.

  In FIG. 14B, the instruction device 250 further includes a light emission switching 253 as compared with the first embodiment. Further, the input unit 202 of the pointing device 250 can be switched by designating a plurality of emission colors in an on state, an off state, and an on state by a user input operation. Button type etc. can be considered. For example, a shape in which a plurality of emission colors and an off state are assigned to a dial scale is conceivable. The light emission switching unit 253 switches the light emission color of the light emission unit 251 based on an input from the user. The light emitting unit 251 emits a plurality of colors in response to an instruction from the light emission switching unit 253.

  Next, the operation target information stored in the operation target information storage unit 303 by the operation target device 300 will be described. FIG. 15 is a diagram illustrating a data structure of operation target information stored in the operation target device 300 according to the second embodiment of the present invention. In FIG. 15, the operation target information 450 includes an operation target name 401, an operation target identifier 402, an operation target attribute 403, a display format 404, a display identifier 405, an attribute 406, and display content 407 other than the list identifier 458. Since it is the same as that of form 1, description is abbreviate | omitted. The list identifier 458 is a value for identifying a plurality of display lists 421 and 422 in the display content information 425. In this embodiment, the value “1” is set in the list identifier 458 of the list 421 related to the video content name, and the value “2” is set in the list identifier 458 of the list 422 related to the music content name. The display unit 102 of the information terminal 150 selects a display content list corresponding to the emission color of the instruction device 250 and displays one item. The list identifier 458 only needs to be assigned a value that does not overlap in the display information. For example, a method of assigning values in ascending order from the top of the list as in this embodiment, or a time stamp when content is generated in the operation target device. And so on.

  Note that the data structure of the operation target information acquisition request generated by the information terminal 150 is the same as the data structure of the operation target information acquisition request 500 shown in the first embodiment, and a description thereof will be omitted.

Next, an operation target information acquisition response generated by the operation target device 300 will be described. FIG. 16 is a diagram illustrating a data structure of an operation target information acquisition response generated by the operation target device according to the second embodiment of the present invention. In FIG. 16, the operation target information acquisition response 650 includes an information terminal identifier 601, an operation target name 602, an operation target identifier 603, an operation target attribute 604, a display format 605, and a display content number 606 excluding the list number 651 and the list identifier 652. Since the display identifier 607, the attribute 608, and the display content 609 are the same as those in the first embodiment, description thereof is omitted. The list number 651 is included in the operation target information acquisition response 650 and represents the number of lists indicated by a list identifier 652 described below. In the present embodiment, the value “2” is set based on the number of lists of operation target information. The list identifier 652 is a value for identifying a plurality of display lists. In the second embodiment, based on the list identifier 652 of the operation target information, the value “1” is set in the display list related to the video content name, and the value “2” is set in the display list related to the music content name.

  Next, emission color information generated by the display switching unit 157 of the information terminal 150 will be described. The generated emission color information is referred to when the display switching unit 157 of the information terminal 150 changes the display of the display unit 102 based on the emission color of the light emitting unit 251 of the pointing device 250. FIG. 17 is a diagram illustrating a data structure of the emission color information stored in the display switching unit 157 according to the second embodiment of the present invention. In FIG. 17, the emission color information 800 indicates the correspondence between the emission color 801 and the list identifier 802. A light emission color 801 indicates a color that the light emitting unit 251 of the pointing device 250 can emit. In the present embodiment, the values “blue”, “red”, and “green” are set. Since the emission color 801 is uniquely determined by the pointing device 250, it is desirable that the emission color 801 is input as a fixed value in advance. The list identifier 802 is set with the value of the list identifier included in the operation target information acquisition response received from the operation target device 300 by the information terminal 150. As a setting method, for example, a method of sequentially assigning from the list identifier 652 at the head of the list included in the operation target information acquisition response 650 can be considered. In the present embodiment, the list identifier “1” is assigned to the emission color “blue”, and the list identifier “2” is assigned to the emission color “red”. When the number of colors that can be developed exceeds the number of items in the list as in this embodiment, it is desirable not to assign an identifier for the remaining light emission colors.

  In addition, since the whole operation | movement of the target object specification specific system concerning this Embodiment is the same as that of Embodiment 1, description is abbreviate | omitted.

  Next, operations of the information terminal 150 and the instruction device 250 that constitute the object instruction specifying system according to the present embodiment will be described. Note that the operation target device 300 has the same operation as that of the first embodiment, and thus description thereof is omitted. First, the operation of the information terminal 150 will be described. FIG. 18 is a flowchart showing the operation of the information terminal 150 according to the second embodiment of the present invention. Note that steps denoted by the same reference numerals as those in FIG. 9 are the same as the operations of the information terminal 100 described in Embodiment 1, and thus description thereof is omitted.

  In step S <b> 501, the video analysis unit 156 analyzes the emission color of the light emitting unit 251 of the pointing device 250 and notifies the operation target information acquisition unit 105 of the information on the emission color.

  In step S <b> 502, the display switching unit 157 generates the emission color information 800 based on the number of display information lists included in the operation target information acquisition response 650 acquired by the operation target information acquisition unit 105 from the operation target device 300.

  In step S503, the display switching unit 157 compares the emission color of the instruction device 250 analyzed by the video analysis unit 156 with the emission color information 800, and determines the list contents to be displayed on the display unit 102 from the acquired operation target information 450. To do.

  In step S504, the display unit 102 displays the display content report determined by the display switching unit 157.

  Next, the operation of the instruction device 250 will be described. FIG. 19 is a flowchart showing the operation of the pointing device according to the second embodiment of the present invention. Note that steps denoted by the same reference numerals as those in FIG. 11 are the same operations as those of the pointing device 200 described in the first embodiment, and thus description thereof is omitted.

  In step S <b> 601, when the switch is turned on, the light emission switching unit 253 notifies the light emission unit 251 of the light emission color based on the input from the input unit 202.

  Next, a screen example displayed on the display unit 102 of the information terminal 150 according to the present embodiment will be described. 20 and 21 are diagrams illustrating examples of display screens of the information terminal 150 according to the second embodiment of the present invention. When the light emitting unit 251 of the instruction device 250 emits light in “blue” color by a user operation or the like, as shown in FIG. 20, a list of video content names that is the first item in the operation target information display list 723 is displayed. On the other hand, when the light emitting unit 251 of the pointing device 250 emits light in “red” color, a list of music content names, which is the second item in the operation target information display list 725, is displayed as shown in FIG. Is done. FIG. 20 shows that the menu “Horror movie” 724 in the list 723 is selected. Similarly, FIG. 21 shows a state where the menu “pops” 726 is selected.

  As described above, the object instruction specifying system according to the present embodiment changes the content displayed on the display unit of the information terminal according to the emission color of the light emitting unit of the indicating device, and further changes the emission color of the indicating device. As a result, it is possible to switch and display information relating to the identified operation target device. In addition, the operation target device can be operated by selecting a menu from the displayed list.

(Embodiment 3)
Next, the object instruction specifying system according to the third exemplary embodiment of the present invention will be described. The object instruction specifying system according to the present embodiment changes the content displayed on the display unit according to the number of blinks of the light emitting unit of the pointing device. The user can appropriately change the displayed information regarding the operation target device by changing the number of blinks of the light emitting unit of the pointing device.

  First, the configuration and operation of an information terminal and an instruction device constituting the object instruction specifying system according to the present embodiment will be described. Note that the configuration and operation of the operation target device according to the present embodiment are the same as those of the operation target device 300 shown in the first embodiment, and thus the description thereof is omitted. FIGS. 22A to 22B are diagrams illustrating configurations of an information terminal and an instruction device according to the third embodiment of the present invention. 22A, the information terminal 160 has substantially the same configuration as the information terminal 150 shown in Embodiment 2, but the operations of the display switching unit 167 and the video analysis unit 166 of the information terminal 160 are different. The video analysis unit 166 determines the number of blinks of the light emitting unit 261 of the pointing device 260 instead of the operation of determining the emission color of the video analysis unit 156 shown in the second embodiment. The display switching unit 167 changes information displayed on the display unit 102 based on the number of blinks analyzed by the video analysis unit 166.

  In FIG. 22B, the instruction device 260 has substantially the same configuration as the instruction device 250 shown in the second embodiment, but the operations of the light emission switching unit 263 and the light emission unit 261 of the instruction device 260 are different. The input unit 202 of the pointing device 260 can be switched between an on state, an off state, and a plurality of blinking times in the on state according to an input from a user. The shape may be a dial type, a switch type, a button type, or the like. Conceivable. For example, a shape in which a plurality of blinking times and an off state are assigned to a dial scale can be considered. The light emission switching unit 263 switches the number of blinks of the light emitting unit 261 by an input operation from the user. The light emitting unit 261 blinks in response to an instruction from the light emission switching unit 263.

  The operation target information, the operation target information acquisition request, and the operation target information acquisition response according to the third embodiment are the same as those in the first embodiment, and thus the description thereof is omitted.

Next, blinking information generated by the display switching unit 167 of the information terminal 160 will be described. The blink information 810 is referred to when the display switching unit 167 changes the display of the display unit 102 based on the number of blinks of the light emitting unit 261 of the pointing device 260. FIG. 23 is a diagram illustrating a data structure of blinking information stored in the display switching unit 167 according to the third embodiment of the present invention. In FIG. 23, blinking information 810 indicates the correspondence between the number of blinks 811 and the list identifier 812. The blinking number 811 is a value indicating the number of blinking times of the light emitting unit 261 of the pointing device 260. In the present embodiment, values “1”, “2”, and “3” are set. Since the blinking number 811 is uniquely determined by the pointing device 260, it is desirable that the blinking number 811 is input as a fixed value in advance. As the list identifier 812, the value of the list identifier included in the operation target information acquisition response received from the operation target device 300 by the information terminal 160 is set. As a setting method, for example, a method of sequentially assigning from the list identifier at the head of the list included in the operation target information acquisition response can be considered. In the present embodiment, the list identifier “1” is assigned to the blink count “1”, and the list identifier “2” is assigned to the blink count “2”. When the blinkable number exceeds the number of items in the list as in the present embodiment, it is desirable not to assign an identifier for the remaining blink number.

  In addition, since the whole operation | movement of the target object specification specific system concerning this Embodiment is the same as that of Embodiment 1, description is abbreviate | omitted.

  Next, operations of the information terminal 160 and the instruction device 260 that constitute the object instruction specifying system according to the present embodiment will be described. The operation of the operation target device 300 is the same as that in the first embodiment, and thus the description thereof is omitted. FIG. 24 is a flowchart showing an operation of the information terminal 160 according to the third embodiment of the present invention. Note that the step operation indicated by the same number as in FIG. 9 is the same as the operation of the information terminal 100 described in Embodiment 1, and thus the description thereof is omitted.

  In step S <b> 701, the video analysis unit 166 analyzes the number of blinks of the light emitting unit 261 of the pointing device 260 and notifies the operation target information acquisition unit 105 of the information on the number of blinks.

  In step S <b> 702, when the operation target information acquisition unit 105 completes the analysis of the operation target information acquisition response 650 received from the operation target device 300, the blink information 810 is obtained based on the display information included in the operation target information acquisition response 650. Generate.

  In step S703, the display switching unit 167 compares the number of blinks of the light emitting unit 261 of the pointing device 260 analyzed by the video analysis unit 166 with the blink information 810, and displays the list displayed on the display unit 102 from the acquired operation target information 450. Determine the contents.

  In step S704, the display unit 102 displays the list information determined by the display switching unit 167.

  Next, the operation of the instruction device 260 will be described. FIG. 25 is a flowchart showing the operation of the pointing device according to the third embodiment of the present invention. Note that in FIG. 25, steps indicated by the same numbers as in FIG. 11 are the same as the operations of the pointing device 200 described in Embodiment 1, and thus description thereof is omitted.

  In step S801, the light emission switching unit 263 notifies the light emitting unit 261 of the number of blinks based on the input from the user when the switch is turned on.

As described above, the object instruction specifying system according to the present embodiment changes the content displayed on the display unit of the information terminal according to the number of blinks of the light emitting unit of the pointing device, and operates according to the emission color shown in the second embodiment. Similarly to the above, information regarding the specified operation target device can be switched and displayed by the blinking operation of the pointing device.

  The object instruction specifying system according to the present invention is suitable for an instruction operation system for instructing a small operation target device at a short distance from a user.

The figure which shows schematic structure of the target object specification specific system concerning Embodiment 1 of this invention. The figure explaining operation | movement of the target object specification specific system concerning Embodiment 1 of this invention. The figure explaining the usage example of the target object specification specific system concerning Embodiment 1 of this invention. (A) The figure which shows the structure of the information terminal concerning Embodiment 1 of this invention (b) The figure which shows the structure of the instruction | indication apparatus concerning Embodiment 1 of this invention (c) It concerns on Embodiment 1 of this invention Diagram showing the configuration of the operation target device The figure which shows the data structure of the operation target information which the operation target apparatus concerning Embodiment 1 of this invention stores. FIG. 3 is a sequence diagram showing the operation of the object instruction specifying system according to the first embodiment of the present invention. The figure which shows the data structure of the operation target information acquisition request which the information terminal concerning Embodiment 1 of this invention produces | generates. The figure which shows the data structure of the operation target information acquisition response which the operation target apparatus concerning Embodiment 1 of this invention produces | generates. The flowchart which shows operation | movement of the information terminal concerning Embodiment 1 of this invention. The flowchart which shows operation | movement of the operation target apparatus concerning Embodiment 1 of this invention. The flowchart which shows operation | movement of the instruction | indication apparatus concerning Embodiment 1 of this invention. The figure which shows the example of a display screen of the information terminal concerning Embodiment 1 of this invention. The figure which shows the example of a display screen of the information terminal concerning Embodiment 1 of this invention. (A) The figure which shows the structure of the information terminal concerning Embodiment 2 of this invention (b) The figure which shows the structure of the instruction | indication apparatus concerning Embodiment 2 of this invention The figure which shows the data structure of the operation target information which the operation target apparatus concerning Embodiment 2 of this invention produces | generates. The figure which shows the data structure of the operation target information acquisition response which the operation target apparatus concerning Embodiment 2 of this invention produces | generates. The figure which shows the data structure of the luminescent color information which the display switching part concerning Embodiment 2 of this invention produces | generates. The flowchart which shows operation | movement of the information terminal concerning Embodiment 2 of this invention. The flowchart which shows operation | movement of the instruction | indication apparatus concerning Embodiment 2 of this invention. The figure which shows the example of a display screen of the information terminal concerning Embodiment 2 of this invention. The figure which shows the example of a display screen of the information terminal concerning Embodiment 2 of this invention. (A) The figure which shows the structure of the information terminal concerning Embodiment 3 of this invention (b) The figure which shows the structure of the instruction | indication apparatus concerning Embodiment 3 of this invention. The figure which shows the data structure of the blink information which the display switching part concerning Embodiment 3 of this invention stores. The flowchart which shows operation | movement of the information terminal concerning Embodiment 3 of this invention. The flowchart which shows operation | movement of the instruction | indication apparatus concerning Embodiment 3 of this invention.

Explanation of symbols

100, 150, 160 Information terminal 101 Video photographing unit 102 Display unit 103 First communication unit 104 Terminal identifier generation unit 105 Operation target information acquisition unit 106, 156, 166 Video analysis unit 157, 167 Display switching unit 200, 250, 260 Instruction device 201, 251, 261 Light emitting unit 202 Input unit 253, 263 Light emission switching unit 300 Operation target device 301 Identification unit 302 Second communication unit 303 Operation target information storage unit 304 Response unit 305 Operation target identifier generation unit 400, 450 Operation Target information 500 Operation target information acquisition request 600,650 Operation target information acquisition response 990 User

Claims (9)

An object instruction specifying system that specifies an operation target device by positioning an instruction device between an information terminal and an operation target device to be specified,
The pointing device is
An input unit for inputting user operations;
A light emitting unit that emits light based on the user operation,
The operation target device is:
A visually identifiable identification unit for identification with other operation target devices;
An operation target information storage unit that stores operation target information indicating the description or state of the device to be operated by itself; a second communication unit that transmits and receives an acquisition request and an acquisition response for the operation target information;
A response unit that generates an acquisition response including the operation target information stored in the operation target information storage unit when receiving an acquisition request from the outside, and
The information terminal
An image capturing unit that captures an image near the operation target device to be identified;
Video that identifies the object that has identified the identification unit as the operation target device when the superimposition of the light emitting unit of the pointing device and the identification unit of the operation target device is detected based on the video captured by the video imaging unit An analysis unit;
A first communication unit that transmits and receives an acquisition request and an acquisition response for the operation target information;
An operation target information acquisition unit that generates an operation target information acquisition request for the operation target device specified by the video analysis unit, and acquires an acquisition response as a response to the generated acquisition request;
An object instruction specifying system comprising: a display unit that displays the operation target information included in the acquisition response acquired by the operation target information acquisition unit in addition to the video captured by the video imaging unit. The operation target information includes commentary information indicating contents describing the device to be operated by itself,
The video analysis unit of the information terminal
Based on the video shot by the video shooting unit, further analyze the display position of the identification unit of the operation target device,
The display unit of the information terminal
The object indication specifying system according to claim 1, wherein the display position of the commentary information is determined and displayed based on the display position of the identification unit of the operation target device analyzed by the video analysis unit. The display unit of the information terminal
The target instruction specifying system according to claim 1, wherein the display position of the comment information is determined so as not to overlap with a display position of an identification unit of the operation target device, and the comment information is displayed in a balloon shape. The operation target information includes menu information indicating contents or operation functions stored in the operation target device,
The video analysis unit of the information terminal
Based on the video shot by the video shooting unit, further analyze the display position of the identification unit of the operation target device,
The display unit of the information terminal
The target instruction specifying system according to claim 1, wherein the display position of the menu information is determined and displayed based on the display position of the identification unit of the operation target device analyzed by the video analysis unit. The pointing device is
Based on the user operation, further comprising a light emission switching unit that switches the light emission state of the light emitting unit, the video analysis unit of the information terminal,
Based on the video shot by the video shooting unit, further analyze the light emission state of the light emitting unit of the pointing device,
The information terminal
The target object according to claim 1, further comprising: a display switching unit that instructs to switch display contents of the operation target information displayed on the display unit according to a light emission state of the light emitting unit of the pointing device analyzed by the video analysis unit. Instruction specific system. The object indication specifying system according to claim 5, wherein the light emission state of the light emitting unit that can be switched by the light emission switching unit of the light emitting unit is a light emission color, brightness, number of blinks, or blinking pattern. An object instruction specifying method for specifying an operation target device by positioning an instruction device between an information terminal and an operation target device to be specified,
Inputting a user operation to the pointing device;
Light-emitting the light-emitting unit of the pointing device based on the input user operation;
A shooting step of shooting an image near the operation target device to be identified;
Based on the video imaged in the imaging step, when the superimposition of the light emitting unit of the pointing device and the visible identification unit for identifying the operation target device from other operation target devices is detected, the identification unit is An identifying analysis step;
Identifying an object having an identification unit identified in the analysis step as an operation target device;
A storage step of storing operation target information indicating an explanation or a state of the own operation target device in the operation target device;
A step of generating an operation request information acquisition request for the specified operation target device; a request step of transmitting the generated acquisition request from the information terminal to the specified operation target device;
A step of generating an acquisition response including the operation target information stored in the storing step when the specified operation target device receives the acquisition request;
A response step of transmitting the generated acquisition response to the information terminal that has transmitted the acquisition request from the operation target device;
Based on the operation target information included in the acquisition response acquired as a response to the acquisition request transmitted in the request step, the information terminal captured the description or state of the specified operation target device in the shooting step. An object instruction specifying method comprising: displaying the image together with an image. An information terminal for identifying the operation target device by positioning an instruction device between the device and the operation target device to be specified,
A video capturing unit that captures video near the operation target device to be identified;
A light emitting unit that emits light based on a user operation of the pointing device based on an image captured by the image capturing unit, and a visually identifiable identifying unit for distinguishing from other operation target devices of the operation target device A video analysis unit that identifies the identification unit and identifies the target object as an operation target device when superposition is detected;
A first communication unit that transmits and receives an operation target information acquisition request and acquisition response indicating the description or status of the operation target device;
An operation target information acquisition unit that generates an operation target information acquisition request for the operation target device identified by the video analysis unit, and acquires an acquisition response as a response to the generated acquisition request;
A display unit for displaying the operation target information included in the acquisition response acquired by the operation target information acquisition unit on the video captured by the video imaging unit;
An information terminal that displays the description or state of the identified operation target device together with the video. A visually identifiable identification unit for identification with other operation target devices;
An operation target information storage unit that stores operation target information indicating the description or state of the device to be operated by itself; a second communication unit that transmits and receives an acquisition request and an acquisition response for the operation target information;
An operation target device comprising: a response unit that generates an acquisition response including the operation target information stored in the operation target information storage unit when an external acquisition request is received.

Images