JP2017146782A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device capable of making an interaction between an operating entity and an apparatus smoother.SOLUTION: The present invention relates to an information processing apparatus comprising: a projection control section which causes a projection section to project guide information for guiding an operation of an operating entity for which the operating entity requests a guide, in response to a projecting instruction of the guide information; and a processing control section by which, if a state where the operation which is guided by the guide information is performed by the operating entity is recognized, processing based on the state is performed. The present invention also relates to an information processing method and a program including: causing a projection section to project guide information for guiding an operation of an operating entity for which the operating entity requests a guide, in response to a projecting instruction of the guide information; and, if a state where the operation which is guided by the guide information is performed by the operating entity is recognized, performing the processing based on the state.SELECTED DRAWING: Figure 4

Description

  The present disclosure relates to an information processing apparatus, an information processing method, and a program.

  In recent years, with the development of information processing technology, interactive devices for users have been developed. Such devices generally output information in response to user input operations. For this reason, techniques for reducing the burden on user input operations have been developed.

  For example, Patent Document 1 includes a projector that displays an operation menu on the floor, and a range sensor that detects an object on the floor that operates the displayed operation menu, and uses the detected position of the object to detect the object. An invention relating to an actuator that performs processing corresponding to a position is disclosed.

JP 2008-6551 A

  However, further smoothing of the interaction between the operation subject and the apparatus is desired. Therefore, the present disclosure proposes a mechanism that can make the interaction between the operation subject and the apparatus smoother.

According to the present disclosure, the guide information is guided by the projection control unit that causes the projection unit to project guide information for guiding the operation of the operation subject requested by the operation subject according to the projection instruction of the guide information. When a state in which an action is performed by the action subject is recognized, a process control unit that performs a process based on the state;
An information processing apparatus is provided.

  Further, according to the present disclosure, using a processor, projecting guide information for guiding the motion of the motion subject that the motion subject requests guidance to the projection unit according to a projection instruction of the guide information; When a state in which an operation guided by guide information is performed by the operation subject is recognized, an information processing method including performing processing based on the state is provided.

  In addition, according to the present disclosure, the projection control function for projecting guide information for guiding the motion of the motion subject that the motion subject requests guidance to the projection unit according to a projection instruction of the guide information, and the guide information When the state in which the guiding operation is performed by the operation subject is recognized, a program for causing a computer to realize a processing control function for performing processing based on the state is provided.

  As described above, according to the present disclosure, a mechanism capable of smoother interaction between an operation subject and an apparatus is provided. Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with or in place of the above effects. May be played.

It is a figure for demonstrating the technique regarding the information processing apparatus which concerns on each embodiment of this indication. It is a figure for demonstrating the technique regarding the information processing apparatus which concerns on each embodiment of this indication. It is a figure for demonstrating the outline | summary of the function which recognizes the operation state of the information processing apparatus which concerns on each embodiment of this indication. It is a figure for demonstrating the outline | summary of the function which recognizes the operation state of the information processing apparatus which concerns on each embodiment of this indication. 2 is a block diagram schematically illustrating an example of a functional configuration of an information processing system according to a first embodiment of the present disclosure. FIG. 5 is a flowchart conceptually showing an example of processing of the information processing apparatus according to the embodiment. It is a figure for demonstrating the process example of the information processing system which concerns on the 1st application example of the embodiment. It is a figure for demonstrating the other example of a process of the information processing system which concerns on the 1st application example of the embodiment. It is a figure for demonstrating the process example of the information processing system which concerns on the 2nd application example of the embodiment. It is a figure for demonstrating the process example of the information processing system which concerns on the 3rd application example of the embodiment. It is a figure for demonstrating the process example of the information processing system which concerns on the 4th application example of the embodiment. It is a figure for demonstrating the process example of the information processing system which concerns on the 5th application example of the embodiment. It is a figure for demonstrating the process example of the information processing system which concerns on the 6th application example of the embodiment. It is a figure for demonstrating the process example of the information processing system which concerns on the 7th application example of the embodiment. It is a figure for demonstrating the process example of the information processing system which concerns on the modification of the embodiment. 14 is a flowchart conceptually illustrating an example of processing of an information processing device according to a second embodiment of the present disclosure. It is a figure for demonstrating the process example of the information processing system which concerns on the 1st application example of the embodiment. It is a figure for demonstrating the process example of the information processing system which concerns on the 2nd application example of the embodiment. FIG. 3 is an explanatory diagram illustrating a hardware configuration of an information processing apparatus according to an embodiment of the present disclosure.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  In the present specification and drawings, a plurality of constituent elements having substantially the same functional configuration may be distinguished by attaching different numbers after the same reference numerals. For example, a plurality of configurations having substantially the same function are differentiated as necessary, such as the device 10A and the device 10B. However, when it is not necessary to distinguish between substantially the same functional configurations, only the same reference numerals are given. For example, when it is not necessary to distinguish between the device 10A and the device 10B, they are simply referred to as the device 10.

The description will be made in the following order.
1. Introduction 2. First embodiment (basic form)
2-1. System configuration 2-2. System processing 2-3. Application example 2-4. Summary of first embodiment 2-5. Modified example 2. Second embodiment (example of error processing)
3-1. System configuration 3-2. System processing 3-3. Application example 3-4. Summary of Second Embodiment 3-5. Modified example 4. 4. Hardware configuration of information processing apparatus according to one embodiment Conclusion

<1. Introduction>
First, a technique related to an information processing apparatus according to each embodiment of the present disclosure will be described.

  The apparatus 10 according to the above technique has a communication function and a projection function. The communication function is a function for receiving information from an external device or transmitting information to an external device. The projection function is a function for projecting an image around the device. For this reason, the apparatus 10 can project an image based on information obtained through communication. 1A and 1B are diagrams for describing a technology related to the information processing apparatus 100 according to each embodiment of the present disclosure.

  For example, consider a case where the user and the device 10 play shogi. The apparatus 10 projects the image of the shogi piece on the area on the apparatus 10 side of the shogi board as shown by the broken line in FIG. 1A. The user moves the piece on the shogi board and inputs to the communication terminal 20 which piece is moved to where. The communication terminal 20 and the apparatus 10 communicate piece movement information. Then, the apparatus 10 changes the position of the image of the frame to be projected based on the received frame movement information. In this way, a shogi battle between the user and the device 10 is realized.

  In addition, another device 10 may be a shogi opponent in place of the user. For example, as shown in FIG. 1B, the device 10A and the device 10B project images of shogi pieces on each device side of the shogi board, respectively. In the case where the device 10A is ahead, the device 10A determines which frame is moved to where, and changes the position of the image of the projected frame to the determined position. In addition, the device 10A transmits the determined piece movement information to the device 10B. The apparatus 10B changes the position of the image of the projected frame based on the received frame movement information. In this way, a shogi battle between the devices 10 is realized.

  However, in the above-described technique, there is a possibility that the interaction between the operation subject and the apparatus becomes complicated. For example, it may be a burden on the user to input state change information such as piece movement information one by one. In addition, a communication function is required for realizing the interaction. For example, if there is no device or communication function corresponding to the communication with the device 10, it is difficult to use the interaction with the device 10.

  Therefore, the information processing apparatus 100 according to each embodiment of the present disclosure has a recognition function that recognizes the operation state of the operation subject. 2A and 2B are diagrams for describing an overview of a function for recognizing an operation state of the information processing apparatus 100 according to each embodiment of the present disclosure.

  For example, the information processing apparatus 100 captures an area corresponding to a shogi board as shown by a dotted line in FIG. 2A and recognizes the user's operation state based on an image obtained by the imaging. When the state in which the user moves the piece is recognized, the information processing apparatus 100 grasps piece movement information based on the position of the piece shown in the image. The information processing apparatus 100 changes the position of the image of the projected frame based on the grasped movement information of the frame. On the other hand, the user only moves the piece and does not perform an input operation for providing the information processing apparatus 100.

  Moreover, the shogi confrontation between the information processing apparatuses 100 will be described. When the information processing apparatus 100A precedes, similarly to the apparatus 10, the information processing apparatus 100A determines which frame to move to and changes the position of the image of the projected frame to the determined position. However, the information processing apparatus 100A does not transmit the piece movement information to the information processing apparatus 100B as illustrated in FIG. 2B. The information processing apparatus 100B captures an area corresponding to the shogi board, and recognizes the operation state of the information processing apparatus 100A based on an image obtained by the imaging. When it is recognized that the information processing apparatus 100A has moved the piece, the information processing apparatus 100B changes the position of the image of the piece to be projected based on the position of the piece shown in the image.

  Thus, the information processing apparatus 100 has a recognition function that recognizes the operation state of the operation subject. For this reason, it is possible to reduce the operation or processing of the operation subject. Accordingly, it is possible to suppress complication of interaction between the operation subject and the apparatus. Hereinafter, an information processing system having the recognition function will be described in detail. For convenience of explanation, like the information processing apparatus 100-1 and the information processing apparatus 100-2, the information processing apparatus 100 according to the first and second embodiments is suffixed with a number corresponding to the embodiment. To distinguish. The same applies to the server 200 described later. 2A and 2B show an example in which the information processing apparatus 100 is a dog-shaped robot, the information processing apparatus 100 is not limited to this, and may be an apparatus other than the robot.

<2. First embodiment (basic form)>
A first embodiment of the present disclosure will be described. In the first embodiment, when guide information is projected based on a guide instruction for guiding information by voice input by the user and a state in which the guide information is guided is recognized, processing corresponding to the state is performed. Executed.

<2-1. System configuration>
First, the functional configuration of the information processing system according to the first embodiment of the present disclosure will be described with reference to FIG. FIG. 3 is a block diagram schematically illustrating an example of a functional configuration of the information processing system according to the first embodiment of the present disclosure.

  As shown in FIG. 3, the information processing system according to the present embodiment includes an information processing apparatus 100-1 and a server 200-1.

(Information processing device)
The information processing apparatus 100-1 includes a voice input unit 102, an imaging unit 104, a control unit 106, a projection unit 108, a voice output unit 110, and a communication unit 112. Note that the information processing apparatus 100-1 may be a portable apparatus.

(Voice input part)
The voice input unit 102 collects voice around the information processing apparatus 100-1. Specifically, the voice input unit 102 collects voice around the information processing apparatus 100-1 and generates information related to the collected voice (hereinafter also referred to as voice information). Note that the voice input unit 102 may collect sound other than voice. Also, the voice information may be acquired from an external voice input device via the communication unit 112.

(Imaging part)
The imaging unit 104 images the periphery of the information processing apparatus 100-1. Specifically, the imaging unit 104 captures an area related to the operation of the information processing apparatus 100-1 (hereinafter also referred to as an operation area), and information related to an image obtained by the imaging (hereinafter referred to as a captured image). Also referred to as information). The operation area is an area where an image is projected or can be projected by the projection unit 108. Note that the image obtained by imaging by the imaging unit 104 may be an image in which the action subject is shown, or may be an image in which the action subject is not shown. The captured image information may be acquired from an external imaging device via the communication unit 112.

(Control part)
The control unit 106 generally controls the functions of the information processing apparatus 100-1. Specifically, the control unit 106 controls processing of the voice input unit 102, the imaging unit 104, and the projection unit 108. For example, the control unit 106 controls processing contents of the voice input unit 102, the imaging unit 104, and the projection unit 108 by setting control parameters and the like.

  Moreover, the control part 106 acquires the recognition result about audio | voice information and captured image information from the server 200-1. For example, the control unit 106 requests the server 200-1 to perform voice recognition processing and image recognition processing for each of the voice information input from the voice input unit 102 and the captured image information input from the imaging unit 104 via communication. To do. Then, the control unit 106 receives the voice recognition result and the image recognition result from the server 200-1 via communication.

  Further, as a projection control unit, the control unit 106 causes the projection unit 108 to project the guide information in response to the guide information projection instruction. The guide information is information for guiding the action of the action subject that the action subject requests for guidance. Specifically, when the voice recognition result provided from server 200-1 indicates a guide information projection instruction, control unit 106 projects information for projecting the guide information (hereinafter also referred to as material information). Obtained from the server 200-1 via communication. Then, guide information is projected on the projection unit 108 based on the acquired material information. For example, the control unit 106 analyzes text information obtained by the speech recognition process, and determines whether the text information indicates a guide information projection instruction. When it is determined that the text information indicates an instruction to project guide information, the control unit 106 requests the server 200-1 to provide material information related to the guide information indicated by the text information via communication. When the material information is provided from the server 200-1, the control unit 106 causes the projection unit 108 to project guide information based on the provided material information. Note that the guide information projection instruction may be a direct instruction of the guide information projection or an indirect instruction. For example, the indirect instruction of the guide information projection may be a request or request to the information processing apparatus 100 related to the guide information. Moreover, material information may be memorize | stored in the information processing apparatus 100-1, and may be produced | generated within the information processing apparatus 100-1. Moreover, the material information provided from the server 200-1 may be processed in the information processing apparatus 100-1.

  Further, as a process control unit, the control unit 106 performs processing based on the target state when a state in which the operation guided by the guide information is performed by the operating subject (hereinafter also referred to as a target state) is recognized. Specifically, the control unit 106 determines whether the operation state recognized based on the captured image information related to the image in which the operation result of the operation subject is reflected is the target state. When it is determined that the operation state is the target state, the control unit 106 performs processing based on the target state. For example, the control unit 106 sets a target state when projection of guide information is started. Next, the control unit 106 determines whether the operation state has reached the target state at predetermined time intervals. Specifically, the control unit 106 determines a change in the position of the action subject (or the position of the object moved by the action subject) based on the position information of the object obtained by the image recognition process. When it is determined that the position of the operation subject has changed to the target state, the control unit 106 determines that the operation state has reached the target state. Then, the control unit 106 determines that the operation state has reached the target state, and after performing processing based on the target state, sets the next target state.

(Projection part)
The projection unit 108 projects an image around the information processing apparatus 100-1. Specifically, the projection unit 108 projects an image related to image information (hereinafter also referred to as projection image information) input from the control unit 106 around the information processing apparatus 100-1. For example, the projection unit 108 projects guide information input from the control unit 106 based on material information to a position designated by the control unit 106. Note that the projection unit 108 may project images separately on a plurality of regions. Further, the projection area may be fixed or variable. When the projection area is variable, the projection area may be controlled by the control unit 106.

(Audio output part)
The audio output unit 110 outputs audio to the outside of the information processing apparatus 100-1. Specifically, the audio output unit 110 outputs audio related to the audio information input from the control unit 106 to the outside of the information processing apparatus 100-1. Note that the audio information related to the output audio may be generated by the control unit 106 or acquired from an external device via the communication unit 112.

(Communication Department)
The communication unit 112 communicates with a device external to the information processing device 100-1. Specifically, the communication unit 112 transmits voice information, captured image information, and an information provision request to the server 200-1, and receives a voice recognition result, an image recognition result, and material information from the server 200-1. Note that the communication unit 112 may communicate with the server 200-1 using a wired communication method or may use a wireless communication method.

(server)
The server 200-1 includes a communication unit 202, a voice recognition unit 204, an image recognition unit 206, an information providing unit 208, and a storage unit 210.

(Communication Department)
The communication unit 202 communicates with a device external to the server 200-1. Specifically, the communication unit 202 receives voice information, captured image information, and an information provision request from the information processing apparatus 100-1, and transmits the voice recognition result, the image recognition result, and the material information to the information processing apparatus 100-1. To do. The communication unit 202 has substantially the same function as the communication unit 112.

(Voice recognition unit)
The voice recognition unit 204 recognizes voice related to voice information. Specifically, the voice recognition unit 204 generates text information based on the voice information. For example, the speech recognition unit 204 estimates characters corresponding to the content of speech related to input speech information, and generates text information as a speech recognition result using the estimated characters. Note that the voice recognition unit 204 may specify a speaker who speaks voice information.

(Image recognition unit)
The image recognition unit 206 recognizes an image related to image information. Specifically, the image recognizing unit 206 recognizes the positional relationship between the objects shown in the image related to the image information. For example, the image recognition unit 206 recognizes an object appearing in an image related to input captured image information, and estimates the arrangement of the recognized object. Then, the image recognition unit 206 generates information regarding the recognized object and the arrangement of the object as an image recognition result. Note that a technology such as SLAM (Simultaneous Localization and Mapping) may be used for the image recognition processing. Further, the image recognition unit 206 may further perform face recognition processing.

(Information provision department)
The information providing unit 208 provides information to the information processing apparatus 100-1. Specifically, the information providing unit 208 selects information to be provided in response to an information provision request from the information processing apparatus 100-1 and communicates the selected information to the information processing apparatus 100-1 via communication. provide. For example, the information providing unit 208 acquires material information stored in the storage unit 210 in response to a material information provision request. Then, the information providing unit 208 causes the communication unit 202 to transmit the acquired material information to the information processing apparatus 100-1.

(Memory part)
The storage unit 210 stores information used for processing of the server 200-1 and information provided to the information processing apparatus 100-1. For example, the storage unit 210 stores information used for voice recognition processing or image recognition processing, and image information or text information as material information.

<2-2. System processing>
Next, processing of the information processing apparatus 100-1 according to the present embodiment will be described with reference to FIG. FIG. 4 is a flowchart conceptually showing an example of processing of the information processing apparatus 100-1 according to the present embodiment.

  The information processing apparatus 100-1 determines whether a guide information projection instruction has been acquired (step S302). Specifically, the control unit 106 causes the server 200-1 to perform voice recognition processing on the voice information input from the voice input unit 102, and acquires a voice recognition result. Then, the control unit 106 determines whether the acquired voice recognition result indicates a guide information projection instruction.

  If it is determined that the guide information projection instruction has been acquired, the information processing apparatus 100-1 sets a target state according to the guide information (step S304). Specifically, when it is determined that the voice recognition result indicates a guide information projection instruction, the control unit 106 sets an operation state corresponding to the guide information related to the projection instruction as a target state.

  Next, the information processing apparatus 100-1 recognizes the operating state (step S306). Specifically, the control unit 106 causes the server 200-1 to perform image recognition processing on the image information input from the imaging unit 104, and acquires an image recognition result.

  Next, the information processing apparatus 100-1 determines whether the recognized operation state is a target state (step S308). Specifically, the control unit 106 determines whether the operation state specified based on the acquired image recognition result matches the target state. Note that an error within a predetermined range may be allowed for the consistency with the target state (reachability to the target state).

  When it is determined that the recognized operation state is the target state, the information processing apparatus 100-1 executes processing based on the target state (step S310). Specifically, when it is determined that the operation state specified based on the image recognition result matches the target state, the control unit 106 executes processing based on the target state.

  When it is determined that the recognized operation state is not the target state, the information processing apparatus 100-1 returns the process to step S306.

  Next, the information processing apparatus 100-1 determines whether the projection of the guide information has ended (step S312), and ends the process when it is determined that the projection of the guide information has ended. If it is not determined that the projection of the guide information has been completed, the process returns to step S304 and the process is repeated.

<2-3. Application example>
The information processing system according to this embodiment has been described above. The information processing system can be modified according to various cases based on the above-described embodiment, and can be applied to each case. Hereinafter, first to seventh application examples will be described.

(First application example)
As a first application example of this embodiment, an example of shogi will be described. In this application example, when the operation on the projected guide information is guided by the guide information, and the target state related to the operation is recognized, the information processing apparatus 100-1 performs the processing of the projection content as the process based on the target state. Make a change. Specifically, the control unit 106 causes the projection unit 108 to project an image showing a shogi piece as guide information, and the operation of the operation subject on the image showing the projected shogi piece reaches the operation related to the target state. If it is determined that the display has been performed, the display position of the image indicating the shogi piece is moved in accordance with the operation related to the target state. Furthermore, the processing of this application example will be described in detail with reference to FIG. FIG. 5 is a diagram for explaining a processing example of the information processing system according to the first application example of the present embodiment.

  In the example of FIG. 5, the user and the information processing apparatus 100-1 are arranged across the shogi board. On the shogi board, a piece as a real object (hereinafter, the piece 30 will be described as a representative) is arranged on the user side in advance. Note that the pieces arranged on the user side may be projected by the information processing apparatus 100-1.

  First, the information processing apparatus 100-1 determines whether an instruction to project guide information related to shogi has been acquired based on a user's voice input. For example, as illustrated at time t <b> 1 in FIG. 5, the control unit 106 obtains text information related to the text “I want to play shogi” from the server 200-1 using the voice information input from the voice input unit 102. Then, the control unit 106 interprets the user instruction “Shogi confrontation” based on the text information, and determines that the projection instruction of “Shogi piece” has been acquired from the interpreted instruction.

  When an instruction to project guide information related to shogi is acquired, the information processing apparatus 100-1 projects the guide information related to shogi to the user's motion area. For example, as shown at time t <b> 2 in FIG. 5, the control unit 106 displays a shogi piece (hereinafter, representatively a piece on the shogi board) on the information processing apparatus 100-1 side of the shogi board shown in the image obtained by the imaging unit 104. G1 will be described.) Is projected on the projection unit 108.

  Next, the information processing apparatus 100-1 sets a state where the user's operation is performed on the guide information related to the shogi projected on the motion area as the target state. Specifically, for example, the control unit 106 sets a state where the piece 30 is moved by the user as a target state.

  Next, the information processing apparatus 100-1 determines whether the operation state has reached the set target state. For example, the control unit 106 determines whether or not the piece 30 on the shogi board has moved based on the result of image recognition processing using image information obtained by imaging by the imaging unit 104. When the piece 30 is moved by the user as shown at time t3 in FIG. 5, the control unit 106 determines that the state in which the piece 30 has been moved by the user, that is, the target state has been reached.

  If it is determined that the operation state has reached the set target state, the information processing apparatus 100-1 changes the projection content based on the operation related to the target state. For example, when the piece 30 is moved, the control unit 106 determines the next move based on the movement of the piece 30. Then, the control unit 106 determines the projection position of the frame G1 based on the next move to be determined, and causes the projection unit 108 to project the frame G1 at the projection position as shown at time t4 in FIG.

  Thereby, the user can enjoy shogi with the information processing apparatus 100-1 without complicated operations.

  Further, the information processing apparatus 100-1 according to the present embodiment is also applied to a shogi battle between the information processing apparatuses 100-1. With reference to FIG. 6, the example of the shogi confrontation between information processing apparatuses 100-1 is demonstrated. FIG. 6 is a diagram for explaining another example of the process of the information processing system according to the first application example of the present embodiment. Note that description that is substantially the same as the description using the example of FIG. 5 is omitted.

  In the example of FIG. 6, the information processing device 100-1A and the information processing device 100-1B are arranged with the shogi board interposed therebetween. Also, nothing is placed on the shogi board. The shogi board may not be prepared. In that case, at least one of the information processing apparatuses 100-1A or 100-1B may project a shogi board.

  First, the information processing apparatus 100-1 determines whether an instruction to project guide information related to shogi has been acquired based on a user's voice input. For example, the information processing apparatuses 100-1A and 100-1B use the voice information input from the voice input unit 102 as shown at time t5 in FIG. Get such text information. Then, the control unit 106 determines that the projection instruction of “shogi piece” is acquired from the user instruction “shogi confrontation” interpreted based on the text information.

  When an instruction to project guide information related to shogi is acquired, the information processing apparatus 100-1 projects the guide information onto the user's motion area. For example, as shown at time t6 in FIG. 6, the shogi piece G1A is projected by the information processing device 100-1A on the information processing device 100-1A side of the shogi board and on the information processing device 100-1B side of the shogi board The shogi piece G1B is projected by the information processing apparatus 100-B.

  Next, the information processing apparatus 100-1 sets, as a target state, a state in which an operation related to the movement of the piece of the information processing apparatus 100-1 of the other party is performed on the guide information projected on the operation area. Specifically, for example, when the information processing apparatus 100-1A is ahead, the information processing apparatus 100-1B sets the state in which the piece G1A is moved by the information processing apparatus 100-1A as the target state. Further, the information processing apparatus 100-1A sets the state where the piece G1B is moved by the information processing apparatus 100-1B as the target state after moving the piece G1A.

  Next, the information processing apparatus 100-1 determines whether the other party's operation state has reached the set target state. For example, the information processing apparatus 100-1B determines whether or not the shogi board piece G1A has moved. When the piece G1A projected by the information processing apparatus 100-1A is moved as shown at time t7 in FIG. 6, the information processing apparatus 100-1B operates the set information processing apparatus 100-1A. It is determined that

  When it is determined that the operation state of the other party has reached the set target state, the information processing apparatus 100-1 changes the projection content based on the operation. For example, when the piece G1A is moved, the information processing apparatus 100-1B determines the next move based on the movement of the piece G1A. Then, the information processing apparatus 100-1B determines the projection position of the frame G1B based on the determined next move, and projects the frame G1B to the determined position as shown at time t8 in FIG. To project.

  Thereby, the user can realize the shogi confrontation between the information processing apparatuses 100-1 without a burden. In addition, although the example of shogi was demonstrated above, the game to which the information processing system which concerns on this embodiment is applied may be another face-to-face game. For example, the applied game may be chess, othello or go. Further, the present invention may be applied to a game with three or more players (three devices).

(Second application example)
An example of information scanning will be described as a second application example of the present embodiment. In this application example, when the target state is recognized, the information processing apparatus 100-1 performs a process corresponding to the projection instruction as a process based on the target state. Specifically, the control unit 106 causes the projection unit 108 to project an image indicating a scan area as guide information, and determines that the operation state has reached the state where the scan target is placed in the projected area. Execute the scan for the scan area. Furthermore, the processing of this application example will be described in detail with reference to FIG. FIG. 7 is a diagram for explaining a processing example of the information processing system according to the second application example of the present embodiment.

  In the example of FIG. 7, consider a case where the user causes the information processing apparatus 100-1 to scan the description content of the paper 50. Note that description of processing that is substantially the same as the processing described above is omitted.

  First, the information processing apparatus 100-1 determines whether a guide information projection instruction related to scanning has been acquired based on a user's voice input. For example, the control unit 106 obtains text information related to the text “I want to scan this paper” from the server 200-1 using the audio information input from the audio input unit 102 as shown at time t <b> 9 in FIG. 7. . Then, the control unit 106 determines that the projection instruction of “scan area” has been acquired from the user instruction “scan paper” interpreted based on the text information.

  When an instruction to project guide information related to scanning is acquired, the information processing apparatus 100-1 projects guide information onto a scan area. For example, as illustrated at time t10 in FIG. 7, the control unit 106 displays a frame G2 indicating the scan area in the scan area of the information processing apparatus 100-1 and text G3 that prompts the user to place the scan target in the frame G2. Is projected on the projection unit 108.

  Next, the information processing apparatus 100-1 sets, as a target state, a state in which an operation related to a user's scan for guide information projected on the scan area is performed. Specifically, for example, the control unit 106 sets, as a target state, a state in which the paper 50 to be scanned is arranged in a frame G2 indicating a scan area by the user.

  Next, the information processing apparatus 100-1 determines whether the operation state has reached the set target state. For example, the control unit 106 determines whether or not the scan target is arranged in the frame G2 indicating the scan area. As shown at time t11 in FIG. 7, when the user places the paper 50 in the frame G2 on which the paper 50 is projected, the control unit 106 recognizes the state in which the paper 50 is placed in the frame G2 based on the image information. Then, it is determined that the set user operation has been performed.

  When it is determined that the user's operation state has reached the set target state, the information processing apparatus 100-1 executes a process corresponding to the projection instruction based on an operation related to the target state. For example, when the paper 50 is placed in the frame G2, the control unit 106 ends the projection of the text G3 as shown at time t11 in FIG. Next, the control unit 106 causes the imaging unit 104 to image the scan area indicated by the frame G2, thereby executing a scan corresponding to the projection instruction for the “scan area”. Then, the control unit 106 stores the image obtained by the imaging in the storage unit in the information processing apparatus 100-1, and captures the voice “captured” indicating that the scan has ended as shown at time t12 in FIG. Is output to the audio output unit 110.

  As a result, the user can easily use the scan. Further, by indicating the scan area or the like, it is possible to suppress scan failure.

(Third application example)
An example of calligraphy support will be described as a third application example of this embodiment. In this application example, when the target state is recognized, the information processing apparatus 100-1 guides the next operation after the operation of guiding the projected guide information as a projection content change, as the change of the projection content. Change to Further, the guide information may be information indicating the order of actions of the action subject. Specifically, the control unit 106 causes the projection unit 108 to project an image indicating the calligraphy target character and the writing order of the character as guide information, and the character portion corresponding to the writing order indicated by the projected image is written. If it is determined that the operating state has reached the state, the projection unit 108 is caused to project the next stroke order after the projected stroke order. Furthermore, the processing of this application example will be described in detail with reference to FIG. FIG. 8 is a diagram for explaining a processing example of the information processing system according to the third application example of the present embodiment.

  In the example of FIG. 8, a case is considered where the user causes the information processing apparatus 100-1 to support calligraphy. Note that description of processing that is substantially the same as the processing described above is omitted.

  First, the information processing apparatus 100-1 determines whether an instruction to project guide information related to calligraphy is acquired based on a user's voice input. For example, the control unit 106 obtains text information related to the text “I want to practice calligraphy” from the server 200-1 using the voice information input from the voice input unit 102 as shown at time t <b> 13 in FIG. 8. . Then, the control unit 106 determines that projection instructions of “characters subject to calligraphy” and “writing order” have been acquired from the user's instruction “support of calligraphy” interpreted based on the text information.

  When an instruction to project guide information related to a calligraphy is acquired, the information processing apparatus 100-1 projects the guide information onto an area used for the calligraphy (hereinafter also referred to as a calligraphy area). For example, the control unit 106 determines a calligraphy area around the information processing apparatus 100-1. In the calligraphy area to be determined, the control unit 106 writes the frame G4 indicating the calligraphy area, the character G5 (“T” in FIG. 8) to be the calligraphy object, and the character G5, as shown at time t14 in FIG. An image G6A indicating the order (hereinafter also referred to as a writing order G6A) is projected on the projection unit 108. The frame G4 may not be projected. The calligraphy area may be paper prepared for calligraphy, and in that case, the control unit 106 may determine the calligraphy area by recognizing the paper.

  Next, the information processing apparatus 100-1 sets, as a target state, a state in which an operation of writing a user's character with respect to guide information projected on the calligraphy area is performed. Specifically, for example, the control unit 106 sets, as a target state, a state in which a character portion corresponding to the writing order G6A is written in the frame G4 by the user. It is not necessary to actually write characters. For example, a state where a writing instrument such as a brush for writing characters or a finger is moved according to the writing order G6A may be set as the target state.

  Next, the information processing apparatus 100-1 determines whether the user's operation state has reached the set target state. For example, the control unit 106 determines whether the part of the character G5 corresponding to the projected writing order G6A has been written. As shown at time t15 in FIG. 8, when the part of the character G5 corresponding to the writing order G6A (the first drawing of “T”) is written by the user, the control unit 106 follows the writing order G6A. It is determined that the state where the part of the character G5 corresponding to is written is reached.

  When it is determined that the user's operation state has reached the set target state, the information processing apparatus 100-1 projects the guide information next to the guide information being projected. For example, when the part of the character G5 corresponding to the writing order G6A (first drawing of “T”) is written, the control unit 106 projects the next writing order G6B as shown at time t16 in FIG. To project.

  Thereby, the user can practice writing characters in the correct writing order. In addition, although the example of the calligraphy of the alphabet was demonstrated above, the character of calligraphy object may be any other characters, such as a Chinese character or an Arabic character. Further, the information processing system according to the present embodiment may be applied in the drawing order of symbols or figures other than characters. Further, the projection order information such as the drawing order may be included in the material information provided from the server 200-1.

(Fourth application example)
As a fourth application example of this embodiment, an example of teaching how to arrange ingredients in cooking will be described. In this application example, the information processing apparatus 100-1 projects guide information indicating the position of the motion of the motion subject. Specifically, the control unit 106 causes the projection unit 108 to project only an image indicating the position where the ingredient is to be arranged as guide information, and the operation state is such that the ingredient is arranged at the arrangement position indicated by the projected image. If it is determined that has reached, the projection position next to the projection position to be projected is projected on the projection unit 108. Furthermore, the processing of this application example will be described in detail with reference to FIG. FIG. 9 is a diagram for explaining a processing example of the information processing system according to the fourth application example of the present embodiment.

  In the example of FIG. 9, consider a case where the user presents an example of cooking arrangement to the information processing apparatus 100-1. Note that description of processing that is substantially the same as the processing described above is omitted.

  First, the information processing apparatus 100-1 determines whether a guide information projection instruction related to the arrangement of ingredients has been acquired based on a user's voice input. For example, the control unit 106 uses the voice information input from the voice input unit 102 as shown at time t17 in FIG. 9 to send text information related to the text “Show me an example of arrangement” from the server 200-1. Get. Then, the control unit 106 determines that the projection instruction of “positioning position” has been acquired from the user instruction “presentation of ingredient arrangement” interpreted based on the text information.

  When the projection instruction of the guide information related to the arrangement of the ingredients is acquired, the information processing apparatus 100-1 displays the area where the ingredients are arranged (hereinafter referred to as the arrangement area). (Also referred to as “.”). For example, the control unit 106 determines a placement area around the information processing apparatus 100-1. The control unit 106 causes the projection unit 108 to project only the image G7A indicating the placement position (hereinafter also referred to as the placement position G7A), as shown at time t18 in FIG. In addition, the said arrangement | positioning area | region may be the dish in which ingredients are arranged, and the control part 106 may determine the arrangement | positioning area | region by recognizing the said dish in that case.

  Next, the information processing apparatus 100-1 sets, as a target state, a state in which an operation for arranging the user's ingredients with respect to guide information projected on the arrangement area is performed. Specifically, for example, the control unit 106 sets, as a target state, a state in which ingredients corresponding to the placement position G7A are arranged at the projected placement position G7A.

  Next, the information processing apparatus 100-1 determines whether the user's operation state has reached the set target state. For example, the control unit 106 determines whether an ingredient has been arranged at the projected placement position G7A. As shown at time t19 in FIG. 9, when the ingredient 60A is placed at the placement position G7A by the user, the control unit 106 determines that the corresponding ingredient has been placed at the placement position G7A. To do.

  When it is determined that the user's operation state has reached the set target state, the information processing apparatus 100-1 projects the guide information next to the guide information being projected. For example, when the ingredient 60A is placed at the placement position G7A, the control unit 106 projects only the next placement position G7B on the projection unit 108 as shown at time t20 in FIG. Then, as shown at time t21 in FIG. 9, the user places the ingredient 60B at the serving position G7B.

  Thereby, even if it is an inexperienced dish or a dish that is difficult to serve, the user can neatly arrange the ingredients.

(Fifth application example)
As a fifth application example of this embodiment, an example of teaching a storage method will be described. In the present application example, the information processing apparatus 100-1 projects guide information indicating the position of the motion of the operation subject, as in the fourth application example. Specifically, the control unit 106 causes the projection unit 108 to project an image indicating the storage position of the object as guide information, and the operation state has reached the state where the object is stored at the storage position indicated by the projected image. If it is determined, the projection unit 108 is caused to project the storage position next to the storage position to be projected. Furthermore, the processing of this application example will be described in detail with reference to FIG. FIG. 10 is a diagram for explaining a processing example of the information processing system according to the fifth application example of the present embodiment.

  In the example of FIG. 10, consider a case where the user presents a storage method to the information processing apparatus 100-1. Note that description of processing that is substantially the same as the processing described above is omitted.

  First, the information processing apparatus 100-1 determines whether a guide information projection instruction related to storage of an object has been acquired based on a user's voice input. For example, the control unit 106 uses the audio information input from the audio input unit 102 as shown at time t22 in FIG. 10 to send the text information related to the text “Tell me how to store” from the server 200-1. obtain. Then, the control unit 106 determines that the projection instruction of “storage position” has been acquired from the user instruction “presentation of storage method” interpreted based on the text information.

  When an instruction to project guide information related to the storage of an object is acquired, the information processing apparatus 100-1 projects the guide information onto an area in which the object is stored (hereinafter also referred to as a storage area). For example, the control unit 106 determines a storage area around the information processing apparatus 100-1. The control unit 106 causes the projection unit 108 to project an image G8A indicating the storage position of the object (hereinafter also referred to as storage position G8A) onto the determined storage area, as shown at time t23 in FIG. The storage area may be an object having a storage function. In this case, the control unit 106 may determine the storage area by recognizing the object having the storage function.

  Next, the information processing apparatus 100-1 sets, as a target state, a state in which the user's object storage operation is performed on the guide information projected on the storage area. Specifically, for example, the control unit 106 sets, as a target state, a state in which an object corresponding to the storage position G8A is arranged at the storage position G8A to be projected.

  Next, the information processing apparatus 100-1 determines whether the user's operation state has reached the set target state. For example, the control unit 106 determines whether an object is placed at the storage position G8A to be projected. As shown at time t24 in FIG. 10, when the user places the object 62A at the storage position G8A, the control unit 106 determines that the corresponding object has been placed at the storage position G8A.

  When it is determined that the user's operation state has reached the set target state, the information processing apparatus 100-1 projects the guide information next to the guide information being projected. For example, when the object 62A is arranged at the storage position G8A, the control unit 106 causes the projection unit 108 to project the storage position G8B of the next object as shown at time t25 in FIG. Then, as shown at time t26 in FIG. 10, the user places the object 62B at the storage position G8B.

  As a result, even a user who is not good at storing can cleanly store things. In addition, the storage space and the storage work time can be made more efficient. Although an example of storing an object in a box has been described above, the information processing system according to the present embodiment may be applied to storing an object in a refrigerator or a bookshelf.

(Sixth application example)
As a sixth application example of this embodiment, an example of teaching how to wrap an object will be described. In this application example, the information processing apparatus 100-1 projects guide information indicating the position of the motion of the motion subject according to the shape of the object existing in the projection area. Specifically, the control unit 106 causes the projection unit 108 to project an image showing a fold of a cloth for wrapping an object as guide information, and the operation state is such that the cloth is folded at the fold indicated by the projected image. If it is determined that the image has been reached, the projection unit 108 is caused to project an image indicating a fold next to the fold indicated by the projected image. Furthermore, the processing of this application example will be described in detail with reference to FIG. FIG. 11 is a diagram for explaining a processing example of the information processing system according to the sixth application example of this embodiment.

  In the example of FIG. 11, a case is considered in which the user causes the information processing apparatus 100-1 to present how to wrap an object. Note that description of processing that is substantially the same as the processing described above is omitted.

  First, the information processing apparatus 100-1 determines whether or not an instruction to project guide information related to how to wrap an object is acquired based on a user's voice input. For example, the control unit 106 obtains the text information related to the text “Tell me how to wrap” from the server 200-1 using the audio information input from the audio input unit 102 as shown at time t27 in FIG. . Then, the control unit 106 determines that the projection instruction of “folding cloth fold” has been acquired from the user instruction “presentation of how to wrap an object” interpreted based on the text information.

  When an instruction to project guide information relating to how to wrap an object is acquired, the information processing apparatus 100-1 projects the guide information along the shape of the object. For example, the control unit 106 recognizes an object and cloth placed around the information processing apparatus 100-1. As shown at time t28 in FIG. 11, the control unit 106 causes the projection unit 108 to project an image G9A (hereinafter also referred to as a fold line G9A) indicating the fold of the cloth surrounding the object.

  Next, the information processing apparatus 100-1 sets a state where an operation of folding the user's cloth for the projected guide information is performed as a target state. Specifically, for example, the control unit 106 sets, as a target state, a state in which the cloth is folded at a fold line that matches the projected fold line G9A and the object is covered with the cloth.

  Next, the information processing apparatus 100-1 determines whether the user's operation state has reached the set target state. For example, the control unit 106 determines whether the cloth is folded like the projected crease G9A. As shown at time t29 in FIG. 11, when the cloth covering the object is folded along the fold G9A by the user, the control unit 106 determines that the cloth covering the object at the fold G9A has been folded. .

  When it is determined that the user's operation state has reached the set target state, the information processing apparatus 100-1 projects the guide information next to the guide information being projected. For example, when the cloth is folded like the crease G9A, the control unit 106 causes the projection unit 108 to project the next crease G9B as shown at time t30 in FIG. Then, as shown at time t31 in FIG. 11, the user wraps the object by folding the cloth at the crease G9B.

  Accordingly, even a user who does not know how to wrap an object or an object that is difficult to wrap can wrap the object neatly.

(Seventh application example)
An example of video reproduction will be described as a seventh application example of the present embodiment. In this application example, as in the second application example, the information processing apparatus 100-1 performs processing corresponding to the projection instruction when the target state is recognized. Specifically, the control unit 106 causes the projection unit 108 to project an image indicating the start of video reproduction as guide information, and when it is determined that the operating state has reached the state in which the projected image has been manipulated. Start video playback. Furthermore, the processing of this application example will be described in detail with reference to FIG. FIG. 12 is a diagram for explaining a processing example of the information processing system according to the seventh application example of the present embodiment.

  In the example of FIG. 12, consider a case where the user causes the information processing apparatus 100-1 to reproduce a desired video. Note that description of processing that is substantially the same as the processing described above is omitted.

  First, the information processing apparatus 100-1 determines whether a guide information projection instruction related to video reproduction has been acquired based on a user's voice input. For example, the control unit 106 obtains text information related to the text “I want to watch an XYZ program” from the server 200-1 using the audio information input from the audio input unit 102 as shown at time t32 in FIG. . Then, the control unit 106 determines that the projection instruction of “video playback button” has been acquired from the user instruction “video playback of XYZ program” interpreted based on the text information.

  When an instruction to project guide information related to video reproduction is acquired, the information processing apparatus 100-1 projects the guide information onto the viewing area of the user. For example, the control unit 106 determines an area where the user can view the video. Then, the control unit 106 causes the projection unit 108 to project the video reproduction button G10 onto the determined viewing area as shown at time t33 in FIG.

  Next, the information processing apparatus 100-1 sets a state where the user's touch operation is performed on the projected guide information as a target state. Specifically, for example, the control unit 106 sets a state in which the video playback button G10 is touched by the user as a target state.

  Next, the information processing apparatus 100-1 determines whether the operation state has reached the set target state. For example, the control unit 106 determines whether the projected video playback button G10 has been touched by the user. When the user's hand and the video playback button G10 are overlapped as shown at time t34 in FIG. 12, the control unit 106 determines that the video playback button G10 has been touched by the user.

  When it is determined that the user's operation state has reached the set target state, the information processing apparatus 100-1 executes a process corresponding to the projection instruction based on an operation related to the target state. For example, when the video playback button G10 is touched, the control unit 106 ends the projection of the video playback button G10 as shown at time t35 in FIG. Next, the control unit 106 starts projecting the video of the XYZ program corresponding to the projection instruction of “video playback of XYZ program”. When the user touches the video of the XYZ program, the control unit 106 stops the playback of the XYZ program and projects the video playback button G10 again as shown at time t36 in FIG.

  Thereby, the user can view the video without hesitation in operation.

<2-4. Summary of First Embodiment>
As described above, according to the first embodiment of the present disclosure, the information processing apparatus 100-1 uses the guide information that guides the motion of the motion subject requested by the motion subject as an instruction to project the guide information. Accordingly, when a state in which the operation guided by the guide information is performed by the subject is recognized, a process based on the state is performed. Conventionally, when the operation for interaction becomes complicated, it becomes difficult for the operation subject to perform the operation, and the interaction may fail. However, according to the present embodiment, it is possible to prevent the user from getting lost in the operation or performing an incorrect operation by inducing the operation related to the interaction. Therefore, the interaction between the operation subject and the apparatus can be made smoother. Further, when the action subject is a person, the correct action can be intuitively understood by guiding the action. Therefore, it is possible to reduce the stress of the user in the interaction. Further, the guide information is projected onto an object that can be visually recognized by a plurality of moving subjects, so that the plurality of moving subjects can share the guide information. Therefore, a plurality of operation subjects can cooperate to perform the operation, and the operation can be speeded up or made efficient.

  Moreover, the process based on the said state includes the change of the projection content. Therefore, the user can automatically change the projection content by operating according to the guide information. Therefore, it is possible to reduce the trouble of changing the projection contents.

  In addition, the change of the projection content includes changing the projection target to guide information that guides the next operation of the operation guided by the projected guide information. Here, if all of the plurality of guide information is projected at once, the guide information projected by the projection destination is filled, and visibility may be lowered. In addition, if all of a plurality of guide information are projected at a time, the viewing user may be confused. On the other hand, according to the above-described projection content changing process, only part of the guide information is projected, and thus visibility can be maintained and user confusion can be suppressed. Further, since the next guide information is projected in accordance with the end of the user's operation, the user can intuitively understand the operation order. Therefore, usability can be maintained even when there is a plurality of guide information.

  The guide information includes only information for inducing a single action to the action subject. For this reason, the operation subject can easily grasp the guided operation. Therefore, it is possible to suppress a possibility that an operation different from the operation in which the operation subject is guided is erroneously performed.

  Further, the process based on the state includes a process corresponding to the projection instruction. Therefore, the user can cause the information processing apparatus 100-1 to perform a desired process by operating according to the guide information. Therefore, it is possible to suppress the possibility of erroneous operation for executing the desired process.

  The guide information includes information indicating a position to be operated by the operating subject. For this reason, the accuracy of the operation performed can be improved by guiding the position of the operation. Therefore, it is possible to suppress the processing based on the operation from being performed due to the displacement of the operation position, and it is possible to suppress discomfort such as user irritation. In other words, it is possible to cause the information processing apparatus 100-1 to smoothly execute a desired process. In addition, the correct operation position (for example, the arrangement position) can be presented to the user. Therefore, the user can learn the position of the correct operation.

  The guide information includes information indicating the order in which the operation subject operates. For this reason, the accuracy of the operation to be performed can be improved by inducing the order of the operations. Accordingly, it is possible to prevent the processing based on the operation from being performed due to an error in the operation order, and it is possible to suppress discomfort such as user irritation. In other words, it is possible to cause the information processing apparatus 100-1 to smoothly execute a desired process. In addition, the correct operation order (for example, the writing order of characters) can be presented to the user. Therefore, the user can learn the correct operation order.

  The guide information is projected according to the shape of the object existing in the projection area. For this reason, the operation | movement according to the shape of the object can be induced | guided | derived. Therefore, it is possible to project the guide information more accurately, and it is possible to prevent the operation subject from performing an incorrect operation.

  Moreover, the said operation | movement contains operation with respect to the said guide information currently projected. For this reason, the operation subject can operate the process corresponding to the projection content or the projection instruction by operating on the projected image. Therefore, it is possible to intuitively operate the processing corresponding to the projection content or the projection instruction. In addition, by performing the processing according to the operation, the operation subject can concentrate on the desired operation.

  The state is recognized based on image information relating to an image in which the operation result of the operation subject is reflected. Therefore, the information processing apparatus 100-1 can recognize the operation state without the user inputting the operation state to the information processing apparatus 100-1. Therefore, it is possible to reduce the labor of the user.

  Moreover, the recognition of the state is performed by an external device of the information processing apparatus 100-1, and the recognition result is provided from the external device. For this reason, the process performed in the information processing apparatus 100-1 can be reduced. Therefore, the configuration of the information processing apparatus 100-1 can be simplified, the processing load can be reduced, and the processing speed can be increased.

  Information for projecting the guide information is provided from a device external to the information processing device 100-1. For this reason, compared with the case where guide information or material information is memorize | stored in the information processing apparatus 100-1, the memory capacity required for the information processing apparatus 100-1 can be reduced. Therefore, it is possible to reduce the cost of the information processing apparatus 100-1.

  Further, the projection instruction is input with the voice of the operation subject. For this reason, even when the user cannot release his / her hand, a projection instruction can be issued. Therefore, the use opportunity of interaction can be increased and the convenience for the user can be improved.

<2-5. Modification>
Heretofore, the first embodiment of the present disclosure has been described. In addition, this embodiment is not limited to the above-mentioned example. Below, the modification of this embodiment is demonstrated.

  As a modified example of the present embodiment, the information processing apparatus 100-1 may perform an operation for generating a sound based on guide information. Specifically, the control unit 106 causes the projection unit 108 to project guide information for guiding a motion generating sound, and when the target state is recognized based on the sound information related to the sound emitted by the operation subject, Processing based on the target state is executed. Furthermore, the process of this modification is demonstrated with reference to FIG. FIG. 13 is a diagram for explaining a processing example of the information processing system according to the modification of the present embodiment.

  In the example of FIG. 13, a case where a user who plays a musical instrument causes the information processing apparatus 100-1 to present a score is considered. Note that description of processing that is substantially the same as the processing described above is omitted.

  First, the information processing apparatus 100-1 determines whether an instruction to project guide information related to a score is acquired based on a user's voice input. For example, as shown at time t <b> 37 in FIG. 13, the control unit 106 obtains text information related to the text “Show the score” from the server 200-1 using the audio information input from the audio input unit 102. . Then, the control unit 106 determines that the instruction to project “score” has been acquired from the user's instruction “presentation of score” interpreted based on the text information.

  When an instruction to project guide information related to the score is acquired, the information processing apparatus 100-1 projects the guide information onto the viewing area of the user. For example, the control unit 106 determines the viewing area, and causes the projection unit 108 to project the score G11A to the determined viewing area as shown at time t38 in FIG.

  Next, the information processing apparatus 100-1 sets a state in which a sound according to guide information related to the score projected on the motion area is emitted as a target state. Specifically, for example, the control unit 106 sets a state in which a melody sound indicated by the score G11A is played by the user as a target state.

  Next, the information processing apparatus 100-1 determines whether the operation state has reached the set target state. For example, the control unit 106 obtains the result of the voice recognition process using the voice information obtained by the sound collection of the voice input unit 102 from the server 200-1. When the result of the speech recognition processing indicates that the melody sound indicated by the score G11A is recognized as indicated at time t39 in FIG. 13, the control unit 106 has played the melody sound indicated by the score G11A. It is determined that the state has been reached. Note that the information processing apparatus 100-1 may present the recognized sound to the user. For example, as shown at time t39 in FIG. 11, the control unit 106 may emphasize (for example, add an underline) the recognized melody portion of the projected score G11A.

  When it is determined that the operation state has reached the set target state, the information processing apparatus 100-1 projects the next guide information. For example, when the melody indicated by the score G11A projected as shown at time t40 in FIG. 13 is played, the control unit 106 indicates the continuation of the melody indicated by the score G11A as indicated at time t41 in FIG. The score G11B is projected.

  Thus, according to the modification of the present embodiment, the operation of guiding the guide information (operation of the operation subject) includes an operation of generating sound. In general, when an operation subject (particularly a human) performs an operation that generates sound, it is often difficult to perform another operation. Therefore, according to the configuration of the present modification, a process corresponding to the desired operation can be performed only by performing a desired operation without directly performing an operation on the guide information or the information processing apparatus 100-1. (For example, a series of projections of guide information) can be performed by the information processing apparatus 100-1. Therefore, the user can concentrate on a desired action for generating a sound. In the above description, an example in which the sound generating operation is a musical instrument performance has been described. However, the sound generating operation may be other operations such as singing or tap dancing.

  Further, the operating state is recognized based on sound information related to the sound emitted by the operating subject. Therefore, the information processing apparatus 100-1 can recognize the operation state without the user consciously inputting the operation state to the information processing apparatus 100-1. Therefore, it is possible to reduce the labor of the user. Note that the above-described recognition processing based on the image information may be used in combination. In this case, it is possible to improve the accuracy and accuracy of the recognized operating state.

<3. Second Embodiment (Example of Error Processing)>
Heretofore, the first embodiment of the present disclosure has been described. Next, a second embodiment of the present disclosure will be described. In the second embodiment, when a state in which an operation different from the operation guided by the projected guide information is recognized, an error is notified.

<3-1. System configuration>
A functional configuration of the information processing system according to the second embodiment of the present disclosure will be described. In addition, the component of the function of the information processing system which concerns on 2nd Embodiment, and the function of the server 200-2 are substantially the same as 1st Embodiment. Hereinafter, only the information processing apparatus 100-2 will be described with respect to functions different from the functions according to the first embodiment.

(Control part)
The control unit 106 performs a process different from the process based on the target state when the action of the action subject is different from the action guided by the guide information. Specifically, the control unit 106 performs error notification as a process different from the process based on the target state.

  More specifically, the control unit 106 performs visual notification as an error notification. For example, when a user operation different from the guide information guiding operation is recognized, the control unit 106 additionally causes the projection unit 108 to project an image indicating that the user operation is different from the guide information guiding operation. . In addition to projecting an image, a visual effect such as a change in color or brightness or blinking may be applied to already projected guide information or background.

  More specifically, the control unit 106 may perform an audible notification as the error notification. For example, when a user operation different from the guide information guiding operation is recognized, the control unit 106 causes the audio output unit 110 to output a sound indicating that the user operation is different from the guide information guiding operation. Note that the notification content may only indicate that the user's operation is different from the operation guided by the guide information, and may further include a notification of a correction method for the operation guided by the guide information.

<3-2. System processing>
Next, processing of the information processing system according to the present embodiment will be described with reference to FIG. FIG. 14 is a flowchart conceptually showing an example of processing of the information processing apparatus 100-2 according to the present embodiment. Note that description of processing that is substantially the same as processing according to the first embodiment will be omitted.

  The information processing apparatus 100-2 determines whether the guide information projection instruction has been acquired (step S402). When it is determined that the guide information projection instruction has been acquired, the information processing apparatus 100-2 sets a target state according to the guide information ( Step S404).

  Next, the information processing apparatus 100-2 recognizes the operation state (step S406), and determines whether the recognized operation state is a target state (step S408). If it is determined that the recognized operation state is the target state, the information processing apparatus 100-2 executes processing based on the target state (step S410). Until the projection of the guide information is completed (step S412), the processes of steps S404 to S410 are repeated.

  On the other hand, when it is determined that the recognized operation state is not the target state, the information processing apparatus 100-2 determines whether the operation state is transitioning (step S414). Specifically, the control unit 106 determines whether or not the change in the user's action grasped from the image recognition result continues.

  If it is determined that the operating state is not transitioning, the information processing apparatus 100-2 executes an error notification process (step S416). Specifically, when it is determined that the change in the user's operation has stopped, the control unit 106 causes the audio output unit 110 to output that the user's operation is different from the operation guided by the guide information, or to that effect. Is projected onto the projection unit 108.

<3-3. Application example>
The information processing system according to this embodiment has been described above. The information processing system can be modified according to various cases based on the above-described embodiment, and can be applied to each case. Below, the 1st and 2nd application example is demonstrated.

(First application example)
As a first application example of this embodiment, an example of teaching how to arrange ingredients in cooking will be described. In this application example, the information processing apparatus 100-2 notifies an error using sound when the operation of the operation subject is different from the operation guided by the guide information. Specifically, the control unit 106 causes the projection unit 108 to project an image indicating the position where the ingredient is placed as guide information, and the ingredient is placed at a position different from the placement position indicated by the projected image. If it is determined that the operating state has been reached, the audio output unit 110 is made to output an error notification. Furthermore, the processing of this application example will be described in detail with reference to FIG. FIG. 15 is a diagram for explaining a processing example of the information processing system according to the first application example of the present embodiment.

  In the example of FIG. 15, a case is considered in which the user presents an example of cooking arrangement to the information processing apparatus 100-2 and the arrangement position is incorrect. Note that description of processing that is substantially the same as the processing described above is omitted.

  First, the information processing apparatus 100-2 determines whether a guide information projection instruction related to the arrangement of ingredients has been acquired based on the user's voice input (time t42 in FIG. 15), and the guide relating to the arrangement of ingredients. When the information projection instruction is acquired, the guide information (laying position G7A) is projected onto the placing area (time t43 in FIG. 15).

  Next, the information processing apparatus 100-2 sets, as a target state, a state in which an operation for placing the user's ingredients with respect to the guide information projected on the arrangement area is performed, and the target state in which the user's operation state is set It is determined whether it has reached.

  If it is determined that the user's operating state has not reached the set target state, the information processing apparatus 100-2 notifies the user of an error. For example, the control unit 106 determines whether an ingredient has been arranged at the projected placement position G7A. Here, as shown at time t44 in FIG. 15, when the ingredient 60A is arranged at a position different from the placement position G7A, the control unit 106 determines that the operation state is different from the target state. Note that the determination of the operation state is not finalized while the user moves the ingredient 60A. Next, as shown at time t <b> 45 in FIG. 15, the control unit 106 causes the audio output unit 110 to output a sound with the content “position is different”. The user who has received the notification by the voice notices that the arrangement of the ingredient 60A is wrong, and can rearrange the ingredient 60A at the projected placement position G7A as shown at time t46 in FIG. it can.

  Thereby, the user can notice that the arrangement position is incorrect. Therefore, it is possible to suppress the state where the next placement position is not projected for a long time due to an error in the placement position.

(Second application example)
As a second application example of the present embodiment, an example of teaching how to arrange ingredients in cooking will be described as in the first application example. In this application example, the information processing apparatus 100-2 changes the projected guide information instead of or together with the error notification when the operation of the operation subject is different from the operation guided by the guide information. Specifically, the control unit 106 records the action of the action subject while the guide information is projected. Then, when it is determined that the operation state has reached the state where the ingredients are arranged at a position different from the arrangement position indicated by the projected image, the control unit 106 determines whether or not the guide information can be changed. If it is determined that the guide information may be changed, the control unit 106 causes the projection unit 108 to project the guide information modified based on the motion performed by the operation subject in the past. Furthermore, the processing of this application example will be described in detail with reference to FIG. FIG. 16 is a diagram for explaining a processing example of the information processing system according to the second application example of the present embodiment.

  In the example of FIG. 16, as in FIG. 15, consider a case where the user causes the information processing apparatus 100-2 to present an example of cooking dishes and misplaces the arrangement position. Note that description of processing that is substantially the same as the processing described above is omitted.

  First, the information processing apparatus 100-2 determines whether an instruction to project guide information related to the arrangement of ingredients has been acquired based on the user's voice input (time t47 in FIG. 16), and the guide related to the arrangement of ingredients. When the information projection instruction is acquired, the guide information (laying position G7A) is projected onto the placing area (time t48 in FIG. 16).

  Next, the information processing apparatus 100-2 sets, as a target state, a state in which an operation for placing the user's ingredients with respect to the guide information projected on the arrangement area is performed, and the target state in which the user's operation state is set It is determined whether it has reached.

  If it is determined that the user's operating state has not reached the set target state, the information processing apparatus 100-2 notifies the user of an error and confirms whether the guide information can be changed. For example, as shown at time t49 in FIG. 16, when the ingredient 60A is disposed at a position different from the placement position G7A, the control unit 106 determines that the operation state is different from the target state. Then, as shown at time t49 in FIG. 16, the control unit 106 causes the audio output unit 110 to output the audio having the content “the position is different”. Further, the control unit 106 causes the audio output unit 110 to output a voice asking the user whether or not the guide information can be changed, “Do you want to change the arrangement?”.

  If it is determined that the guide information may be changed, the information processing apparatus 100-2 projects the guide information corrected based on the past operation history. For example, when there is an affirmative response to the question as to whether or not the guide information can be changed as shown at time t49 in FIG. 16, the control unit 106 has already arranged the placement position G7A based on the past placement position of the user. It changes to the position where 60A is arranged. Further, the control unit 106 also changes the arrangement of the placement position G7B next to the placement position G7A. Then, as shown at time t50 in FIG. 16, the control unit 106 causes the projection unit 108 to project the changed placement position G7B. As shown at time t51 in FIG. 16, the user places the ingredient 60B at the post-change placement position G7B.

  Thereby, when the arrangement position is different from the arrangement position to be presented, the arrangement position corrected according to the user can be presented again. Therefore, usability can be improved by respecting the user's intention.

  In the above description, the guide information is corrected based on the past operation history. However, the guide information may be corrected based on the current operation different from the operation guided by the guide information. In this case, by respecting the current operation, it is possible to give the user a flexible impression on the processing of the information processing apparatus 100-2.

<3-4. Summary of Second Embodiment>
Thus, according to the second embodiment of the present disclosure, the information processing apparatus 100-2 performs a process different from the process based on the target state when the operation of the operation subject is different from the operation guided by the guide information. For this reason, by performing processing according to the fact that the target state has not been reached, it is possible to urge the operating subject to move toward the target state. Accordingly, it is possible to suppress the target state from being unreached and to shorten the time taken to reach the target state.

  Further, processing different from the processing based on the target state includes error notification. For this reason, by notifying the operating subject that the target state has not been reached, it is possible to directly prompt the operating subject to move toward the target state. Accordingly, it is possible to more surely suppress the target state from being unreached and reduce the time taken to reach the target state.

  Further, the process different from the process based on the target state includes a change of the projected guide information. For this reason, the projected guide information can be changed to guide information adapted to the subject. Therefore, usability can be improved.

  Further, the change of the guide information includes a change of the projection target to the guide information corrected based on the action performed by the action subject in the past. Therefore, the guide information can be changed to guide information that matches the nature of the operation subject. Therefore, the guide information can be more reliably adapted to the operation subject.

  The guide information is changed based on whether or not the guide information can be changed. Here, when the guide information is automatically changed, there is a possibility that the user may feel uncomfortable or confused when the operation subject is the user. On the other hand, according to the above-described guide information change process, it is possible to confirm the change of the guide information with the user. Therefore, it is possible to suppress the possibility of giving the user an unpleasant feeling or confusing.

<4. Hardware configuration of information processing apparatus according to one embodiment>
The information processing apparatus 100 according to each embodiment of the present disclosure has been described above. The processing of the information processing apparatus 100 described above is realized by cooperation of software and hardware of the information processing apparatus 100 described below.

  FIG. 17 is an explanatory diagram illustrating a hardware configuration of the information processing apparatus 100 according to an embodiment of the present disclosure. As illustrated in FIG. 17, the information processing apparatus 100 includes a processor 132, a memory 134, a bridge 136, a bus 138, an observation device 140, an input device 142, an output device 144, a connection port 146, and a communication device 148.

(Processor)
The processor 132 functions as an arithmetic processing unit, and realizes the function of the control unit 106 in the information processing apparatus 100 in cooperation with various programs. The processor 132 operates various logical functions of the information processing apparatus 100 by executing a program stored in the memory 134 or another storage medium using the control circuit. For example, the processor 132 may be a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or a SoC (System-on-a-Chip).

(memory)
The memory 134 stores a program used by the processor 132 or an operation parameter. For example, the memory 134 includes a RAM (Random Access Memory), and temporarily stores a program used in the execution of the processor 132 or a parameter that changes as appropriate in the execution. The memory 134 includes a ROM (Read Only Memory), and the function of the storage unit is realized by the RAM and the ROM. Note that an external storage device may be used as a part of the memory 134 via the connection port 146 or the communication device 148.

  The processor 132 and the memory 134 are connected to each other by an internal bus including a CPU bus.

(Bridge and bus)
The bridge 136 connects the buses. Specifically, the bridge 136 includes an internal bus to which the processor 132 and the memory 134 are connected, and a bus 138 that connects the observation device 140, the input device 142, the output device 144, the connection port 146, and the communication device 148. Connecting.

(Observation equipment)
The observation device 140 observes the information processing apparatus 100 and its surroundings, and realizes the functions of the voice input unit 102 and the imaging unit 104. Specifically, the observation device 140 includes an audio sensor and a camera sensor. For example, a microphone may be included as the audio sensor, and an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) may be included as the camera sensor. Note that the observation device 140 may further include another sensor such as an acceleration sensor, an angular velocity sensor, or a GPS sensor.

(Input device)
The input device 142 is used for a user to operate the information processing apparatus 100 or input information to the information processing apparatus 100. For example, the input device 142 includes input means for a user to input information, an input control circuit that generates an input signal based on an input by the user, and outputs the input signal to the processor 132. The input means may be a mouse, a keyboard, a touch panel, a switch or a lever. A user of the information processing apparatus 100 can input various data or instruct a processing operation to the information processing apparatus 100 by operating the input device 142.

(Output device)
The output device 144 is used to notify the user of information, and realizes the functions of the projection unit 108 and the audio output unit 110. The output device 144 may include a projection device and an audio output device. For example, the output device 144 includes a projector and a device such as a speaker or headphones. The output device 144 may further include a display device such as a liquid crystal display (LCD) device or an organic light emitting diode (OLED) device. Further, the output device 144 may be a module that performs output to various devices.

(Connection port)
The connection port 146 is a port for directly connecting a device to the information processing apparatus 100. For example, the connection port 146 may be a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, or the like. The connection port 146 may be an RS-232C port, an optical audio terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) port, or the like. By connecting an external device to the connection port 146, data may be exchanged between the information processing apparatus 100 and the external device.

(Communication device)
The communication device 148 mediates communication between the information processing device 100 and the external device, and realizes the function of the communication unit 112. Specifically, the communication device 148 performs communication according to a wired communication method. For example, the communication device 148 performs wired communication such as signal line communication, wired WAN (Wide Area Network) or wired LAN (Local Area Network) communication. The communication device 148 may perform communication according to a wireless communication method. For example, the communication device 148 includes Bluetooth (registered trademark), NFC (Near Field Communication), wireless USB, or near field communication such as TransferJet (registered trademark), WCDMA (registered trademark) (Wideband Code Division Multiple Access), WiMAX. Wireless communication may be executed according to an arbitrary wireless communication method such as a cellular communication method such as (Registered Trademark), LTE (Long Term Evolution) or LTE-A, or a wireless LAN method such as Wi-Fi (Registered Trademark).

  Note that the information processing apparatus 100 may not have a part of the configuration described with reference to FIG. 17 or may have an additional configuration. In addition, a one-chip information processing module in which all or part of the configuration described with reference to FIG. 17 is integrated may be provided.

<5. Conclusion>
As described above, according to the first embodiment of the present disclosure, it is possible to suppress the user from getting lost in the operation or performing the wrong operation by inducing the operation related to the interaction. Therefore, the interaction between the operation subject and the apparatus can be made smoother. Further, when the action subject is a person, the correct action can be intuitively understood by guiding the action. Therefore, it is possible to reduce the stress of the user in the interaction.

  Further, according to the second embodiment of the present disclosure, by performing processing according to the fact that the target state has not been reached, it is possible to urge the operating subject to move toward the target state. Accordingly, it is possible to suppress the target state from being unreached and to shorten the time taken to reach the target state.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

  For example, in the above embodiment, the information processing apparatus 100 is used by an individual such as a consumer, but the present technology is not limited to such an example. Specifically, the information processing apparatus 100 may be used in business such as an office or a factory. For example, in the product assembly operation, the information processing apparatus 100 projects guide information indicating the installation location of parts.

  Moreover, although the said embodiment demonstrated the example which guide | induces the position of an operation | movement by guide information, the timing of an operation | movement may be induced | guided | derived. For example, guide information that guides the timing of adding seasonings in cooking may be projected.

  In the above-described embodiment, an example in which an error notification is performed when the operation state is different from the target state has been described. May be allowed. For example, when the frequency at which an incorrect operation is performed for the arrangement position G7A as shown at time t44 in FIG. 15 is higher than other induced arrangement positions, the arrangement position G7A is highlighted. As highlighting, there is a visual effect such as a change in color or luminance of the projected guide information or blinking. As a result, it is possible to reduce the possibility that the operation subject will perform an operation different from the operation guided by the guide information.

  Further, the effects described in the present specification are merely illustrative or exemplary and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

  In addition, the steps shown in the flowcharts of the above-described embodiments are executed in parallel or individually even if they are not necessarily processed in time series, as well as processes performed in time series in the order described. Including processing to be performed. Further, it goes without saying that the order can be appropriately changed even in the steps processed in time series.

  It is also possible to create a computer program for causing hardware built in the information processing apparatus 100 to exhibit functions equivalent to the functional configurations of the information processing apparatus 100 described above. A storage medium storing the computer program is also provided.

The following configurations also belong to the technical scope of the present disclosure.
(1)
A projection control unit that causes the projection unit to project guide information for guiding the operation of the operation subject that the operation subject requests guidance, according to a projection instruction of the guide information;
When a state in which the operation guided by the guide information is performed by the operation subject is recognized, a processing control unit that performs processing based on the state;
An information processing apparatus comprising:
(2)
The processing based on the state includes a change of projection content,
The information processing apparatus according to (1).
(3)
The change of the projection content includes changing the projection target to the guide information that guides the next operation of the operation guided by the projected guide information.
The information processing apparatus according to (2).
(4)
The guide information includes only information for inducing a single action to the action subject.
The information processing apparatus according to any one of (1) to (3).
(5)
The guide information includes information indicating a position to be operated by the operating subject.
The information processing apparatus according to any one of (1) to (4).
(6)
The guide information includes information indicating an order in which the operation subject operates.
The information processing apparatus according to any one of (1) to (5).
(7)
The operation includes an operation on the projected guide information.
The information processing apparatus according to any one of (1) to (6).
(8)
The operation includes an operation of generating sound,
The information processing apparatus according to any one of (1) to (7).
(9)
The state is recognized based on image information relating to an image in which the operation result of the operation subject is reflected.
The information processing apparatus according to any one of (1) to (8).
(10)
The state is recognized based on sound information related to a sound emitted by the operating subject.
The information processing apparatus according to any one of (1) to (9).
(11)
The recognition of the state is performed by a device external to the information processing device, and a recognition result is provided from the external device.
The information processing apparatus according to any one of (1) to (10).
(12)
Information for projecting the guide information is provided from an apparatus outside the information processing apparatus.
The information processing apparatus according to any one of (1) to (11).
(13)
The projection instruction is input by the voice of the operation subject.
The information processing apparatus according to any one of (1) to (12).
(14)
The process control unit performs a process different from the process based on the state when the operation of the operation subject is different from the operation guided by the guide information.
The information processing apparatus according to any one of (1) to (13).
(15)
The processing different from the processing based on the state includes an error notification.
The information processing apparatus according to (14).
(16)
The process different from the process based on the state includes a change of the projected guide information.
The information processing apparatus according to (14) or (15).
(17)
The change of the guide information includes a change to the guide information modified based on the action performed by the action subject in the past.
The information processing apparatus according to (16).
(18)
The change of the guide information is performed based on whether or not the guide information can be changed.
The information processing apparatus according to (16) or (17).
(19)
Using a processor
Projecting guide information for guiding the action of the action subject that the action subject requests guidance to the projection unit according to a projection instruction of the guide information;
When a state in which the operation guided by the guide information is performed by the operation subject is recognized, processing based on the state is performed.
An information processing method including:
(20)
A projection control function that causes the projection unit to project guide information for guiding the operation of the operation subject that the operation subject requests guidance, according to a projection instruction of the guide information;
When a state in which the operation guided by the guide information is performed by the operation subject is recognized, a processing control function for performing processing based on the state;
A program to make a computer realize.

DESCRIPTION OF SYMBOLS 100 Information processing apparatus 102 Voice input part 104 Imaging part 106 Control part 108 Projection part 110 Voice output part 112 Communication part 200 Server 202 Communication part 204 Voice recognition part 206 Image recognition part 208 Information provision part 210 Storage part

Claims (20)

A projection control unit that causes the projection unit to project guide information for guiding the operation of the operation subject that the operation subject requests guidance, according to a projection instruction of the guide information;
When a state in which the operation guided by the guide information is performed by the operation subject is recognized, a processing control unit that performs processing based on the state;
An information processing apparatus comprising: The processing based on the state includes a change of projection content,
The information processing apparatus according to claim 1. The change of the projection content includes changing the projection target to the guide information that guides the next operation of the operation guided by the projected guide information.
The information processing apparatus according to claim 2. The guide information includes only information for inducing a single action to the action subject.
The information processing apparatus according to claim 1. The guide information includes information indicating a position to be operated by the operating subject.
The information processing apparatus according to claim 1. The guide information includes information indicating an order in which the operation subject operates.
The information processing apparatus according to claim 1. The operation includes an operation on the projected guide information.
The information processing apparatus according to claim 1. The operation includes an operation of generating sound,
The information processing apparatus according to claim 1. The state is recognized based on image information relating to an image in which the operation result of the operation subject is reflected.
The information processing apparatus according to claim 1. The state is recognized based on sound information related to a sound emitted by the operating subject.
The information processing apparatus according to claim 1. The recognition of the state is performed by a device external to the information processing device, and a recognition result is provided from the external device.
The information processing apparatus according to claim 1. Information for projecting the guide information is provided from an apparatus outside the information processing apparatus.
The information processing apparatus according to claim 1. The projection instruction is input by the voice of the operation subject.
The information processing apparatus according to claim 1. The process control unit performs a process different from the process based on the state when the operation of the operation subject is different from the operation guided by the guide information.
The information processing apparatus according to claim 1. The processing different from the processing based on the state includes an error notification.
The information processing apparatus according to claim 14. The process different from the process based on the state includes a change of the projected guide information.
The information processing apparatus according to claim 14. The change of the guide information includes a change to the guide information modified based on the action performed by the action subject in the past.
The information processing apparatus according to claim 16. The change of the guide information is performed based on whether or not the guide information can be changed.
The information processing apparatus according to claim 16. Using a processor
Projecting guide information for guiding the action of the action subject that the action subject requests guidance to the projection unit according to a projection instruction of the guide information;
When a state in which the operation guided by the guide information is performed by the operation subject is recognized, processing based on the state is performed.
An information processing method including: A projection control function that causes the projection unit to project guide information for guiding the operation of the operation subject that the operation subject requests guidance, according to a projection instruction of the guide information;
When a state in which the operation guided by the guide information is performed by the operation subject is recognized, a processing control function for performing processing based on the state;
A program to make a computer realize.

Images